QRS Best Practices Categorized
Annotated CRFs | |
Topic | QRS FAQ |
Annotation Placement | 27-Oct-2021: If the scores and reference text are present in each response, annotate the scores in the first question only with QSSTRESC/QSSTRESN. Draw a red or blue box around the scores and place the annotation next to it. Use of the same colored arrows are also useful. Here is an example annotated CRF: published BPRS-A aCRF |
Format | 23-Jun-2021: CRF annotations need to follow the Study Data Tabulation Model Metadata Submission Guidelines (SDTM-MSG) V2.0 example going forward. https://www.cdisc.org/standards/foundational/sdtmig Legacy Instrument CRF Annotations will be maintained; unless a change control is needed in the future and then the effort to revise will be made.
Draft guidance for CDISC created QRS instrument CRFs: (this will be updated as more CRFs are created)
CDISC created CRF Template: SDS QRS CRF Template.docx |
--EVAL/--EVALID | 27-Oct-2021: Team decided that --EVAL/–EVALID variables should not be annotated on the CRF. (See the explanation below in the QRS supplement section) |
Response sets | 26-Jun-2013: Team made these rules for annotating the CRF and these should be added to the FAQ document:
|
Footenote the instrument's reference document | 2023-01-09: It was mentioned during previous reviews of CDISC created CRF's the a best pracitce is to include the reference document as part of the footer on all CDISC created CRFs for future reference. |
ADQRS ADaM Supplements | |
Topic | ADQRS FAQ |
ADaM Supplements | 15-Jul-2015: The team reviewed the ADaM QRS teams concerns with having a single QRS Supplement to handle both the SDTM and ADaM dataset creation. It was agreed to have separate supplements for each. The Web page QRS Supplement table was reviewed and it was decided that it would be best to have separate rows within each QRS measure to indicate the SDTM and ADaM dataset creation supplement documents separately. |
QRS SDTMIG Supplements | |
Topic | QRS FAQ |
WIKI QRS Supplement page namimg convention | 2023-03-24: A new CDISC WIKI policy was implemented in March 2023 that performs automatic deletion of copies of pages over 5 years old. This action applies only to copies of pages in the version history that are 5+ years old and to versioned copies of attachments that are 5+ years old. Current, versions of pages and files is unaffected. Based on this, the QRS Supplement page names need to include the supplement version number to avoid deletion of the pages in change control with a differnt version number in 5 years. Example: Rutgeerts Score (RUTGEERTS SCORE) Supp V1.0 |
Comment Questions | 18-Jan-2017: QRS COMMENT question implementation summary. QRS Summary on How Comments, Descriptive or Explanatory text is handled
The ASEX-FEMALE and ASEX-MALE QS supplements can be referenced as examples on implementing this type of comment in a QRS measure.
|
Copyright | 27-Apr-2016: The TAUGs are using copyright status terms that don't necessarily agree with the terms used by this team. These should be harmonized. During the discussion, this team felt that our current use of "Not granted" is too broad as it includes both a "no" response (denied) and no response received (after ~3 requests). Suggestion was to add "No response received" to the TAUG list (Dana) and to re-categorize the QRS list on the web from "Not granted" to "Denied" or "No response received." The action item form the 4/27 meeting was reviewed to update the NOT GRANTED list of QRS measures with terms used for the TAUG QRS section to be more specific on why the measure was not granted permission. These will include: Denied and No Response Received in the revisions to this table. |
--EVAL/–EVALID not included in QRS domains/datasets | 27-Oct-2021: History: SDTMIG v3.3 was published in November, 2018; reviews started a year earlier. When reviews for SDTMIG v3.3 were in progress, meetings had already been happening with the FDA about logically skipped items and evaluator for a long time, but it was too late to change anything in the SDTMIG and we didn't have a solution yet anyway. Dana Booth began draft recommendations for changing --EVAL in February, 2019. (There are QRS subteam notes from 2013 and 2014 where the QRS team was discussing the problems with --EVAL) There were ongoing discussions thru early 2020 regarding --EVAL. The earliest documented decision found stating we won't be using --EVAL for QRS is in the QRS Subteam meeting notes for 2020-04-01, but it may have appeared in earlier minutes as well. (The QRS subteam space has all minutes.) Historically, if the value was not collected on the CRF, the SDS QRS supplements had an assumption as to what the value of --EVAL should be based on the CRF and whatever other information was had about the instrument. In practice, --EVAL has been used many, many ways, with several variations of planned and actual investigator, making it difficult to know what it truly means when you see a value in the variable and making pooling of data next to impossible. In addition, evaluator is rarely collected on a CRF and more often than not, it's not called "evaluator". EPDS has "Administered/Reviewed by"; ADCS-ADL MCI has "Examiner initials"; DAD has "Respondent" and "Rater"; "DRS has "Name of Person Completing Form"; GOSE has "Respondent" and "Interviewer", etc. Of all of the instruments that CDISC has created QRS supplements for, there are less than 2 dozen instruments that collect administrative information, and don't have even one example where it was called "evaluator". In all of those datasets where there's a value for --EVAL, what is it really? interviewer? respondent? examiner? person completing form? and is it planned? or actual? and what is it really for those that have nothing on the form or in the instructions (when instructions actually exist)? Hopefully this all helps clarify the problem. So now we only represent administrative information if it is explicitly collected on an instrument. No assumptions are made for any of this kind of data; if it's not on the CRF, it's not in the data. We have held several webinars and stated it often in QRS subteam meetings to try and spread the word. The suppquals we are using now are a series of non-standard variables that represent collected administrator value, administration device, etc., and Steve Kopko has added these to the non-standard variable registry. For example, we have QSCOLAVL which is "Collected Administrator Value" and QSCOLRVL which is "Collected Respondent Value". We're also in the process of developing a QX domain for SDTMIG v4.0, which will be a QRS reference domain. We'll give an update on QX in future QRS Subteam meetings as it recently went thru CDISC Internal Review, but one of the use cases will be to show what the CRF has for preprinted administrator, preprinted respondent, etc. So if the QRS instrument has places for "Interviewer", "Respondent", etc, these values would be in QX so we know what the instrument has on it. |
General Visual Analog Scales (VAS) and General Numeric rating Scales (NRS) | 2018-03-26: Historically clinical research has led to many variations and uses of “QRS like” instruments across companies. The CDISC definition of the of the questionnaire (QS) domain in SDTMIG is “A findings domain that contains data for named, stand-alone instruments designed to provide an assessment of a concept. Questionnaires have a defined standard structure, format, and content; consist of conceptually related items that are typically scored; and have documented methods for administration and analysis.” Many of sponsor’s general visual analog scale (VAS) and numeric rating scale (NRS) type instruments meet this definition and the QRS subteam has been approached many times seeking input on how they could be developed. Based on the definition above, the QRS subteam views each of these different general VAS or NRS instruments as separate QRS type instruments based on a specific topic, unless items can be grouped together under a specific concept/name of a QRS instrument. The QRS subteam decided, based on this variability and the inventory of existing standard QRS instruments to develop, that these general VAS or NRS instruments will not be developed as CDISC QRS standards. These types of unique sponsor general VAS and NRS instruments were reviewed with the SDSELT meeting on 3/26/18. https://wiki.cdisc.org/pages/viewpage.action?pageId=54289510&src=contextnavpagetreemode The subteam needs to draft some guidance on this for sponsors in the future, Sponsors may choose to define and create custom general VAS or NRS datasets based on the questionnaires (QS) domain. As more guidance on this topic is created, it will be posted on the QRS Best Practices WIKI page |
Hybrid QRS Instruments | 2016-09-14: In the review of QRS instruments to convert from one QRS domain to another QRS domain with the new clinical classification concept, a number of instruments were defined as a hybrid of a Patient-reported outcome (PRO) and a Clinician-reported outcome (ClinRO). In these instances the patient reported outcome is most important and the instrument will be considered a questionnaire concept and mapped to the QS domain. |
--LOC | 11-Dec-2020: The LOC variable is supported by the anatomical location CT codelist from which you can choose any LOC value out of the many terms. However, functional tests (FT), questionnaires (QS), or clinical classifications (RS) instruments are different in that if anatomical locations are pre-specified by an instrument, the LOC codelist should not be used because you cannot choose locations other than the ones described by the instrument. You are also not able to use existing anatomical location terms because they are created based on CDISC terminology business rules and conventions. They will NOT look like the wording used in a QRS instrument, thus modifying the QRS naming rule to create verbatim tests of the instrument items. (Note: the Pain Intensity (PI) supplement has examples on when the LOC variable can be used basedon this summary) |
--ORRES >200-Characters | 27-Oct-2021: Original results stored in QSORRES when greater than the 200 characters limit currently mandated by the FDA, and hence, CDISC: To accommodate this situation in QSORRES the QRS Subteam has decided to shorten the responses to accommodate the 200 character limit without removing the clear meaning of the result. The original and shortened values are summarized in the QRS Supplement. Section 4 of the QRS supplements will include the origianl and shortened repsonses for reference. |
--ORRES case sensitivity | 27-Oct-2021: QRS Subteam decided that QSORRES values should be in the same case as the printed results on the instrument itself. Public domain QRS instrument's repsones are in the process of being included in the CDISC controlled terminology. The QRS CT team approves this controlled terminology. This is a step in implementing the QRS instruments in the CDISC Library. |
--ORRES as represented on CRF | 22-May-2013: Team discussed QSORRES rules around case sensitivity and Yes/No vs. Y/N. Our rule is that QSORRES should represent the result text exactly as presented on the instrument itself, including case, punctuation, and spelling. Exceptions are: text > 200 characters may be abbreviated; reference text that merely qualifies a result may be omitted (for example, use just “Very Mild” for “Very Mild: e.g., occasionally exhibits poor eye contact”). Should there be a case where |
--ORRES Check boxes | 14-Aug-2013: For a question in the “check-all-that-apply” format, a separate test code will be assigned for each of the items in the checklist. For items that are checked, use “CHECKED” as the result; for items that are not checked, use “NOT CHECKED” as the result. If the checklist is not completed at all based on a certain response to an opening question, the checklist items are considered null and not submitted in SDTM. |
--ORRES Non-QRS Data | 30-May-2023: Due to the increasing use of non-QRS responses (e.g., DM, VS, RE, etc.) represented on QRS instruments, the QRS subteam re-reviewed the handling of non-QRS data relationship with the QRS domains (FT, QS, RS) to better document this process. After additional questions and consideration by the QRS subteam and review with ISMT referencing the CDASHIG V2.0 section 4.2 CRF Best Design Practices and the SDTMIG V3.4 section 4.2.6 Grouping Variables and Categorization, the following guidelines were created. DM, SC, VS, etc. data in header or body of QRS CRFs that is not scored:
DM, VS, LB data are referenced as composite domain-specific data that is scored in clinical classification QRS instruments CRFs:
DM, VS, RE, etc. responses represented on QRS instrument’s CRFs and scored are included in the QRS domain: The following rationale determines the representation of scored non-QRS responses with the instrument’s QRS domain when used to evaluate the instrument’s responses.
If a QRS supplement exists for an instrument, it needs to be followed because it is likely the Regulatory Authorities will follow for the review of the QRS instrument’s supplement. This also ensures consistency across all use cases of the QRS instrument. If the QRS instrument does not have a published QRS supplement, is the QRS subteam recommends that the user follow this CDISC QRS best practice for consistency in using the instrument. |
--ORRES text applying to more than 1 numeric repsonses | 11-Sep-2013: The QRS Subteam reviewed the follow-up on QSORRES for the SDS instruments. Through various e-mail exchanges among with the QRS Subteam, the favored approach for QSORRES is to concatenate number and text for cases where text applies to more than one number. Examples: 1-Mildly, 2-Mildly. Action: This is the decision until a better solution is developed (e.g. metadata solution). |
--ORRES trailing punctuation | 13-Nov-2013: When trailing punctuation or extraneous words such as “or” are used only to make a list of response choices a readable sentence format, they should be excluded from QSORRES. For example, QSORRES = ‘Severe’ when ‘4’ is selected; QSORRES = ‘Very severe’ when 5 is selected. See below. |
--ORRES text responses | 07-May-2014: Decision by team is to use any text for QSORRES if text is available, else use number. This routinely applies to Low/High reposnes on visual analog scales (VAS) items and numeric rating scale items. This is consistent also with feedback we received from FDA a couple of years ago (Amy Malla). Team also discussed and decided that SUPPQS should hold the endpoint values (until a better “metadata solution” is developed. |
--ORRES --STRESN --STRESC Binary responses | 14-May-2022: QRS responses CT for items assessed on a binary type of response
Additional ORS item binary responses will follow the same pattern as they arise. |
--STRESN --STRESC Numeric responses | 13-Mar-2019: CRF numeric responses -There are cases where QRS CRFs do not include numeric “standardized responses” assigned to text responses (ex. mild, moderate, severe being 1, 2, 3). It is clearly in everyone's best interest to include the numeric “standardized responses” in the SDTMIG QRS dataset. This is only done when the numeric “standardized responses” are documented in the QRS CRF instructions, a user manual, website specific to the QRS instrument, or other reference document, that provides a clear explanation and rationale for providing them in the SDTMIG QRS dataset. |
QRS Batteries | 17-Sep-2014: QRS Battery Implementation was decided with the SDS team. It was decided to follow the simple rule to produce the CT lists for any instrument (including a summary or battery) that we have copyright permission for. Treat each instrument including a battery/summary as independent items and develop terminology specifically for that item. Do not consider syncing question from an individual QRS instrument with the QRS battery. Any relationships between questions can and should be described in the Define file and/or the SDTM Reviewers Guide. CAT values can be created whether copyright permission is held or not. So all instruments and batteries/summaries that have developed terminology should have a defined --CAT value also. In the case of a battery/summary where we don’t have copyright for the individual instrument --CAT values should also be defined for the individual instruments when we are aware of those. See the embedded Meeting minutes with the attachments explaining this decision. 2014-08-25 SDS Extended Leadership Team Meeting Notes 2014-09-08 SDSLT Meeting Notes |
QRS Derivation | 05-Dec-2018 update to 30-Apr-2014: The QRS derivation continually surfaces with the QS Subteam. The previous QRS Subteam and terminology team decision was that if a total or subtotal score is not listed on the CRF it is not included in the QRS CT and not represented in SDTM. Many instruments have totals and subtotals described in their documentation, user manuals, or they are supplied to sponsors as part of eData. Additional derivations may also be provided to be part of the ADaM analysis representation of the data with CT to enable analysis and further data integration. Based on these known reasons, the QRS Subteam has agreed on the following rules for “Total” type scores in QRS datasets.
05-Dec-2018 update to 27-May-2015: subtotal and Total scores represented in QRS domains are considered as capture data. The “QRS short name” total score, which is the sum of the points for each of the measures, is populated in --ORRES, --STRESC and --STRESN. Since the total score is (listed on the CRF or documented in the reference document or documented in the user manual or provided as eData to sponsors), it is considered a captured total score without any knowledge of the sponsor data management processes related to the score. These scores may be submitted in SDTM or derived in the Analysis Data Model (ADaM) per scoring instructions from [Insert copyright holder's name or other source.].
This statement will be included in |
QRS Naming Convention | 25-May-2016: The agreement reached was that we will not use the CC abbreviation when stating clinical classifications in general to avoid confusion and we will use the names of Questionnaires, Functional Tests and Clinical Classifications supplements. We also will retain the plural version of Clinical Classifications; |
--SCAT case sensitivity | 22-May-2013: Team discussed QSSCAT case sensitivity. We propose use of upper case always for consistency with QSCAT and SDTM guidance |
--SCAT controlled terminology | 23-Oct-2013: Use the QSCAT value in the supplement even if it differs from the commonly known instrument abbreviation. The commonly known instrument abbreviation will be contained in the QSCAT Definition in Terminology. Make a note in the supplement if applicable. |
Subtotal and Total Scores Rule | 13-Mar-2019 and updated 03-May-2023 to remove the reference to the --DRVFL variable:
|
| 29-Jun-2016 updated 04-May-2023 replacing supplemental qualifirs with non-stnadard variables: :If a QRS |
16-Jul-2021 updated 04-May-2023 replacing supplemental qualifirs with non-stnadard variables: GGG review of the draft QX domain agreed that new QRS NSV | |
--TESTCD (Missing Responses) | 21-Apr-2021 and revised 04-05-2023 for conditonally branched items: Missing responses: Records are created in qs.xpt for every item on the instrument: For items with no data, QSORRES, QSSTRESC, and QSSTRESN are all missing and QSSTAT = "NOT DONE". If the reason is known then that reason is represented in QSREASND (e.g., QRS instruments with skipped items: The QRS
Some items QRS instruments may be conditionally branched per the instrument instructions. A record is created inQRS dataset for all items. When an item is considered a conditionally branched item, it is represented as follows: When assigned conditionally branched responses are not documented in the instrument’s instructions or a user manual:
When assigned conditionally branched responses are documented in instructions or a user manual:
|
--TESTCD | 08-Jan- 2014: If consecutive QSTESTCDs fall within a QSSCAT, list the QSTESTCDs in the format 1 – n in the supplement mapping strategy section. If multiple, non-consecutive QSTESTCDs fall within a QSSCAT, list each individually. Team decided to hold off on this until we see more cases and what we prefer. One issue with listing QS test codes in ranges like 1 – n is that each test code is then not searchable. But is the intent really that these supplements should be searchable? |
Timing of QRS Instrument Diary Data | 12-Oct-2021:
|
Version History | 19-Feb-2014: For updated supplement versions, provide a summary of changes in the Version History of the revised supplement |
Version History | 18-May-2016: A suggestion was made to add the SDTM version in which the QRS measure was developed in the "Notes to Readers" section. After discussion and review of the QRS web page that describes the use of the RS domain, it was decided not to add this item. Based on the evolution of the standards and revised mapping and/or new variable decisions, the maintenance of all QRS historic documents for anything but major change controls will be necessary. We want to foster the approach to follow current thought processes on the QRS mapping for historic document that might be may not have used new variable approved since they were developed. |