QRS Supplement Best Practices

Annotated CRFs

Topic

QRS FAQ

Annotation Placement

Format

23-Jun-2021: CRF annotations need to follow the Study Data Tabulation Model Metadata Submission Guidelines (SDTM-MSG) V2.0 example going forward. https://www.cdisc.org/standards/foundational/sdtmig Legacy Instrument CRF Annotations will be maintained; unless a change control is needed in the future and then the effort to revise will be made.

• 2021-06-23 QRS ** Agenda and Meeting Notes ** documents the QRS subteam moving forward with the SDTM-MSG V2.0 CRF annotations.

Draft guidance for CDISC created QRS instrument CRFs: (this will be updated as more CRFs are created)

• CDISC-created CRFs will have 3 columns, labeled "Item Number", "Item Description" , and "Item Response", no quotes, all bolded.
• Times New Roman 12 point font should be used for the item text as appropriate.
• Times New Roman 18 point font size should be used for the instrument name.
• The instrument's reference article should always be included at the end of the CRF.
• If the instrument uses multiple pages, the headers should be repeated on each page.
• ... (to be continued as additonal use cases are identified)

--EVAL/--EVALID

27-Oct-2021: Team decided that --EVAL/–EVALID variables should not be annotated on the CRF.

Response sets

ADQRS ADaM Supplements

Topic

ADQRS FAQ

ADaM Supplements

15-Jul-2015: The team reviewed the ADaM QRS teams concerns with having a single QRS Supplement to handle both the SDTM and ADaM dataset creation. It was agreed to have separate supplements for each. The Web page QRS Supplement table was reviewed and it was decided that it would be best to have separate rows within each QRS measure to indicate the SDTM and ADaM dataset creation supplement documents separately.

QRS SDTMIG Supplements

Topic

QRS FAQ

Comment Questions

18-Jan-2017: QRS COMMENT question implementation summary.

QRS Summary on How Comments, Descriptive or Explanatory text is handled

1. QRS measures that include a general COMMENTS question that encompasses general comments across the entire CRF (usually at the bottom of a CRF page) are stored in the COMMENTS (CO) domain. This will work whether the dataset consists of a single QRS measure or multiple QRS measures in determining the relationship of the comment to the QRS record:
2. The RDOMAIN variable with represent either the QS, FT, or RS domain.
3. The IDVAR and IDVARVAL variables represent the QRS measure in which the comment is requested as: IDVAR=--CAT and IDVARVAL=”the value of –CAT”, since the comment is associated with the QRS measure CRF. 
4. The COREF variable is not populated in the QRS comment representation.
5. The COEVAL variable is a permissible variable and is not populated if the –EVAL value is populated on the source QRS measure domain. It would be redundant to populate this field in this case.
6. The VISITNUM variable is permissible and is added to the CO domain file. The value is associated with the visit number in which the comment was made.
7. The CODTC variable is permissible and is used with value will equal to the –DTC variable value for the date of the comment in which the QRS measure is represented.

The ASEX-FEMALE and ASEX-MALE QS supplements can be referenced as examples on implementing this type of comment in a QRS measure.

1. QRS measures that include additional questions such as –TESTCD=DESCRIB or EXPLAIN, which are viewed as more details about a specific question on the CRF are represented as a new –TEST in the domain. This additional information is specific to a question and is considered additional data related to the question and not considered a general comment.

Copyright

27-Apr-2016: The TAUGs are using copyright status terms that don't necessarily agree with the terms used by this team. These should be harmonized. During the discussion, this team felt that our current use of "Not granted" is too broad as it includes both a "no" response (denied) and no response received (after ~3 requests). Suggestion was to add "No response received" to the TAUG list (Dana) and to re-categorize the QRS list on the web from "Not granted" to "Denied" or "No response received." The action item form the 4/27 meeting was reviewed to update the NOT GRANTED list of QRS measures with terms used for the TAUG QRS section to be more specific on why the measure was not granted permission. These will include: Denied and No Response Received in the revisions to this table.

1. Sponsors should always consult the published QRS Supplements for guidance on submitting derived information in a SDTMIG QRS domain. Derived variables in QRS are normally considered captured data. If sponsors operationally derive variable results, then the derived records that are submitted in a QRS domain should be flagged by --DRVFL.
   1. The following rules apply for “Total” type scores in QRS datasets.
      1. QRS subtotal, total, etc. scores listed on the CRF are considered as captured data and are included in the instrument’s controlled terminology.
      2. QRS subtotal, total, etc. scores not listed on the CRF, but documented in an associated instrument manual or reference paper are considered as captured data and are included in the instrument’s controlled terminology.
      3. QRS subtotal, total, etc. scores not listed on the CRF, but known to be included in eData by sponsors are considered as captured data and are included in the instrument’s controlled terminology. The QRS instrument’s CT is considered extensible for this case and the subtotal or total score should be requested to be added.
         1. Any imputations/calculations done to numeric “standardized responses” to produce the total score via transforming numeric “standardized responses” in any way would be done as ADaM derivations.
   2. The QRS instrument subtotal or total score, which is the sum of the numeric responses for an instrument, is populated in --ORRES, --STRESC and –STRESN. It is considered a captured subtotal or total score without any knowledge of the sponsor data management processes related to the score.
      1. If operationally derived by the sponsor, it is the sponsor's responsibility to set the --DRVFL flag based on their eCRF process to derive subtotal and total scores. An investigator derived score written on a CRF will be considered a captured score and not flagged. When subtotal and total scores are derived by the sponsor, the derived flag (--DRVFL) is set to Y. However, when the subtotal and total scores are received from a central provider or vendor, the value would go into --ORRES and --DRVFL would be null [See SDTMIG v3.3 Section 4.1.8.1, Origin Metadata for Variables https://wiki.cdisc.org/display/SDTMIG33/Origin+Metadata+for+Variables].

--EVAL/–EVALID not included in QRS domains/datasets

27-Oct-2021: History: SDTMIG v3.3 was published in November, 2018; reviews started a year earlier. When reviews for SDTMIG v3.3 were in progress, meetings had already been happening with the FDA about logically skipped items and evaluator for a long time, but it was too late to change anything in the SDTMIG and we didn't have a solution yet anyway.   I began draft recommendations for changing --EVAL in February, 2019. (There are QRS subteam notes from 2013 and 2014 where the QRS team was discussing the problems with --EVAL) There were ongoing discussions thru early 2020 regarding --EVAL. The earliest documented decision that I found stating we won't be using --EVAL for QRS is in the QRS Subteam meeting notes for 2020-04-01, but it may have appeared in earlier minutes as well. (The QRS subteam space has all minutes.)

Historically, if the value was not collected on the CRF, the SDS QRS supplements had an assumption as to what the value of --EVAL should be based on the CRF and whatever other information was had about the instrument. In practice, --EVAL has been used many, many ways, with several variations of planned and actual investigator, making it difficult to know what it truly means when you see a value in the variable and making pooling of data next to impossible.

In addition, evaluator is rarely collected on a CRF and more often than not, it's not called "evaluator". EPDS has "Administered/Reviewed by"; ADCS-ADL MCI has "Examiner initials"; DAD has "Respondent" and "Rater"; "DRS has "Name of Person Completing Form"; GOSE has "Respondent" and "Interviewer", etc. Of all of the instruments that CDISC has created QRS supplements for, there are less than 2 dozen instruments that collect administrative information, and I don't have even one example where it was called "evaluator".

In all of those datasets where there's a value for --EVAL, what is it really? interviewer? respondent? examiner? person completing form? and is it planned? or actual? and what is it really for those that have nothing on the form or in the instructions (when instructions actually exist)? Hopefully this all helps clarify the problem.

So now we only represent administrative information if it is explicitly collected on an instrument. No assumptions are made for any of this kind of data; if it's not on the CRF, it's not in the data. We have held several webinars and stated it often in QRS subteam meetings to try and spread the word.

The suppquals we are using now are a series of non-standard variables that represent collected administrator value, administration device, etc., and Steve is in the process of getting these added to the non-standard variable registry. For example, we have QSCOLAVL which is "Collected Administrator Value" and QSCOLRVL which is "Collected Respondent Value".

We're also in the process of developing a QX domain for SDTMIG v4.0, which will be a QRS reference domain.  We'll give an update on QX in future QRS Subteam meetings as it recently went thru CDISC Internal Review, but one of the use cases will be to show what the CRF has for preprinted administrator, preprinted respondent, etc. So if the QRS instrument has places for "Interviewer", "Respondent", etc, these values would be in QX so we know what the instrument has on it.

--ORRES

27-Oct-2021: Original results stored in QSORRES when greater than the 200 characters limit currently mandated by the FDA, and hence, CDISC: To accommodate this situation in QSORRES the QRS Subteam has decided to shorten the responses to accommodate the 200 character limit without removing the clear meaning of the result. The original and shortened values are summarized in the QRS Supplement. Section 4 of the QRS supplements will include the origianl and shortened repsonses for reference.

--ORRES

27-Oct-2021: QRS Subteam decided that QSORRES values should be in the same case as the printed results on the instrument itself. Public domain QRS instrument's repsones are in the process of being included in the CDISC controlled terminology. The QRS CT team approves this controlled terminology. This is a step in implementing the QRS instruments in the CDISC Library.

--ORRES

22-May-2013: Team discussed QSORRES rules around case sensitivity and Yes/No vs. Y/N. Our rule is that QSORRES should represent the result text exactly as presented on the instrument itself, including case, punctuation, and spelling. Exceptions are: text > 200 characters may be abbreviated; reference text that merely qualifies a result may be omitted (for example, use just “Very Mild” for “Very Mild: e.g., occasionally exhibits poor eye contact”). Should there be a case where result choices are not pre-printed on the instrument, in the reference, or in a supplemental “key”, then SDTM controlled terminology should be used (e.g. Y or N for Yes or No).

--ORRES

14-Aug-2013: For a question in the “check-all-that-apply” format, a separate test code will be assigned for each of the items in the checklist. For items that are checked, use “CHECKED” as the result; for items that are not checked, use “NOT CHECKED” as the result. If the checklist is not completed at all based on a certain response to an opening question, the checklist items are considered null and not submitted in SDTM. 

--ORRES

11-Sep-2013: The QRS Subteam reviewed the follow-up on QSORRES for the SDS instruments. Through various e-mail exchanges among with the QRS Subteam, the favored approach for QSORRES is to concatenate number and text for cases where text applies to more than one number. Examples: 1-Mildly, 2-Mildly. Action: This is the decision until a better solution is developed (e.g. metadata solution).

--ORRES

--ORRES

07-May-2014: Decision by team is to use any text for QSORRES if text is available, else use number. This routinely applies to Low/High reposnes on visual analog scales (VAS) items and numeric rating scale items. This is consistent also with feedback we received from FDA a couple of years ago (Amy Malla). Team also discussed and decided that SUPPQS should hold the endpoint values (until a better “metadata solution” is developed.

--STRESN

--STRESC

QRS Batteries

17-Sep-2014: QRS Battery Implementation was decided with the SDS team. It was decided to follow the simple rule to produce the CT lists for any instrument (including a summary or battery) that we have copyright permission for. Treat each instrument including a battery/summary as independent items and develop terminology specifically for that item. Do not consider syncing question from an individual QRS instrument with the QRS battery. Any relationships between questions can and should be described in the Define file and/or the SDTM Reviewers Guide. CAT values can be created whether copyright permission is held or not. So all instruments and batteries/summaries that have developed terminology should have a defined --CAT value also. In the case of a battery/summary where we don’t have copyright for the individual instrument --CAT values should also be defined for the individual instruments when we are aware of those. See the embedded Meeting minutes with the attachments explaining this decision. 2014-08-25 SDS Extended Leadership Team Meeting Notes 2014-09-08 SDSLT Meeting Notes

QRS Derivation

05-Dec-2018 update to 30-Apr-2014: The QRS derivation continually surfaces with the QS Subteam. The previous QRS Subteam and terminology team decision was that if a total or subtotal score is not listed on the CRF it is not included in the QRS CT and not represented in SDTM. Many instruments have totals and subtotals described in their documentation, user manuals, or they are supplied to sponsors as part of eData. Additional derivations may also be provided to be part of the ADaM analysis representation of the data with CT to enable analysis and further data integration. Based on these known reasons, the QRS Subteam has agreed on the following rules for “Total” type scores in QRS datasets.

1. QRS subtotal, total, etc. scores listed on the CRF are considered as captured data and are included in the instrument’s controlled terminology.
2. QRS subtotal, total, etc. scores not listed on the CRF, but documented in an associated instrument manual or reference paper are considered as captured data and are included in the instrument’s controlled terminology.
3. QRS subtotal, total, etc. scores not listed on the CRF, but known that they can be included in eData by sponsors are considered as captured data and are included in the instrument’s controlled terminology.

QRS Derivation

05-Dec-2018 update to 27-May-2015: subtotal and Total scores represented in QRS domains are considered as capture data. There is no way for a QRS implementer to determine if the --DRVFL flag should be set based on an example CRF, without knowing the sponsor's data management processes. This is the statement for inclusion in supplements and the SDTMIG QS/FT/RS domain documents:

 The “QRS short name” total score, which is the sum of the points for each of the measures, is populated in --ORRES, --STRESC and --STRESN. Since the total score is (listed on the CRF or documented in the reference document or documented in the user manual or provided as eData to sponsors), it is considered a captured total score without any knowledge of the sponsor data management processes related to the score.

1. 1. If operationally derived by the sponsor, it is the sponsor's responsibility to set the --DRVFL flag based on their eCRF process to derive subtotals and total scores. An investigator derived score written on a CRF will be considered a captured score and not flagged. When subtotal and total scores are derived by the sponsor, the derived flag (--DRVFL) is set to Y. However, when the subtotal and total scores are received from a central provider or vendor, the value would go into --ORRES and --DRVFL would be null [See SDTM Section 4.1.8.1, Origin Metadata for Variables https://wiki.cdisc.org/display/SDTMIG33/Origin+Metadata+for+Variables].

This statement will be included in any derived data assumption going forward and also added to the assumption in the QS/FT/RS domain documents for the next SDTMIG release.

QRS Naming Convention

25-May-2016: The agreement reached was that we will not use the CC abbreviation when stating clinical classifications in general to avoid confusion and we will use the names of Questionnaires, Functional Tests and Clinical Classifications supplements. We also will retain the plural version of Clinical Classifications;

--SCAT

22-May-2013: Team discussed QSSCAT case sensitivity. We propose use of upper case always for consistency with QSCAT and SDTM guidance

--SCAT

23-Oct-2013: Use the QSCAT value in the supplement even if it differs from the commonly known instrument abbreviation. The commonly known instrument abbreviation will be contained in the QSCAT Definition in Terminology. Make a note in the supplement if applicable.

Supplemental Qualifiers

29-Jun-2016: :If a QRS supplemental qualifier applies to all of the QRS --TESTCDs in a category, the IDVAR and IDVARVAL should be the --CAT or --CAT values. If it does not apply to all tests at this level they should have the specific QRS --TESTCDs to which it applies. Sponsor discretion needs to take precedence in all other cases.

--TESTCD (Missing Responses)

21-Apr-2021: 

Missing responses: Records are created in qs.xpt for every item on the instrument:

For items with no data, QSORRES, QSSTRESC, and QSSTRESN are all missing and QSSTAT = "NOT DONE". If the reason is known then that reason is represented in QSREASND (e.g., QSREASND = "LOGICALLY SKIPPED ITEM" or QSREASND = "PREFER NOT TO ANSWER"). If the reason is unknown, then QSSTAT = "NOT DONE" and QSREASND is missing.

QRS instruments with skipped items: The QRS skipped items will apply to all QRS domains (FT, QS, RS) as needed.

Some items on the QRS instruments may be logically skipped per the instrument instructions. (Ex. Only one of multiple items needs to be completed. The others will be treated as a logically skipped item.) Some items are only completed when the response to a specific item a specific response. A record is created in the QRS dataset (--.xpt) for all items. When an item is considered a logically skipped item, it is represented as follows:

RSSTAT = "NOT DONE".

RSREASND = "LOGICALLY SKIPPED ITEM".

RSORRES, RSSTRESC, and RSSTRESN are set to null (missing).

--TESTCD

08-Jan- 2014: If consecutive QSTESTCDs fall within a QSSCAT, list the QSTESTCDs in the format 1 – n in the supplement mapping strategy section. If multiple, non-consecutive QSTESTCDs fall within a QSSCAT, list each individually.

Team decided to hold off on this until we see more cases and what we prefer. One issue with listing QS test codes in ranges like 1 – n is that each test code is then not searchable. But is the intent really that these supplements should be searchable?

Version History

19-Feb-2014: For updated supplement versions, provide a summary of changes in the Version History of the revised supplement

Version History

18-May-2016: A suggestion was made to add the SDTM version in which the QRS measure was developed in the "Notes to Readers" section. After discussion and review of the QRS web page that describes the use of the RS domain, it was decided not to add this item. Based on the evolution of the standards and revised mapping and/or new variable decisions, the maintenance of all QRS historic documents for anything but major change controls will be necessary. We want to foster the approach to follow current thought processes on the QRS mapping for historic document that might be may not have used new variable approved since they were developed.