Copy of QRS Supplement Best Practices

Annotated CRFs

Topic

QRS FAQ

Annotation Placement

Format

23-Jun-2021: CRF annotations need to follow the Study Data Tabulation Model Metadata Submission Guidelines (SDTM-MSG) V2.0 example going forward. https://www.cdisc.org/standards/foundational/sdtmig Legacy Questionnaire CRF Annotations will be maintained; unless a change control is needed in the future and then the effort to revise will be made.

• 2021-06-23 QRS ** Agenda and Meeting Notes ** documents the QRS subteam moving forward with the SDTM-MSG V2.0 CRF annotations.

Draft guidance for CDISC created QRS instrument CRFs: (this will be updated as more CRFs are created)

• CDISC-created CRFs will have 3 columns, labeled "Item Number", "Item Description" , and "Item Response", no quotes, all bolded.
• Times New Roman 12 point font should be used for the item text as appropriate.
• Times New Roman 18 point font size should be used for the instrument name.
• The instrument's reference article should always be included at the end of the CRF.
• If the instrument uses multiple pages, the headers should be repeated on each page.

Prior QRS Annotated CRF Guidance

15-May-2013: CRF annotations need to follow the Study Data Tabulation Model Metadata Submission Guidelines (SDTM-MSG) V1.0 example going forward. https://www.cdisc.org/standards/foundational/sdtmig Legacy Questionnaire CRF Annotations will be maintained; unless a change control is needed in the future and then the effort to revise will be made.

    i.     Per the Metadata example, QS=Questionnaires appears at the top of the page in Arial 18 point font, black color, bolded, and italicized.  It is enclosed in a black 1 point thickness text box with a light blue fill color.

   ii.     Variable annotations are in Arial 10 or 12 point font, red color, bolded, and italicized.  They are enclosed in a black 1 point thickness text box with a light blue fill color.  Be consistent within a page and use 12 point unless space is an issue and then use 10 pt.

Format

22-May-2013: Follow-up on CRF annotation rules: we should use “when” not “for” when annotating QSORRES, e.g. “QSORRES when QSTESTCD = MMSEA1”.

EVAL

15-May-2013: Team decided that --EVAL should not be annotated on the CRF if it is assigned and not collected.

Response sets

QRS ADaM Supplements

Topic

ADQRS FAQ

ADaM Supplements

15-Jul-2015: The team reviewed the ADaM QRS teams concerns with having a single QRS Supplement to handle both the SDTM and ADaM dataset creation. It was agreed to have separate supplements for each. The Web page QRS Supplement table was reviewed and it was decided that it would be best to have separate rows within each QRS measure to indicate the SDTM and ADaM dataset creation supplement documents separately.

QRS SDTM Supplements

Topic

QRS FAQ

Comment Questions

18-Jan-2017: QRS COMMENT question implementation summary.

QRS Summary on How Comments, Descriptive or Explanatory text is handled

1. QRS measures that include a general COMMENTS question that encompasses general comments across the entire CRF (usually at the bottom of a CRF page)  are stored in the COMMENTS (CO) domain.   This will work whether the dataset consists of a single QRS measure or multiple QRS measures in determining the relationship of the comment to the QRS record:
2. The RDOMAIN variable with represent either the QS, FT, or RS domain.
3. The IDVAR and IDVARVAL variables represent the QRS measure in which the comment is requested as: IDVAR=--CAT and IDVARVAL=”the value of –CAT”, since the comment is associated with the QRS measure CRF. 
4. The COREF variable is not populated in the QRS comment representation.
5. The COEVAL variable is a permissible variable and is not populated if the –EVAL value is populated on the source QRS measure domain.  It would be redundant to populate this field in this case.
6. The VISITNUM variable is permissible and is added to the CO domain file.  The value is associated with the visit number in which the comment was made.
7. The CODTC variable is permissible and is used with value will equal to the –DTC variable value for the date of the comment in which the QRS measure is represented.

The ASEX-FEMALE and ASEX-MALE QS supplements can be referenced as examples on implementing this type of comment in a QRS measure.

1. QRS measures that include additional questions such as –TESTCD=DESCRIB or EXPLAIN, which are viewed as more details about a specific question on the CRF are represented as a new –TEST in the domain.  This additional information is specific to a question and is considered additional data related to the question and not considered a general comment.

Copyright

27-Apr-2016: The TAUGs are using copyright status terms that don't necessarily agree with the terms used by this team.  These should be harmonized.  During the discussion, this team felt that our current use of "Not granted" is too broad as it includes both a "no" response (denied) and no response received (after ~3 requests).  Suggestion was to add "No response received" to the TAUG list (Dana) and to re-categorize the QRS list on the web from "Not granted" to "Denied" or "No response received."  The action item form the 4/27 meeting was reviewed to update the NOT GRANTED list of QRS measures with terms used for the TAUG QRS section to be more specific on why the measure was not granted permission.  These will include: Denied and  No Response Received in the revisions to this table.

1. Sponsors should always consult the published QRS Supplements for guidance on submitting derived information in a SDTMIG QRS domain. Derived variables in QRS are normally considered captured data. If sponsors operationally derive variable results, then the derived records that are submitted in a QRS domain should be flagged by --DRVFL.
   1. The following rules apply for “Total” type scores in QRS datasets.
      1. QRS subtotal, total, etc. scores listed on the CRF are considered as captured data and are included in the instrument’s controlled terminology.
      2. QRS subtotal, total, etc. scores not listed on the CRF, but documented in an associated instrument manual or reference paper are considered as captured data and are included in the instrument’s controlled terminology.
      3. QRS subtotal, total, etc. scores not listed on the CRF, but known to be included in eData by sponsors are considered as captured data and are included in the instrument’s controlled terminology. The QRS instrument’s CT is considered extensible for this case and the subtotal or total score should be requested to be added.
         1. Any imputations/calculations done to numeric “standardized responses” to produce the total score via transforming numeric “standardized responses” in any way would be done as ADaM derivations.
   2. The QRS instrument subtotal or total score, which is the sum of the numeric responses for an instrument, is populated in --ORRES, --STRESC and –STRESN. It is considered a captured subtotal or total score without any knowledge of the sponsor data management processes related to the score.
      1. If operationally derived by the sponsor, it is the sponsor's responsibility to set the --DRVFL flag based on their eCRF process to derive subtotal and total scores. An investigator derived score written on a CRF will be considered a captured score and not flagged. When subtotal and total scores are derived by the sponsor, the derived flag (--DRVFL) is set to Y. However, when the subtotal and total scores are received from a central provider or vendor, the value would go into --ORRES and --DRVFL would be null [See SDTMIG v3.3 Section 4.1.8.1, Origin Metadata for Variables https://wiki.cdisc.org/display/SDTMIG33/Origin+Metadata+for+Variables].

--EVALID

24-Jul-2013: QSEVALID:  Team agreed that placing reviewer initials in QSEVALID is an appropriate use of --EVALID per the SDTMIG, contrary to statement at SDS Team call that --EVALID is only for a “role”.  SDSeLT 23-Jun-14 meeting confirmed that is a valid use of QSEVALID and the Governance Team will update the definition in the SDTM 1.4 document for the next release.  There is also the issue of the need for additional variables for who provided vs who evaluated the response. This was brought to the SDSLT for review.

This question was part of the SDTMIG 3.3 Batch 3 Public Review.  The final resolution on 2-Mar-2017 to this is listed in the JIRA Issue https://jira.cdisc.org/browse/SDS-965.

 SDTM will not expand the use of EVALID to include rater's initials. An additional variable to hold the evaluator name or initials in the future. If this information is needed, a supplemental qualifier can be used for this information.

--ORRES

8-May-2013: Original results stored in QSORRES when greater than the 200 characters limit currently mandated by the FDA, and hence, CDISC: To accommodate this situation in QSORRES the QRS Sub-Team has decided to shorten the responses to accommodate the 200 character limit without removing the clear meaning of the result. The original and shortened values are summarized in the QRS Supplement.

--ORRES

15-May-2013: QRS Sub-Team decided that QSORRES values should be in the same case as the printed results on the questionnaire itself.  Although not specifically controlled through terminology, per SDTM IG Assumption 4.1.3.2, the questionnaire can be considered an external reference.

--ORRES

22-May-2013: Team discussed QSORRES rules around case sensitivity and Yes/No vs. Y/N.  Our rule is that QSORRES should represent the result text exactly as presented on the questionnaire itself, including case, punctuation, and spelling.  Exceptions are:  text > 200 characters may be abbreviated; reference text that merely qualifies a result may be omitted (for example, use just “Very Mild” for “Very Mild: e.g., occasionally exhibits poor eye contact”).  Should there be a case where result choices are not pre-printed on the questionnaire, in the reference, or in a supplemental “key”, then SDTM controlled terminology should be used (e.g. Y or N for Yes or No).

--ORRES

14-Aug-2013: For a question in the “check-all-that-apply” format, a separate test code will be assigned for each of the items in the checklist.  For items that are checked, use “CHECKED” as the result; for items that are not checked, use “NOT CHECKED” as the result.  If the checklist is not completed at all based on a certain response to an opening question, the checklist items are considered null and not submitted in SDTM. 

--ORRES

11-Sep-2013:  The QRS sub-team reviewed the follow-up on QSORRES for the SDS questionnaire.    Through various e-mail exchanges among with the QRS Sub-Team, the favored approach for QSORRES is to concatenate number and text for cases where text applies to more than one number.  Examples:  1-Mildly, 2-Mildly.  Action:  This is the decision until a better solution is developed (e.g. metadata solution).

--ORRES

--ORRES

07-May-2014: Decision by team is to use any text for QSORRES if text is available, else use number.  This is consistent also with feedback we received from FDA a couple of years ago (Amy Malla).  Team also discussed and decided that SUPPQS should hold the endpoint values (until a better “metadata solution” is developed.

--STRESN

--STRESC

QRS Batteries

17-Sep-2014: QRS Battery Implementation was decided with the SDS team.  It was decided to follow the simple rule to produce the CT lists for any instrument (including a summary or battery) that we have copyright permission for. Treat each instrument including a battery/summary as independent items and develop terminology specifically for that item. Do not consider syncing question from an individual QRS instrument with the QRS battery. Any relationships between questions can and should be described in the Define file and/or the SDTM Reviewers Guide. CAT values can be created whether copyright permission is held or not. So all instruments and batteries/summaries that have developed terminology should have a defined --CAT value also. In the case of a battery/summary where we don’t have copyright for the individual instrument --CAT values should also be defined for the individual instruments when we are aware of those.  See the embedded Meeting minutes with the attachments explaining this decision. 2014-08-25 SDS Extended Leadership Team Meeting Notes 2014-09-08 SDSLT Meeting Notes

QRS Derivation

05-Dec-2018 update to 30-Apr-2014: The QRS derivation continually surfaces with the QS Sub-Team.  The previous QRS sub-team and terminology team decision was that if a total or subtotal score is not listed on the CRF it is not included in the QRS CT and not represented in SDTM.  Many questionnaires have totals and subtotals described in their documentation, user manuals, or they are supplied to sponsors as part of eData. Additional derivations may also be provided to be part of the ADaM analysis representation of the data with CT to enable analysis and further data integration. Based on these known reasons, the QRS sub-team has agreed on the following rules for “Total” type scores in QRS datasets.

1. QRS subtotal, total, etc. scores listed on the CRF are considered as captured data and are included in the instrument’s controlled terminology.
2. QRS subtotal, total, etc. scores not listed on the CRF, but documented in an associated instrument manual or reference paper are considered as captured data and are included in the instrument’s controlled terminology.
3. QRS subtotal, total, etc. scores not listed on the CRF, but known that they can be included in eData by sponsors are considered as captured data and are included in the instrument’s controlled terminology.

QRS Derivation

05-Dec-2018 update to 27-May-2015: subtotal and Total scores represented in QRS domains are considered as capture data.   There is no way for a QRS implementer to determine if the --DRVFL flag should be set based on an example CRF, without knowing the sponsor's data management processes. This is the statement for inclusion in supplements and the SDTMIG QS/FT/RS domain documents:

 The “QRS short name” total score, which is the sum of the points for each of the measures, is populated in --ORRES, --STRESC and --STRESN.  Since the total score is (listed on the CRF or documented in the reference document or documented in the user manual or provided as eData to sponsors), it is considered a captured total score without any knowledge of the sponsor data management processes related to the score.

1. 1. If operationally derived by the sponsor, it is the sponsor's responsibility to set the --DRVFL flag based on their eCRF process to derive subtotals and total scores. An investigator derived score written on a CRF will be considered a captured score and not flagged. When subtotal and total scores are derived by the sponsor, the derived flag (--DRVFL) is set to Y. However, when the subtotal and total scores are received from a central provider or vendor, the value would go into --ORRES and --DRVFL would be null [See SDTM Section 4.1.8.1, Origin Metadata for Variables https://wiki.cdisc.org/display/SDTMIG33/Origin+Metadata+for+Variables].

This statement will be included in any derived data assumption going forward and also added to the assumption in the QS/FT/RS domain documents for the next SDTMIG release.

26-Jun-2013: Questionnaire metadata, such as the evaluation interval, are included now in the QRS domain datasets in lieu of an alternative overall metadata solution for QRS instruments.  Once such a solution is published, some questionnaire metadata elements, such as the evaluation interval, may be removed from the QRS domain datasets in the QRS Supplements.

QRS Naming Convention

25-May-2016:  The agreement reached was that we will not use the CC abbreviation when stating clinical classifications in general to avoid confusion with domain names and we will use the names of Questionnaires, Functional Tests and Clinical Classifications supplements.  We also will retain the plural version of Clinical Classifications;

--SCAT

22-May-2013: Team discussed QSSCAT case sensitivity.  We propose use of upper case always for consistency with QSCAT and SDTM guidance

--SCAT

23-Oct-2013: Use the QSCAT value in the supplement even if it differs from the commonly known questionnaire abbreviation.  The commonly known questionnaire abbreviation will be contained in the QSCAT Definition in Terminology.  Make a note in the supplement if applicable.

Supplemental Qualifiers

29-Jun-2016: :If a QRS supplemental qualifier applies to all of the QRS --TESTCDs in a category, the IDVAR and IDVARVAL should be the --CAT or --CAT values. If it does not apply to all tests at this level they should have the specific QRS --TESTCDs to which it applies. Sponsor discretion needs to take precedence in all other cases.

--TESTCD

19-Jun-2013: Per SDTM, the rule for assigning --TESTCD when an entire set of tests is not done is to use xxALL, where xx is the domain name.  So if an entire questionnaire is not done, QSTESTCD = ‘QSALL’.  (Note that QSCAT would be needed to differentiate among multiple QSALL values across questionnaires.

08-Jan- 2014: If consecutive QSTESTCDs fall within a QSSCAT, list the QSTESTCDs in the format 1 – n in the supplement mapping strategy section.  If multiple, non-consecutive QSTESTCDs fall within a QSSCAT, list each individually.

Team decided to hold off on this until we see more cases and what we prefer.  One issue with listing QS test codes in ranges like 1 – n is that each test code is then not searchable.  But is the intent really that these supplements should be searchable?

Version History

19-Feb-2014: For updated supplement versions, provide a summary of changes in the Version History of the revised supplement

Version History

18-May-2016: A suggestion was made to add the SDTM version in which the QRS measure was developed in the "Notes to Readers" section.  After discussion and review of the QRS web page that describes the use of the RS domain, it was decided not to add this item.  Based on the evolution of the standards and revised mapping and/or new variable decisions, the maintenance of all QRS historic documents for anything but major change controls will be necessary.  We want to foster the approach to follow current thought processes on the QRS mapping for historic document that might be may not have used new variable approved since they were developed.

15-May-2013: Question for SRC:  Should --EVAL = INVESTIGATOR be included in QS data set if this just represents an assigned value, i.e. not explicitly collected on the CRF?  SRC originally requested this to be included. Isn’t it understood that SDTM data in general falls under this category?

      Waiting for a SDS Team decision

21-May-2014: Team discussed the FTEVAL issue.  Suggestion from SDS ELT for QSEVAL was to always include the variable in the supplements, even if not collected.  Team not sure this should apply to FT since FT is not “subjective” data, which is the way --EVAL is currently defined in SDTM.  TUG is clearly just an objective measurement.  Decision was to remove FTEVAL and FTEVALID from supplement example and assumptions.  [

1. FTEVAL and FTEVALID will only be used if specified on a CRF or indicated in a user manual

Waiting for a SDS Team decision

23-Mar-2016: The question on how to deal with the skipped question on this CRF was posed by F. Wood, who suggested to create records for the logically skipped questions.  E. Kiely suggested we do this also in a prospective manner with related CDASH STAT & PERFORM variable values.  Although most QRS CRFs do not provide methods to handle missing responses, many sponsor implement them via their EDC systems.  Based on this ii was decide this needs further consideration.  Eanna will propose a review of the use of related CDASH QRS missing value approaches at a future CDASH team meeting.