2015-07-28-29 Terminology Summit Meeting Notes - Arlington, VA

• "First Principles" 
• Focus on new thinking, what are we trying to accomplish, examine assumptions, why do we have CDISC terminology
• 100.000+ terms in the NCIt
• Definitions
• Use available terminology - CTCAE, MedDra. But mainly build on what Sponsors are currently using?
• TEST codes are the driving factor. LOINC is the only on out there, but this is not suitable for our needs. But we should leverage it more than we do.
• Published assessment methods.
• Terminology is"reactive", the space is already defined.  
  • May be better to begin at the concept and then develop terminology. Code to questions rather than the answers. There is a lot of data gathered where the information is in the question. 
  • Other EVS/caDSR users are doing this, getting requests to create a question. But this depends on methodology and definitions. Striving to be generic that can be re-used. Need to be careful with the questions, there are slight differences for which there is no model to govern the content. Currently we are only capturing the provenance.
  • Hard to figure out where to put things, FA, procedure-interventions, Lab. Observation, observation test result.
• Is NLM putting in structured data? Lori thinks it must be questionnaires that they are working on in LOINC, UMLS.
• Potential new principle - justify why you are creating new terms.
  • 8 character test code required by FDA. 8 character rule is driven by the the need to use SAS transport files
  • 40 character rule for test names, driver is as above.
  • there are some commonly used LOINC test codes done by Phil Pochon
  • CDASH does not comply with the the 8 character rule
• New principle - use existing terminology, not just existing CDISC terms.
  • Caution - terminology might exist, but if its not bundled correctly its not useful. Taging is needed, etc.
  • Address - notion of making the terminology fit for use needs to be explored
  • CDISC needed terminology that fit in the structures that we defined (8 character, etc.) Connection to concepts was done by EVS. Slots that fit within CDISC structures.
  • Make better use of the C-codes

• CT Requirements Survey
  • Top 6 - focus on implementer needs and not creation of new terms

    • • 1-New Term Request Alerts and Updates

        • Keep the requester notified through every step of the review/implementation/rejection process

      • 2-Versioning

      • 3-Terminology Release Frequency

        • 4x a year, As Needed, 3x a year, 2x a year

      • 4-Streamline the Process

        • Turnaround time quicker, more efficient

      • 5-Subsets

      • 6-Root Variable

        • Connect terminology to the SDTM variable

• Key Issues
  
  • • Value sets (subsetting)
    • System integration - SHARE & NCI
    • Requesting C-codes for objects
    • UCUM annotations for UNIT/PKUNIT code lists
      • work going on with Joseph Aertz
      • Need a preferred representation for UCUM - should be possible to give a UCUM representation. At some point just go to UCUM and store in SHARE. Should be assessable, UCUM looks funny to folks
    • CDASH terminology focus
      • Subsets of existing SDTM code list is in Appendix of version 1.1
      • Should be removed - deprecated. These can be stored in SHARE as subsets.
    • Clinical Data Element Glossary publication - deprecation?
      • created by CDISC for CV (Acute Cardiovascular syndrome and Endpoints) - defined a subsets, these are not connected to any coddles.
      • Maybe being used by some pharma (Merck?)
• Deprecation Policy - need to review and include in the CDISC Standards Development Process documents
  • Be careful of clutter on the ftp. COA has been subsumed by QRS. has not been updated since 2013. We don't want it but someone must want it!?
  • Recommendation - review and update deprecation policy - notify users, so they can plan.
• caDSR - is undergoing change now, but uses a completely different platform.
• Issue - Because we have different teams working on different aspects, there is no clear process to bring these groups together??????????, approve and have the c-codes applied.
• Other Lab related topics
  • LOINC and CDISC Lab Model
    • Link to CDISC Lab test value to the actual analyze - where to plug-in the X.
      
      

See powerpoint

• Root variable - connect terminology to the SDTM variable
  • can easily be added to eSHARE
• Versioning and release frequency - go together
  • May need item level versioning
  • Look at MEDRA process - put out a provisional version that can be finalized later
    • company perspectives - validation process
    • EVS perspective - if you create a term you have created a term, once a C-code is attached its permanent. We don't want to create junk. Would have to retire them, etc.
    • We cannot not create garbage. Vetting one term is easier than many.

See slides

• In 2014 - 5574 new term requests that we managed by the team.
• Action: would be useful to know the kinds of companies that are submitting requests and how many have submitted multiple term requests. Erin to gather this information.
• Clear upward trend from 2013-14
• TA requests - 523 in 2013 and 1085 in 2014. Not as many as expected but mainly related to QS requests. Erin to add number of request submissions to the graph.
• Create Education program for new term requestors. Maybe include some warnings in the system to alert users and screening out some of the mistakes.
  • Create an on-line education program/certification for new term requestors, focus on how to submit terms and what type of information should be included in the request.
  • Impose a rule that only those that have gone through the training may enter new term requests
  • Training to include:
    • how to request a new term - do's and don'ts
    • what context to provide
    • timelines

• Review of CT Process Flow Diagram

  • Colapse the grade 1-3 to 1-2 - 1 = require administrative review and 2 = changes that need expert review. Review definitions.
    • synonym definition may not need public review
  • changes (diff file) to be added to the flow diagram, as not needed vetting but still calling that out
  • Need to review pull down list for Request Type; none, create new term, New code list, other....
  • Add note to the confirmation page that an email will be sent to the submitter.
  • An email goes to Jordan, Margaret, Erin and others at EVS. This happens every time the submitter hits the send button.
  • Jordan logs the request into JIRA by copy/paste. Not currently set up to bulk load. Need to assess the use of JIRA, needs to be open and protect PII (Personal Identifying Information). Consider moving the administrative part of this to CDISC and manage in JIRA, migrate the system to CDISC JIRA. Requests only need to go to EVS much later in the process.
  • Triaging of the new term requests. Unpack multiple term requests and collect metrics upload onto the CDISC portal. Report weekly to CDISC. 
  • Assigning a work item to a team. Paste request into the working teams document. Identify TA request as priority.

• Team Workflow

  • 11 active terminology teams - LB, PK, QRS, SEND, Oncology, ECG, Devices, Virology, CV, General, PGx
  • The teams have a list of Clinical Experts to consult for terminology questions. Can also leverage experts from the CFAST TA teams.
    • Perhaps SAC can be engaged to identify clinical experts
  • Bernice to clarify the quorum rules - at minimum two knowledgable team member plus CDISC or NCI-EVS rep must vote.
  • TEST and PARM - add the possibility of a long name. Can use the NCI preferred term column and rename to full name/description. Or just communicate to users that the NCI Preferred Term column can be used in this way.
  • Allow for the ability to include submission values?????????? - Note from EM: What does this mean?
  • Reasons for making changes to existing terms are noted in the working documents. There is no pull down list of canned responses because the reasoning is too broad for different term types.
  • Same for denials.
  • Hundreds of terminology files are in the portal. You can find terms by request numbers. The is one cumulative file that goes back to 2009 posted on cancer.gov CDISC page, so you can trace what happened to a request by request number. This probably go into SHARE. Searching across the portal is not easy.
  • Files for Public Review - plan a pilot for using SHARE to expedite the PR. But need to have the C-Codes.- Note from EM: Clarify what does this mean? Do you mean 'Need to include the request number with each item'?
  • Noted that the CT PR package contains items that may be managed differently within SHARE and TA development projects.
  • For rules documents, mapping files, denied request files, etc: Rather than include documents in the PR package, it was suggested that links be used as much as possible.
  • Feedback during the PR period - Sometimes receive 3-4 comments from a single person/company. Probably should establish some rules for what constitutes a minimal public review.
    • what constitutes a minimal number of comments
    • CT teams are using the comment tracker
    • Include an option where reviewers who have no comments can check a box that says that they reviewed and had not comments. Add this to the training program. Currently people just make comments that the terminology is ok.
    • Post PR comments and resolution on the CDISC website, like we do for all CDISC standards, add a blanket statement to the terminology webpage stating that this can be found on the comment tracker.
      • This system works well for CT. CT teams currently document all resolutions in the comment tracker due to the relatively small number of comments received.

• EVS QA Process

  • Post public review, other checks are done within the EVS system.
  • "CNEW" content is reviewed internally by additional EVS editors
  • Issues are sent to CDISC for resolution as needed
  • ~6-10 issues are removed from packages routinely

• QA Findings

• • relay back to the terminology team
  • Suggestion to change the name of the 'Final Public Review' document to 'Final Post Public Review' document
  • Example - 2 changes for 80 terms in one package - pretty good
  • A package many contain between 300-700 new terms. QS packages contain 200-300 terms but are currently being held from publication because decision needs to be made about CC.
  • QC off the diff file - comparing from the last publication. They do find things..seen as important part of QA process.
  • Final formatting changes - currently done manually, exploring automation..
  • Can't link the change request numbers to the changes files themselves so this part is done by hand
  • Rob then creates the ODM.XML, and other files and posts them to the ftp. EVS staff check all the links and informs CDISC (Bernice).
    
    

• Process timeline

• • 4 CT Packages per year
  • 27 week package cycleTeams keep working

• • Category 1 changes into as soon as they are corrected
  • Category 2 - changes go through the process

Other items updated quarterly

• • Changes - diff file
  • New term request tracker spreadsheet
  • List of available CT code lists
  • Requests denied spreadsheet
  • CT Mapping/Alignment Across Code lists - subsets beginning of concepts. Some of this is in SHARE now.
  • UCUM annotations for all new terms

Proposed a pilot to test using JIRA to track terminology and move to a more integrated approach with the TAUG development.

• • Since we don't have C-CODES, we could assign a screening CODE
  • Make the diff file program publicly available? Rob/Erin to investigate making the diff program publically available on the Cancer.gov CDISC web page.
    
    

  • UCUM

  • • CDISC can decide rules for a "preferred" UCUM code in cases where UCUM has multiple representations. So there is still work to be done, even of UCUM is used.
    • This does not solve the issue of mathematical synonomy and CDISC requiring one submission value per term/codelist.
    • Decision: Publish the CDISC preferred UCUM code. Need to figure how to publish in SHARE
    • Currently UCUM code is in the Concept code within NCIt. So you can query it.
      • Action item for Erin/Rob/Jordan: write up a short instruction on how to find a UCUM annotation for a CDISC unit
      • CDISC to come up with rules for what is the preferred UCUM representation is for a CDISC value in the unit codelist and make it explicit somewhere
    • This is a quick win, as this links to HCL.
    • Mathematical synonymy - recognize when different values are synonymous. The premise has to be that we are using the right unit for the right job.
    • • • Verbatim unit vs collected/reported unit, this goes back to traceability
        • C-Codes have a single CDISC representation (submission value) within a code list
        • Can we use synonyms to control this problem? Use numerical conversions
        • Create a CDASH variable for verbatim, original units. CDISC Action item: propose new CDASH variable - VRESU
      • Actions:
      • • Step 1 - Post a short and informative document on the CDISC web site with link - how to get to UCUM from EVS. If you can use the API it would be good. Action assigned to Bernice and Erin
        • Step 2 - Add to SHARE backlog - Evaluate how to get into and out of eSHARE.
        • Step 3 - Setting up the rules for picking alternative units. Establish preferred units.
        • Step 4 - Deal with mathematical synonymy

• CDASH

• Versioning

  • Convention will be to use the date provided for each package publication in ISO format.
  • Action: write a process/policy for versioning, post and communicate. Sam to tweet.
  • Make tool for generating diff files publicly available. Rob and Erin to check with EVS to see if they can make the tool public.
    
    
• Supplemental Qualifiers
  • Need to figure out the process for assigning a normative to a SUPPQUAL. Needs an extra degree of scrutiny.
  • SUPPQUALS are needed because we don't have variables. This is an area of need and requires SDTM governance.
  • Since TAUGs are standards maybe not best practice to assign SUPPQUALs without intent to make more official later on. 
  • Maybe need domain specific variables (DSV) and not create any SUPPQUALS.
  • Then it would go through SDTM governance as usual - then to terminology. May need help from the CT team with definitions.
  • Decision - C-Codes should not be assigned to Supplemental Qualifiers

Top Six issues

That are holding up the developing of terminology:

1. Verbatim collected Unit – add a new CDASH variable for verbatim unit
2. LOC, LAT, DIR – deprecate older versions of variables. Create new variables specific for specimen collection, test and result locations and qualifiers. Determine how to treat PORTOT and ANTREG.
3. Morphology/physiology - one or multiple domains?
4. CC - clinical classifications. Fold into RS?
5. Clarify content of ANMETH variable
6. SDTM variable definitions and domains

QS Supplements

Need to look at reducing the QS supplements, perhaps distill to just the meta and tables.

Observations

General Concern expressed about who is making the final decisions. Need to get quicker and more clear SDS team decisions. JIRA should track the decisions.

JIRA Review

•  Moving the new term request system from NCI/EVS to CDISC needs to be analyzed, this will take some time and will require resources
  • • setup meeting and determine what we would want the request system to look like in the CDISC JIRA
    • how would we link the request system to the working spreadsheet type document
      • • at the moment the request system and team working spreadsheet will be separate
    • need a prototype approach - needs to be defined in detail
    • cannot be done at this meeting, need a JIRA developer, probably a two month process
    • will need to make a proposal to CDISC upper management for developing a new term request system in CDISC
      • need support from Bron, Sam, Diane, and Rhonda to make this happen
      • might be common ground with developing the SDTM governance new variables (Sam)
      • including an external request form which includes security
      • spending money on developing new term request system  is less expensive than hiring a new employee, number of requests is increasing dramatically

• Requirements file by stakeholders
• TA streamline of terminology
  • transparency will help a great deal, they will be able to see exactly where the TA request is in the terminology process
  • going forward the terminology multiple request form out with the internal review and public review - maybe
    • maybe try with a newly started TA, cannot slow down the current process in the already working TAs
      • CV Imaging TA would be a good project to try sending out the terminology with internal review and public review
• would like to be able to search all requests and all working terms - for the future
• Collaboration
  • use JIRA - when everything is setup correctly, then try to get collaboration on using JIRA to review items between meetings
  • make the spreadsheet more easily available to everyone
• Terminology Release
  • rolling - provisional possibility - maybe through SHARE
    • then maybe we could release 2 times a year
• Public Review
  • could possibly be done in JIRA in the future
  • need more reviewers
  • need to see the rational/team discussions for what is added or changed
    • this could be a drill down situation In JIRA to show the discussions
• Terminology Development
• Europe/Japan/etc  need to be able to participate - use Wiki for the spreadsheet info which gives the ability to allow comments
• The terminology teams use existing terminology whenever possible
• NCI Thesaurus, LOINC, UCUM, etc
• CDISC is organizing existing terms into meaningful codelists
• CDISC needs to make it well known on what the position is for LOINC and UCUM
• Sam is working on a LOINC position paper
• Erin and Diane write a position blog that CDISC terminology creates codelists that can be used with SDTM data structures
• The CDISC RACE variable is for FDA submissions only, this needs to be made clear
  • we could have a large race codelist and a subset for the FDA values, make this an action item, NCI/EVS already has the total list but we need the CDISC SDTMIG resolution too.
• QRS - keep current coding process and build in a process that identifies the similar questions (with an index)
  • the instrument owner needs to make the statement that these questions are exactly the same before we would say they are identical and re-usable across multiple instruments.
    • This would be a very large amount of work
• SEX - needs to be made clear to the users in an IG or FAQ. 
  • definition is clear. SEX is self-reported, more like the question of Gender than physiological sex.
  • new term for 'sex at birth' being added to SCTEST/TESTCD.
• New domains
  • Best practice to involve terminology during the design instead of after the domain is final and released
• Test and testcd on the same row
  • in SHARE you can view the test and testcd pair together
  • SHARE could publish an alternate terminology view that would have the test and testcd on the same row - action item
• multiple submission values for the unit codelist for true mathematically synonymous terms
• Decision is made to not allow this. Instead proposal will be made to create a new variable for verbatim units in CDASH (VRESU)
• add synonym when there is a definite, confirmed use case through change request.

• Subsets
• we should create some subsets that are clear and well defined in SHARE
• needs a process to determine when we actually need a subset
• define the criteria is a Collaboration Curation Initiative CCI topic (Sam)
• need a decision tree for who approves what?
• need some identifier for the subsets in SHARE- need a C-code for the subset, only for persistent subsets
  • • once we identify a subset
      • submit a new term request which identifies this as a subset request - this will be handled by NCI/EVS
• what is the correct process flow?
• TA will be requesting subsets
  • Need a decision authority that determines if this is a valid subset? A formal metadata governance team - action item Diane
    • original requestor puts in a request to the metadata governance
      
      • when approved request goes to SHARE (a new request type) and a request to the NCI/EVS new term request system for a c-code
• how do we know what subsets exists?
  • publish a separate list?  include a link to the parent codelist?
  • need to make subset different than superset?
  • there should only be two levels at this time (no subsets of subsets)

• codelists
  • need clear explanation on the use of synonyms
    • add to Terminology User Guide
    • maybe put frequently asked questions on the wiki/JIRA
  • the codelist name should not be the same as the variable name
  • must publish the codelist to SDTM mapping document
  • citation must be part of the definition according to NCI/EVS (attribute field not visible in the NCIt browser so they can't split this up currently)

• LOINC
  • link LOINC to a lab concept
  • backfill common labs with LOINC to get a skeleton of lab tests

• Develop a JIRA system to manage new term request
• JIRA pilot for CDISC new term request system
  • use existing team working spreadsheets during this time
• FInd a new solution for team working spreadsheets
  • wiki, jira, Share, database, or something else
• Get CDISC upper management approval to finance the JIRA development for CDISC change requests
  • help from Bron, Diane, Rhonda, and Sam