Dataset-JSON was adapted from the Dataset-XML Version 1.0 specification, but uses JSON format. Like Dataset-XML, each Dataset-JSON file is connected with a Define-XML file containing detailed information about the metadata. One aim of Dataset-JSON is to address as many of the relevant requirements in the PHUSE 2017 Transport for the Next Generation paper as possible, including the efficient use of storage space.

Dataset-JSON is using lowerCamelCase notation for attribute names, comparing to Dataset-XML PascalCase (e.g., clinicalData vs ClinicalData).

JSON format does not allow to specify or control order of attributes. Despite of that, as most JSON engines allow to control the order of attributes it is strongly recommended to follow the attribute order specified in details. The reason for that is that due to a possible large size of Dataset-JSON files, following the specified order will enable a software using steaming approaches to read the file to work in an efficient and fast way.

Dataset-JSON must contain only one dataset per file. 

Top Level Attributes

At the top level of Dataset-JSON object, there are technical attributes and two main optional attributes: clinicalData and referenceData, corresponding to Dataset-XML elements. At least 1 of the main attributes must be provided. Subject data is stored in clinicalData and non-subject data is stored in referenceData.

{
    "creationDateTime": "2023-03-22T11:53:27",      
    "datasetJSONVersion": "1.0.0",
    "fileOID": "www.sponsor.xyz.org.project123.final",
    "asOfDateTime": "2023-02-15T10:23:15",
    "originator": "Sponsor XYZ",
    "sourceSystem": "Software ABC",
    "sourceSystemVersion": "1.0.0",

...

...

...

...

Introduction

JSON representations for exchange standards are widely used in today’s architectures. In RESTful web services, JSON is often the preferred format for the service response, due to its compactness and ease of use in mobile applications. Other standards used in healthcare, such as HL7-FHIR support JSON as well as XML, together with other formats such as RDF.

JSON and XML are however not 1:1 interoperable, as they are based on different principles. For examples, JSON does not have a native mechanism for namespaces (as it wants to remain "lightweight"). Also JSON does not have an equivalent for XML "text content". In JSON, "text content" is treated in the same way as "attribute pairs" of XML.

...

Starting from ODM version 2 (ODMv2) a JSON representation for ODM is available.
This document explains the principles of the JSON representation, and the conventions used. These are based on the "Flickr conventions" for JSON (https://www.flickr.com/services/api/response.json.html).

Main principles

For the JSON implementation of ODM, the following main principles apply:

• Just like XML, JSON is case-sensitive.
• JSON is based on sets of name-value pairs. These are separated by a colon. Name and Value are embedded in double quotes.
  Example: "OID": "MyStudy"
• XML elements are represented as JSON objects.
• Arrays of objects or name-value pairs are represented by and embedded in square brackets.
  For example: ["a","b","c"] represents a list of the objects with name "a", "b" and "c".
• In JSON, an object is an unordered set of name-value pairs. An object begins with a left brace '{' and ends with right brace '}', preceded by the object name (in double quotes). Each name is followed by colon and the name/value pairs are separated by a comma.
  Example:
  

"Protocol": {
    "StudyEventRef": [{
        "Mandatory": "Yes",
        "OrderNumber": 1,
        "StudyEventOID": "BASELINE"
    }]
}

The example above shows the JSON serialization of the XML element "Protocol" with an array of child "StudyEventRef" elements (as can be seen from the curly brackets, which has the attributes "Mandatory" (with value "Yes"), "OrderNumber" (with value 1) and "StudyEventOID" (with value "BASELINE"). More than one StudyEventRef may be included since it is defined as an array ("StudyEventRef": [...]). Attributes are represented as name-value pairs, such as "Mandatory": "Yes".

Note that the indentation is completely arbitrary, and (just like in XML), does not imply anything. Also, line breaks used to format the JSON do not have a meaning: very complex JSON or XML files of 1 GB in size can just consist of one single line. However, line breaks within strings (content surrounded with double quotes), have meaning, for example line breaks are not allowed and must be replaced with \n.

• XML text content is treated as a name-value pair with the name being "_content".
  Example:

"StudyName": {"_content": "Test Study 003"}

and combined with the parent element "GlobalVariables":

"GlobalVariables": {
    "StudyName": {"_content": "Test Study 003"},
    "StudyDescription": {"_content": "Test Study 003 created by API"},
    "ProtocolName": {"_content": "Test Study 003 created by API"}
}

• Namespaces are ignored.
  For ODM, this essentially means that the attribute "xml:lang" translates into "lang".
  Example:

"Description": {
    "TranslatedText": [{
        "lang": "en",
        "_content": "Unique identifier for a study."
    }]
}

Representing the ODM-XML element "Description" element with child element "TranslatedText", having the "xml:lang" attribute with the value "en" and the text content "Unique identifier for a study."

• As usual in JSON, the root element is not explicitly named:
  Example:

{
    "CreationDateTime": "2011-10-24T10:05:00",
    "Description": "JSON test",
    "FileOID": "JSON_Test_2020",
    "FileType": "Snapshot",
    "Granularity": "Metadata",
    "ODMVersion": "2.0",
    "Originator": "MySystem",
    ...
    ...
}

Representing the ODM element with attributes "CreationDateTime", "Description", "FileOID", "FileType", "Granularity", "ODMVersion", and "Originator".

Dataset-JSON

Dataset-JSON is based on the Dataset-XML specification, but represents a different approach from the one described above. It utilizes JSON format specifics to efficiently store data.  Each Dataset-JSON file is connected with a Define-XML file, containing detailed information about the metadata. One aim of Dataset-JSON is to address as many of the relevant requirements in the PHUSE 2017 Transport for the Next Generation paper as possible, including the efficient use of storage space.

At the top level of Dataset-JSON object, there are two optional attributes: clinicalData, referenceData, corresponding to Dataset-XML elements.

{
    "clinicalData": { ... },
    "referenceData": { ... }
}

ClinicalData and ReferenceData Attributes

Both clinicalData and referenceData have the same structure. Each of these attributes contains study and metadata OIDs as well as , optional reference to the metadata file and an object describing an item group (dataset). The following attributes are defined on this level.

one or more item groups (datasets).  Values of the studyOID and metaDataVersionOID must match corresponding values in the Define-XML file.

{
    "clinicalData": {
        "studyOID": "xxx",
        "metaDataVersionOID": "xxx",
        "metaDataRef": "https://metadata.location.org/api.link",
        "itemGroupData": { ... }
}

ItemGroupData attribute

itemGroupData is an object with attributes a single attribute corresponding to individual datasetsan individual dataset. There must be only one dataset per Dataset-JSON file. The attribute name is OID of a described dataset, which must be the same as the OID of the corresponding itemGroup itemGroupDef in the Define-XML file.

"itemGroupData": { 
    "IG.DM": { ... }
}

The dataset description contains basic information about the dataset itself and its items.

...

...

...

...

"IG.DM": {
    "records": 100,
    "name": "DM",
    "label": "Demographics",
    "items": [ ... ],
    "itemData": [ ... ]
}

items is an array of basic information about dataset variables. The order of the elements in the array must be the same as the order of variables in the described dataset.

...

The first element always describes the Record Identifier (ITEMGROUPDATASEQ).

...

...

...

"items": [    
    {
        "OID": 100"ITEMGROUPDATASEQ",
        "name": "ITEMGROUPDATASEQ",
        "label": "Record identifier",
        "type": "integer",
    },
    {
        "OID": "IT.DM.STUDYID",
        "name": "STUDYID",
        "label": "DemographicsStudy Identifier",
        "type": "float""string",
        "length": 12,
        "keySequence": 1,
    },
    ...
]

itemData is an array of records with variables values. Each record itself is also represented as an array of variables values. The first value is a unique sequence number for each record in the dataset.

"itemData": {[
   [1, "MyStudy", "001", "DM", 56],
   [2, "MyStudy", "002", "DM", 26],
}
   ...
]

Missing values are represented by empty elements of an array: [null in the case of numeric variables, and an empty string in case of character variables: [1, "MyStudy", "", "DM", null]

The following is a full example of a Dataset-JSON file:

{
    "creationDateTime": "2023-03-22T11:53:27",      
    "datasetJSONVersion": "1.0.0",    
    "fileOID": "www.sponsor.org.project123.final",
    "asOfDateTime": "2023-02-15T10:23:15",
    "originator": "Sponsor XYZ",
    "sourceSystem": "Software ABC",
    "sourceSystemVersion": "1.2.3",
    "clinicalData": {
        "studyOID": "xxx",
        "metaDataVersionOID": "xxx",
        "metaDataRef": "https://metadata.location.org/api.link",
        "itemGroupData": {
            "IG.DM": {
                "records": 600,
                "name": "DM",
                "label": "Demographics",
                "items": [                      
                    {"OID": "ITEMGROUPDATASEQ", "name": "ITEMGROUPDATASEQ", "label": "Record identifier", "type": "integer"},
                    {"OID": "IT.STUDYID", "name": "STUDYID", "label": "Study identifier", "type": "string", "length": 7, "keySequence": 1}, 
                    {"OID": "IT.USUBJID", "name": "USUBJID", "label": "Unique Subject Identifier", "type": "string", "length": 3, "keySequence": 2}, 
                    {"OID": "IT.DOMAIN", "name": "DOMAIN", "label": "Domain Identifier", "type": "string", "length": 2},
                    {"OID": "IT.AGE", "name": "AGE", "label": "Subject Age", "type": "integer", "floatlength": 2}
                ],
                "itemData": [
                    [1, "MyStudy", "001", "DM", 56],
                    [2, "MyStudy", "002", "DM", 26],
                    ...
                ]
            }
        }
    },
    "referenceData": {
        ... Same structure as clinical data
    }
}
}

The TypeScript model representation and the JSON schema for Dataset-JSON version 1.0 can be found at https://github.com/cdisc-org/DataExchange-DatasetJson.