Often, analysis results are created and represented as static PDF documents that may contain hundreds of tables. These tables are often difficult to navigate and there can be significant variability between sponsors. Generating these reports is expensive and are typically only used once, offering limited reusability.
A typical workflow for producing analysis results involves the end user generating the display in a static format such as RTF or PDF from the Analysis Data Model (ADaM) dataset. The Analysis Results Metadata (ARM) for Define-XML is created retrospectively as high-level metadata documentation about analysis displays and results (Figure 1); however, there is no formal model or structures to describe analysis results and associated metadata, leaving a gap in standardization. The current process is expensive, time-consuming, and lacks automation and traceability, leading to unnecessary variation in analysis results reporting.
Figure 1: Current State of Display and ARM Generation (insert higher resolution picture)
The vision for the future state of analysis results reporting is where analysis results are machine-readable, easily navigable, and highly reusable. We envision the following:
- A logical model for describing analysis and results data
- Automated generation of machine-readable results data
- Improved navigation and reusability of analysis and results data
- Support for the storage, access, processing, and reproducibility of results data
- Traceability to the study protocol/statistical analysis plan (SAP) and to input ADaM data
- Open-source tools for designing, specifying, building, and generating analysis results data
To achieve these goals, a logical model to fully describe analysis results metadata has been developed. This logical model will enable the implementation of an Analysis Results Metadata Technical Specification and an Analysis Results Data framework. ARM-TS can be used to support automation, traceability, and the creation of data displays while the ARD framework will support reuse and reproducibility of results data.
To address the current limitations of analysis results and associated metadata reporting, we are proposing a new workflow that shifts the focus from retrospective reporting to prospective planning. Specifically, we propose that end-users generate the a technical specification to l prior to generating a display, rather than after the display has been created. This approach will allow for better planning and standardization of the analysis process, resulting in more consistent and traceable reporting.
The proposed workflow (Figure 2) involves several steps. First, the end-user will develop a technical specification, which will include metadata about the statistical methods, data sources, and displays to be generated (Figure 2, Use Case 1). Once the technical specification has been developed, the end-user will use it to generate an analysis results dataset, which will contain the results data needed to generate the display (Figure 2, Use Case 2). The analysis results dataset will be designed to support reuse and reproducibility of the results data, enabling more efficient and effective analysis reporting.
Finally, the machine-readable analysis results dataset serves as the ‘single source of truth’ capturing the analysis results metadata and results data in a standardized format. This dataset can then be used to generate displays for multiple reporting purposes, such as traditional analysis reporting for the clinical study report (CSR), in-text tables for the CSR, safety reporting, meta-analyses, dynamic applications, ClinicalTrials.gov, publications, and presentations. This streamlined approach ensures consistency and accuracy in the generation of displays across various deliverables, making it more efficient and reliable for reporting and communication of analysis results.
Figure 2: Workflow with Future Extensions and Use Cases
Overall, this new workflow will enable end-users to generate analysis results metadata prospectively, with greater standardization, consistency, and traceability of analysis results reporting, enabling better decision-making and regulatory submissions. By shifting the focus from retrospective reporting to prospective planning, we believe that this approach will help to address many of the current limitations of analysis results reporting and support the development of more efficient and effective analysis standards.
In our vision for the future state, we anticipate the availability of open-source or community tools, such as the TFL Designer Community [1], in the industry. These tools will empower users to create machine-readable analysis metadata, which can automate the generation of analysis results data and displays. We also hope that such a tool can seamlessly integrate with existing analysis programs and report creation tools, enabling an end-to-end automation of the analysis and reporting process.
Use of LinkML for the development of the ARS Logical Model
Many of the same metadata components are needed both to create a prospective technical specification of analyses to be performed and to give context to reported results. Therefore a single logical model comprising components needed to specify analyses, to represent contextualized results and to indicate how the results are displayed. The logical model is being developed using LinkML.
LinkML is an open-source schema development language and framework for generating machine-readable models [2]. The LinkML Generator framework generates downstream artifacts, including JSON-Schema, ShEx, RDF, OWL, GraphQL, and SQL DDL.
The logical model acts as a blueprint or a set of rules that describes how things should be organized and structured. In our case, it is being used to create a model that describes how analysis results data should be organized and structured. LinkML has allowed the creation of a standardized and consistent approach to describing analysis results data, enabling greater interoperability and collaboration across different stakeholders. Additionally, it has the added benefit of being able to convert them into various machine-readable formats such as JSON, YAML, OWL, or XML.
Furthermore, LinkML is a flexible and extensible tool, meaning that the model can be easily modified as needed to incorporate new data elements or requirements. By creating a machine-readable model, analysis results data can be more easily understood, shared, and reused by both humans and machines. LinkML will also support the development of validation rules that can be used to ensure the integrity and quality of the data, which is essential in our highly regulated pharmaceutical industry.