Matt Shearer

We are developing our Text AI Media Classification Products to help Cochrane automatically classify Clinical Studies onto Snomed-CT and PICO structured data

AI Classification of Clinical Studies with Text AI to fight Covid-19

We've just received the green light on this public-funded innovation project,
bringing two areas of significant Data Language expertise into play to help
combat Covid-19.

The mission combines two of our favourite things:
1. Clinical Evidence Knowledge Graphs
We've been working with Cochrane on their knowledge graph and data strategy
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19] 
since we started the project with them in 2014. As part of this, we recently
helped them launch their Covid-19 Study Register
[https://datalanguage.com/blog/covid-19-rapid-response-technical-delivery-at-cochrane]
, which summarises the current state of Covid-19 clinical evidence from around
the world that has been released into the public domain via authoritative trial
registries and journal databases.

2. Our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] for
automated text classification
Over that time, we have also cemented our position as industry leaders in
machine learning text classification and analytics, and have developed
particular expertise in high volatility domains (e.g. academic research,
breaking news, R&D) and complex ML engineering challenges. This led us to
creating our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services].
[https://datalanguage.com/tagmatic-ai-document-classification]

Mission: to ensure Covid-19 clinical decision making is informed by the latest,
most comprehensive reliable evidence possible.
With the help of the UKRI [https://www.ukri.org/], Digital Catapult's Machine
Intelligence Garage
[https://www.digicatapult.org.uk/for-startups/acceleration-programmes/machine-intelligence-garage]
, and Innovate UK [https://www.gov.uk/government/organisations/innovate-uk], we
are now bringing our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] into play
with our subject matter expert partner Cochrane to help the fight against
Covid-19.

We will be launching this new HealthTech AI beta before December. 

Get in touch if you want to know more.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common specialist and scaling problems.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common digital transformation challenges.

Data Language AI - News and AI Product Updates

Data Language AI

Automated Classification of Healthcare Evidence

Who is Cochrane?
Cochrane [https://www.cochrane.org/] is the global not-for-profit organisation
whose core mission is to put evidence at the heart of health decision-making all
over the world. Over 90% of WHO guidelines are informed by Cochrane’s systematic
reviews.

What is the Covid-19 Study Register?
The Register is a precisely scoped, authoritative source of the latest clinical
evidence from around the world concerning the Covid-19 pandemic, that is being
used to inform point-of-care clinical decision making and evolving healthcare
policy internationally.

Healthcare professionals can use it to locate all relevant, reliable clinical
evidence about specific healthcare questions from the latest Randomised
Controlled Trial (RCT), observational, modelling and other studies to support
the fight against Covid-19.

This crucial, trusted resource is essential at a time when the evidence base is
developing rapidly, as healthcare professionals require structured and reliable
up-to-date information.

Partners with Cochrane since 2014
We first worked with Cochrane in 2014 on a digital transformation project,
designing and delivering a radical new platform architecture centred around
linked-data and knowledge graph technologies. Since then we have worked to
refine, iterate and scale their platform to create a suite of applications to
publish evidence in healthcare using structured data.

Solid data architecture makes rapid innovation possible
This Covid-19 Studies Register is the latest example of how Cochrane is able to
innovate rapidly on the scalable knowledge graph platform architecture delivered
by Data Language. The structure and flexibility means that new use cases can be
delivered rapidly and efficiently.

See the Covid-19 Study Register here [https://covid-19.cochrane.org/].

Find out more
 * Cochrane's Covid-19 Resources and News
   [https://www.cochrane.org/coronavirus-covid-19-cochrane-resources-and-news] 
   (page updated regularly).
 * More about Cochrane and the rapid delivery of the Covid-19 Study Register
   project
   [https://datalanguage.com/blog/covid-19-rapid-response-technical-delivery-at-cochrane]
   .
 * Our work with Cochrane since 2014 - Information Architecture, PICO Ontology
   and more
   [https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
   .
 * More about our Knowledge Graph Engineering
   [https://datalanguage.com/knowledge-graph-engineering].

A knowledge graph is a collation of information and data typically within the domain of your business context. The information is stored in a database, often a graph database such that it can be queried, maintained, and interacted with by humans or machines. We have exceptional experience in the design, implementation and deployment of knowledge graphs. We have been working with graph databases since our inception and have delivered knowledge graphs solutions for global organisations.

We have exceptional experience in the design and implementation of knowledge graphs and have delivered knowledge graphs solutions for global organisations.

Knowledge Graphs - our Work, Features, News and Updates

Knowledge Graphs

Launching the Cochrane Covid-19 Study Register

Paul Wilton

How Cochrane has integrated Data Graphs [https://datagraphs.com] with their own
enterprise graph database of evidence-based healthcare as a complete linked data
management solution.

Population
Data Language has been working with Cochrane as a technical partner for a number
of years now, and we have blogged extensively
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19] 
on the digital transformation at Cochrane, including how linked data and an RDF
based knowledge graph have become a core information backbone for their business
processes.

The Cochrane vocabularies of some 300k concepts provide the linked reference
data that describes the PICO domain of evidence-based healthcare, describing the
Populations (and their conditions, ages, sexes, social context), the
Interventions (drugs, devices, procedures, materials etc), Comparisons, and the
Outcomes. These concepts are annotated onto clinical evidence as PICO
micrograph
data structures [https://data.cochrane.org/ontologies/pico/index-en.html] to
describe the questions being answered in clinical evidence. All of Cochrane’s
vocabularies, content metadata and PICO annotations are stored in an enterprise
graph database, and exposed by a cohesive set of microservice APIs that can be
consumed and recombined to allow rapid innovation.

Data Language has provided the technical architecture, design and implementation
of the knowledge graph, microservices, APIs, and tooling that allows the
Cochrane subject matter experts to curate and QA these PICO data structures as
part of their daily business processes. This linked data platform has allowed
Cochrane to rapidly innovate: enhancing the Cochrane Library with linked data
query capability; and build at lightning speed, the largest Covid-19 database of
clinical evidence in the world, comprising more than 100k studies, described
with well-structured PICO metadata,  serving high volumes of requests and
queries to researchers globally.

Condition
While tooling and processes for Cochrane’s SMEs to create and maintain PICO
graphs describing the Cochrane systematic reviews and studies is state of the
art, the tooling around the management of the reference data vocabularies is
less sophisticated. The Cochane vocabularies consist of a highly curated set of
linked data concepts that form a graph in their own right with class and
property semantics, transitivity, and multi-parent taxonomic structures.
Although Cochrane has an existing visual editor tool to perform operations on
individual concepts in the reference data, it is limited in functionality, and
does not allow for bulk ingest and updates, rich search and discovery operations
for quality assurance, data cleansing, and other operations such as merge,
migrate, and workflow.

To solve the vocabulary management problem, we have integrated our product Data
Graphs [https://datagraphs.com] into the Cochrane linked data ecosystem.

What is Data Graphs ?
Data Graphs is a SaaS product designed to lower the barrier of entry to
Knowledge Graphs. While it can be used purely as a Knowledge Graph in its own
right, it can also be used as a linked data vocabulary management tool, and
integrated with an enterprise graph database, or other business systems using
its API and Web-hook event triggers.  It has an intuitive user experience for
managing linked data, and powerful search, discovery and visualisation
capability, effectively making an ideal vocabulary management tool for Cochrane.

Data Graphs allows you to design or replicate any domain model or ontology, and
populate it with linked data, so it was a straightforward task to replicate the
existing Cochane concept model
[https://data.cochrane.org/ontologies/core/index-en.html] and ingest their
vocabularies. The screenshot below shows the cochrane model recreated in Data
Graphs. 


Cochrane Core Concept Model
Data Graphs provides a number of benefits to Cochrane for managing their linked
data vocabularies :

 * Fully decoupling the workflow and processes around managing their linked
   reference data from their business critical systems.
 * Being able to flexibly and easily modify their core concept model without
   impacting their downstream processes.
 * An intuitive user experience for search, discovery, match, merge and
   migration of data.
 * Graph visualisation for exploring their linked data, and identifying
   anomalies and circular relationships.
 * Easy identification of duplicates
 * A full audit history of all changes made to the vocabularies
 * Injection workflow states into the linked data concepts - for managing
   pre-publication, publication and deprecation of terms.

Cochrane’s Data Architecture
Cochrane’s linked data architecture is centred around an enterprise graph
database (Ontotext’s RDF-based GraphDB
[https://www.ontotext.com/products/graphdb/]), a set of microservices and online
tools for annotating systematic reviews and clinical studies with creating well
formed PICO graphs that describe the evidence. A slice of the logical
architecture  is shown below and is discussed in more detail here
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
:

Cochrane Linked Data ArchitectureIntervention
Integrating Data Graphs
In order to integrate Data Graphs we needed an integration pattern that is
robust and also allows Cochrane SME’s to understand the impact of changes they
may make to the vocabularies, such that they can make an informed decision
whether or not they should proceed with the updates.

For integration with an organisation’s business systems, Data Graphs comes with
an expressive, contemporary REST API, serving JSON-LD payloads (pull), and a
suite of webhooks that can be used for event-driven integrations (push) that can
be configured for different core operations, Create, Update, Delete, Merge etc.
For Cochrane’s purposes the webhooks are ideal. 

Pre-Commit Hooks
Furthermore, for each web-hook, a pre-commit remote endpoint can be configured,
which will be invoked before a write operation is committed in Data Graphs, on
which the target business system can respond with a payload of information that
Data Graphs will render to the user as an impact-assessment if the transaction
is completed. This gives the user an opportunity to back out. These pre-commit
hooks allows the target system to respond for a variety of scenarios :

 1. Silent commit - no impact, just commit the transaction silently, the user is
    not blocked.
 2. Force reject - in which the target system determines the transaction should
    not proceed, and the user is not allowed to commit the change.
 3. An impact assessment, with a risk score, an optional summary of the impact,
    and an optional table of data, that the Data Graphs user will be able to
    make an informed decision upon.

Resilience
When consuming web-hooks (essentially an HTTP endpoint or API in a target
business system, that is configured and invoked on some trigger condition), we
need to ensure to all reasonable endeavours that the event fired will not be
lost due to unforeseen circumstances (eg target system failures, connectivity
issues etc.).  To achieve this, the target system endpoint needs to be as robust
as possible and ideally decoupled from other downstream systems. 

The best way of achieving this in the target system is to build an extremely
lightweight microservice that simply receives the HTTP event payload and places
it on a topic or queue. This queue can then be consumed at the target system's
leisure and can cope with failure downstream (the message will not be lost if
the consumer fails temporarily). The target system can take this further with
dead-letter queues and retries to cope with a variety of challenging
circumstances. This is the pattern adopted at Cochrane and is a pattern we would
generally recommend for consuming webhooks between other products and platforms.
You can see the logical architecture for this below :

Data Graphs Integration Architecture at CochraneThe key integration points
(numbered in red)  within the Cochrane technical stack are :

 1. The Pre-commit web-hook handler. This is a simple webservice that is invoked
    by Data Graphs for a specific transaction event before the transaction is
    committed (for example a Concept Delete). The service makes a query against
    the Cochrane evidence graph to ascertain the impact the change will have.
    For example if a concept is used on thousands of PICO graphs, it is likely
    the user should not be deleting this concept, and the service can respond
    with a straight reject, or return a table of the PICOs affected.  This
    information is presented back to the user to make an informed decision.
 2. The Web-hook handler. This is a lightweight service that is invoked by Data
    Graphs for a specific transaction type (Concept Delete, Merge, Update etc)
     after the change is committed. This service simply places the data event on
    a queue, ready to be processed.
 3. Concept Event Handler. This service does the heavy lifting, taking events
    off the queue, and writing updates to concepts and PICOs in Cochrane’s Graph
    Database using SPARQL. If this fails for whatever reason, the message is
    retained on the queue, and can be tried later. For Cochrane much of this
    code already existed in the legacy LDGraphs API, which was performing a
    similar function for their original linked data editor, so was a case of
    porting this into the event handler.

The webhooks in Data Graphs are easy to configure, as can be seen on the
following screenshots.

First, choose an event type to trigger a web-hook, then configure the endpoint
in the target system, an optional pre-commit hook, and any headers / security
tokens that need to be sent with the request :

Webhook Configuration in Data GraphsMigrating the vocabularies into Data Graphs
The final step was to load the Cochrane linked data vocabularies into Data
Graphs, conforming the domain model we created earlier. This can be done in
different ways

 * Using the Data Graphs REST API
 * Using CSV upload using the CSV to Ontology mapping tool within Data Graphs

In this case, the API was the easiest approach as we could read pages of JSON-LD
concepts from Cochrane’s concepts API, and pipe them into the Data Graphs write
API. As the ontology model’s in both were identical, this was a relatively
trivial migration script.

Outcome
With this in place, Cochrane now has a state of the art linked data management
toolkit. The fully whitelabelled user experience engenders user-ownership.
Cochrane’s information experts can organise their core entities in logical
datasets for rapid navigation:

Cochrane Datasets
Drilling into each dataset, a SME can search for concepts, and easily find
potential duplicates through a complex query or use the label/property facets :

Faceted SearchAnd merge those duplicates with an intuitive UI :

Merging Concepts in Data Graphs
The resulting merge transaction will be handled by the webhook event, not just
to update and merge the concepts in the Cochrane Graph Database, but also update
all PICO graphs in which the source concept was previously used to reference the
target concept instead.  Similarly, deletes will remove the concept reference
from all content and PICO annotations.

Making any changes, such as deleting a concept results in the pre-commit hooks
requesting an impact assessment, so the user can make an informed decision to
continue or cancel:

Pre-commit Impact AssessmentsThe knowledge graph can be explored visually to
help find and correct anomalies in the vocabulary, such as circular
relationships :

Graph Explorer in Data GraphsThe Cochrane bioinformatics SMEs now have many more
capabilities at their disposal taking their ability to deliver the next
generation of evidence-based healthcare innovations to a new level.

.

Data Graphs at Cochrane: a Case Study

A few weeks ago Cochrane launched their COVID-19 Study Register and I described
the data and technical architecture in detail here
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
.

I described how PICO graphs [https://data.cochrane.org/ontologies/] are used to
augment the Studies with structured linked data that describe the Population,
Intervention, Comparator and Outcome of the trial. Cochrane have now started in
earnest annotating PICO graphs onto the Studies, and these are now being
surfaced in the Study Register application [https://covid-19.cochrane.org/].

Here, we can see the labels of the PICO concepts rendered in the UI, but in the
data itself, these are arranged as a PICO graph structure conforming to the 
PICO
ontology [https://data.cochrane.org/ontologies/pico/index-en.html].

We have also added faceted search by PICO concept into the sidebar. This lets
users quickly filter down their search results using the Cochrane vocabulary
terms.

Once we have a critical mass of PICOs annotated in the register, each vocabulary
concept will be searchable in the search bar at the top of the app, extending
the previously full-text search into a controlled-vocabulary search. A user can
now find studies using a combination of terms chosen from the Cochrane linked
data vocabulary, to deliver extremely accurate results. Moreover we can combine
a metadata driven search with full-text where needed.

Schema.org
Each Study page rendered in the application is also marked up with schema.org
[https://schema.org]structured data. So while internally we are describing our
content and PICO data with the Cochrane bounded context ontologies, at render
time, we are transforming our model to the schema.org MedicalEntity model to
provide machine readable content for search engines and other content consumers.
You can read more about the schema.org medical schema here.
[https://schema.org/docs/meddocs.html]

As an example, for this Studyhttps://covid-19.cochrane.org/studies/crs-13266121 
when viewing the source of the page, you can see we are marking up the clinical
trial as a MedicalStudy in the schema.org ontology, and identifying the 
healthCondition terms, and the interventions, including the Drug Chloroquine.
Each term has associated medical codes mapped to bioinformatics taxonomies
including SNOMED-CT, MeSH, RxNorm and the ATC codes for Drugs where available.
The resulting schema.org JSON-LD for this study is :

{
  "@context": "http://schema.org",
  "@graph": [
    {
      "@type": "MedicalStudy",
      "id": "http://data.cochrane.org/studies/crs-13266121",
      "name": "NCT04333628",
      "identifier": "NCT04333628",
      "description": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
      "mainEntityOfPage": {
        "@type": "WebPage",
        "url": "https://covid-19.cochrane.org/studies/crs-13266121",
        "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
        "description": "Clinical study : Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial"
      },
      "healthCondition": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        }
      ],
      "studySubject": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        },
        {
          "type": "MedicalProcedure",
          "id": "http://data.cochrane.org/concepts/r4hp5z141f1t",
          "url": "https://data.cochrane.org/concepts/r4hp5z141f1t",
          "name": "Usual Care",
          "alternateName": [
            "Routine Prenatal Care",
            "Routine Care",
            "Standard Care",
            "Providing care according to standard",
            "SOC"
          ],
          "code": [
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "372921003"
            }
          ]
        },
        {
          "type": "Drug",
          "id": "http://data.cochrane.org/concepts/r4hp13k8jxft",
          "url": "https://data.cochrane.org/concepts/r4hp13k8jxft",
          "name": "Chloroquine",
          "alternateName": [
            "Chloroquine",
            "Product containing chloroquine"
          ],
          "code": [
            {
              "codingSystem": "MeSH",
              "codeValue": "D002738"
            },
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "14728000"
            },
            {
              "codingSystem": "RxNorm",
              "codeValue": "2393"
            },
            {
              "codingSystem": "ATC",
              "codeValue": "P01BA01"
            }
          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "name": "Pharmacological Interventions"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "name": "Other"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        }
      ],
      "subjectOf": [
        {
          "@type": "MedicalScholarlyArticle",
          "datePublished": "2020-01-01",
          "url": "https://clinicaltrials.gov/show/NCT04333628",
          "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19",
          "abstract": "19 COVID is a deadly viral disease that has been spreading around the world for several months, and is caused by a CORONA family virus (COVID-19). Following IN-VITRO evidence of the antiviral effect of CHLOROQUINE in CORONA viruses, this drug has been used empirically for COVID-19 patients and is currently recommended in Israel for the treatment of intermediate and severity disease. The mechanism of action of chloroquine is in part by inhibiting the virus distribution, and changing the intracellular acidity, the virus distribution site. The intracellular chloroquine concentration is determined by a pump called PGP that removes the drug from the cell and is activated by the drug. In the treatment of malaria, the benefit of low dosage of the drug has been shown to be effective due to the fact that the intracellular concentration of the drug is probably higher, and therefore the logic to examine this issue in COVID-19 treatment. The purpose of this study is to test whether a low dose of Chloroquine will reduce the duration of the viral shedding and prevent the disease from worsening",
          "author": [
            {
              "type": "Person",
              "name": "HaEmek Medical Center"
            }            
          ]
        }
      ]
    }
  ]
}

PICO and Schema.org
The PICO model and process (https://en.wikipedia.org/wiki/PICO_process) is now
widely adopted for framing research questions in evidence based healthcare with
the BMJ citing this model as best practice
[https://bestpractice.bmj.com/info/toolkit/learn-ebm/how-to-clarify-a-clinical-question/]
.

PICO data is applied by clinicaltrials.gov in a semi-structured tabular format
on most of their published trials and PubMed also has a PICO search tool for
querying MEDLINE, the  database from the US National Library of Medicine.

Supporting PICO would be a welcome addition to the schema.org model, allowing
any publisher of medical literature to mark up their content in a standard way
with machine readable structured PICO data as Cochrane and others are now. I can
imagine a future where Google indexes the schema.org PICO markup and provides
the ultimate PICO based search tool across all the medical literature citations
at their disposal.

At Data Language, as co-authors of the PICO Ontology, we would be more than
happy to help define some simple extensions to the schema.org MedicalEntity
model to make this happen.

Data Language is an industry leader in Information Architecture, ontology design and practical knowledge graph implementation. We use Domain Driven Design practises and user-centred design to focus information and data design on core business needs.

Find Information Architecture news and product innovation updates here, including ontology design, vocabulary management, domain & data modelling and more.

Information Architecture - News, Product Innovation and Technology

Information Architecture

PICO, Cochrane and Schema.org

Julian Everett

Over the past five years it has been our privilege to be a technology partner of 
Cochrane [https://www.cochrane.org/]. For anyone unfamiliar with their work,
they are globally recognised as the gold standard in evidence-based healthcare.
Historically their primary published output has been systematic reviews, which
evaluate and summarise the current state of human knowledge about specific
clinical questions. These then inform both healthcare policy and clinical
guidelines for point-of-care decision making around the world. For example, over
90% of WHO guidelines are informed by Cochrane reviews. 

The review production process starts with the formalisation of the clinical
question of interest in terms of the Population it relates to (e.g. age, sex,
condition), the Interventions being investigated (e.g. drugs and/or procedures),
the Comparators against which the Interventions are being evaluated (e.g.
placebo), and the Outcomes against which effects are measured. This is then
translated into search protocol and run against trial registries and journal
databases to identify all relevant clinical study information in the public
domain. The matches are then evaluated for risk of bias, and the subset of
results which meet Cochrane’s stringent data quality thresholds are subsequently
aggregated and analysed to determine the cumulative effect estimates. 

In essence, this is about the identification of causal relationships between
interventions and outcomes. To that end, up until now Cochrane has focussed on
double-blind randomised controlled trials. They provide the highest quality
input data: randomisation greatly reduces the likelihood of alternative
explanations for the observed results as the number of participants increases;
blinding of field researchers and participants (i.e. hiding information about
specific interventions) removes the cognitive and behavioural bias risks that
might arise if, for example, placebos are known. 

Finally, review production is not a static process. New information is being
released into the public domain all the time both in the form of additional
journal articles about completed studies as well as information about new
studies. As a result, ongoing evidence surveillance and timely meta-analysis
updates are important additional requirements.

During our time working with Cochrane, we have built a new data platform using
domain-driven design techniques which has an explicit information model encoded
into a knowledge graph at its core. The model is published here
[https://data.cochrane.org/ontologies/]. This is complemented by a loosely
coupled, service-based architecture which ensures each business capability in
the value stream is cohesively encapsulated. In a follow up article, my
colleague Paul Wilton will review how the technical architecture hangs together
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
. The platform design allows services to be very quickly deployed in new
choreographies as the business context changes - something which has proved
invaluable over the last few weeks. 

So, what have been the business function and information flow impacts of the
Covid-19 pandemic on Cochrane? Firstly it has created an unprecedented critical
demand for the most up-to-date, reliable evidence to inform frontline clinical
decision-making around the world in order to save as many lives as possible. To
that end, they have been asked by the WHO to produce a precisely scoped,
authoritative register of Covid-19 studies. This essentially inverts their
business model from one based on a wide coverage of clinical questions using
only the highest quality primary evidence to one based on a single clinical
question which urgently extracts every bit of informational value from every
piece of evidence available. That means using all types of RCTs and
interventional studies, as well as observational studies, modelling studies,
diagnostic/prognostic studies and anything else that might improve outcomes. The
surveillance requirement for updates and new evidence switched from months to
days. The cycle time for systematic review production needed to be cut, and a
new Rapid Review format has been developed as a result to ensure baseline
datasets can be pulled together as quickly as possible and then augmented with
streamlined editorial content to meet clinical need. At the same time, the
feedback loop to frontline workers was closed to ensure that their highest
priority clinical questions are the ones being addressed first.

The Covid-19 Study Register [https://covid-19.cochrane.org/] was launched
internally on Monday 30th March and publicly on Monday 6th April. In all, two
back-end platforms were integrated together and a new front-end web application
was designed, built and shipped in just under 3 weeks. It has necessarily been
our fastest ever delivery by some distance, and was only even remotely possible
because of three core enablers:

 1. A data architecture, technical architecture and continuous integration
    infrastructure which minimised the technical risks and impacts of adapting
    to the changing organisational context.
 2. A small number of individually, amazingly talented developers.
 3. The tacit/emergent team structure and pre-existing relationships between
    those developers, established over a long track record of successful
    delivery on other demanding client projects, which meant that as a delivery
    unit they functioned at a truly optimal level from day one.

Since internal launch, our focus has been on getting as much data as possible
into the information model by any and every means we can think of, as quickly as
possible and at the highest possible quality levels. This is covering expert
crowd annotation, machine learning, and other text analysis and informational
reverse engineering techniques. One of the biggest automation/ML challenges
currently is the lack of training data due to the rapid emergence and rate of
evolution of the Covid-19 domain. We have previous experience of this class of
problem from our work with media newsrooms and automated classification of
breaking news stories. It is likely the crisis will drive extensive further
innovation in this space. 

At the same time, we have started receiving comments from stakeholders and end
users across the organisation. It has been some of the most inspiring and
touching feedback we have ever received both as an organisation and as
individuals, especially in light of the wider situation right now. The high
point so far was one very senior Cochrane staff member who, in wonderfully
memorable terms, said he couldn't overstate how useful the register is to all
aspects of his work and that he's now going to shout about it to everyone who
will listen.

A beautiful moment in the midst of demanding times.

Technical Strategy

Covid-19: Rapid Response Technical Delivery at Cochrane

News and updates about Data Language HealthTech products and services, including HealthTech AI, Knowledge Graph and AI Computer Aided Coding Systems (CACS).

News and Updates from experts in HealthTech AI, Knowledge Graph, Product Innovation, Computer Aided Coding Systems (CACS) and more.

HealthTech

HealthTech Blog Posts

News about HealthTech