Over the past five years it has been our privilege to be a technology partner of Cochrane. For anyone unfamiliar with their work, they are globally recognised as the gold standard in evidence-based healthcare. Historically their primary published output has been systematic reviews, which evaluate and summarise the current state of human knowledge about specific clinical questions. These then inform both healthcare policy and clinical guidelines for point-of-care decision making around the world. For example, over 90% of WHO guidelines are informed by Cochrane reviews.
The review production process starts with the formalisation of the clinical question of interest in terms of the Population it relates to (e.g. age, sex, condition), the Interventions being investigated (e.g. drugs and/or procedures), the Comparators against which the Interventions are being evaluated (e.g. placebo), and the Outcomes against which effects are measured. This is then translated into search protocol and run against trial registries and journal databases to identify all relevant clinical study information in the public domain. The matches are then evaluated for risk of bias, and the subset of results which meet Cochrane’s stringent data quality thresholds are subsequently aggregated and analysed to determine the cumulative effect estimates.
In essence, this is about the identification of causal relationships between interventions and outcomes. To that end, up until now Cochrane has focussed on double-blind randomised controlled trials. They provide the highest quality input data: randomisation greatly reduces the likelihood of alternative explanations for the observed results as the number of participants increases; blinding of field researchers and participants (i.e. hiding information about specific interventions) removes the cognitive and behavioural bias risks that might arise if, for example, placebos are known.
Finally, review production is not a static process. New information is being released into the public domain all the time both in the form of additional journal articles about completed studies as well as information about new studies. As a result, ongoing evidence surveillance and timely meta-analysis updates are important additional requirements.
During our time working with Cochrane, we have built a new data platform using domain-driven design techniques which has an explicit information model encoded into a knowledge graph at its core. The model is published here. This is complemented by a loosely coupled, service-based architecture which ensures each business capability in the value stream is cohesively encapsulated. In a follow up article, my colleague Paul Wilton will review how the technical architecture hangs together. The platform design allows services to be very quickly deployed in new choreographies as the business context changes - something which has proved invaluable over the last few weeks.
So, what have been the business function and information flow impacts of the Covid-19 pandemic on Cochrane? Firstly it has created an unprecedented critical demand for the most up-to-date, reliable evidence to inform frontline clinical decision-making around the world in order to save as many lives as possible. To that end, they have been asked by the WHO to produce a precisely scoped, authoritative register of Covid-19 studies. This essentially inverts their business model from one based on a wide coverage of clinical questions using only the highest quality primary evidence to one based on a single clinical question which urgently extracts every bit of informational value from every piece of evidence available. That means using all types of RCTs and interventional studies, as well as observational studies, modelling studies, diagnostic/prognostic studies and anything else that might improve outcomes. The surveillance requirement for updates and new evidence switched from months to days. The cycle time for systematic review production needed to be cut, and a new Rapid Review format has been developed as a result to ensure baseline datasets can be pulled together as quickly as possible and then augmented with streamlined editorial content to meet clinical need. At the same time, the feedback loop to frontline workers was closed to ensure that their highest priority clinical questions are the ones being addressed first.
The Covid-19 Study Register was launched internally on Monday 30th March and publicly on Monday 6th April. In all, two back-end platforms were integrated together and a new front-end web application was designed, built and shipped in just under 3 weeks. It has necessarily been our fastest ever delivery by some distance, and was only even remotely possible because of three core enablers:
Since internal launch, our focus has been on getting as much data as possible into the information model by any and every means we can think of, as quickly as possible and at the highest possible quality levels. This is covering expert crowd annotation, machine learning, and other text analysis and informational reverse engineering techniques. One of the biggest automation/ML challenges currently is the lack of training data due to the rapid emergence and rate of evolution of the Covid-19 domain. We have previous experience of this class of problem from our work with media newsrooms and automated classification of breaking news stories. It is likely the crisis will drive extensive further innovation in this space.
At the same time, we have started receiving comments from stakeholders and end users across the organisation. It has been some of the most inspiring and touching feedback we have ever received both as an organisation and as individuals, especially in light of the wider situation right now. The high point so far was one very senior Cochrane staff member who, in wonderfully memorable terms, said he couldn't overstate how useful the register is to all aspects of his work and that he's now going to shout about it to everyone who will listen.
A beautiful moment in the midst of demanding times.
