Paul Wilton

As a data company, data ethics has always been important to us, but in the last
few years, as Machine learning and AI have become front and centre as a
significant part of digital transformation in the enterprise, data ethics has
become a hot topic. It bears such a considerable weight in predictive analytics,
not just in conveying trust to consumers of machine intelligence, but also in
ensuring predictive models deliver genuine value, make unbiased predictions, and
reflect the true state of the environment within which they live. When data
ethics go wrong, it can go badly wrong, potentially bringing down an entire
organisation, with substantial direct and indirect collateral damage,
potentially even on national democracy and social cohesion - the Cambridge
Analytica incident a case in point.

We all need to take it seriously.

In 2018 Data Language was privileged to have been accepted onto the Digital
Catapult “Machine Intelligence Garage
[https://www.digicatapult.org.uk/for-startups/acceleration-programmes/machine-intelligence-garage]
” programme. As well as supporting us with a terrific programme including
compute resource, strategic help and industry engagement, Digital Catapult also
took Data Language through their ethics programme. This was like a cold-shower,
really waking us up to depth and breadth of data ethics. A realisation that data
ethics is not just about data (technically), but about the way you approach your
entire business. 

With all this in mind, we were motivated to develop a data ethics framework that
we could use internally at Data Language to assess all our predictive analytics
projects, and that could guide us practically during implementation. We call
this framework BOTT.

The BOTT Framework
BOTT (all good frameworks need a memorable acronym) stands for Bias, Outcome,
Target and Trust. Through these four pillars, it allows our data science team to
examine, and assess a predictive analytics model within its local and external
ecosystem for quality, bias, and accuracy, and ensure it meets high standards of
data ethics.

Bias
An analysis of potential bias that may be baked into the model during its
life-cycle, either literally through data or code, or intangibly through for
example, some conscious or subconscious human bias. We examine bias from the
following perspectives :

Training data bias - is the training data diverse enough and comprehensive
enough to accurately reflect the environment of the predictive model. What
potential biases might skew predictive results either for good or bad outcomes -
is there any chance these biases might exist in the training data we have used.

Algorithmic bias - is the model algorithm, its mathematical basis, or code
potentially biased?

Real-time input data - if real-time (predict time) input data is involved in
making some prediction, what might these data biases be, and how can we prevent
them from impacting outputs.

Team bias - has or could conscious or unconscious bias of the model development
team contribute to model bias, and have we mitigated against it. Often just an
open and frank team discussion around bias (with respect to the project under
consideration) is enough to flush out these sort of issues

Political, environmental, and social bias - similar to the team bias but looking
at potential external factors through a similar lens - for both ideally the team
needs to be very comfortable talking about potentially politically or socially
sensitive topics in an open, friendly and non-defensive manner.

Output sampling - essentially QA - have you done effective output sampling and
for quality assurance - does the risk of bias in the predictive model call for
independent third party reviews in QA workflows, etc

Outcome
An analysis and survey of the full range of possible outcomes of the model:

Catalogue - construct a catalogue of the potential outcome scenarios

Impact - What does success and failure of the model mean :

 * in the context of the target use-cases
 * best and worst case scenario modelling
 * model failure consequences, e.g. could somebody die if the model fails ?

Target
Analysis of who or what the model is targeted at and the context in which these
users operate.

Users -  Who are target users of the model  - who or what is the direct consumer
of the model ?

Purpose - What problem is the model solving?

Context: Is it part of an ensemble of models, is there a wider predictive
ecosystem the model sits within - what impact does this model have within this
context ?

Life span -  What is the life expectancy of the model, and how does its
environment change over time

Subversion - Can other users or consumers subvert the model for
unintended/undesirable ends?

Catalogue - Construct  a catalogue of potential unintended uses/consequences,
and ascertain if those vulnerabilities can be designed out of the architecture?

Trust
Providing some visibility of the "behind the scenes" implementation can raise
confidence in the models and foster a trust relationship between consumers and
predictive services.

Explainable AI (XAI) is becoming a field in its own right, and an important part
of machine learning and predictive analytics. Codable frameworks are emerging to
aid explainability (for example Google Model Cards
[https://modelcards.withgoogle.com/about]). This codability will be key in being
able to compare and assess models through some normative evaluation mechanism.

Transparency - what should we be exposing of the lineage and decision making
pipeline of the model prediction algorithm :

 * training cycles / training samples
 * outputs of intermediate predictions
 * decision inputs/outputs

Impact - Why should these be exposed ? - perform an impact analysis of the
possible effects of not exposing model lineage, with respect to legal,
environmental, and model improvement consequences.

Data Ethics

The BOTT Framework for Practical Data Ethics

Jim Howard

Get top tips and insights from Jim Howard on valuable use cases for your Sports Data, and avoid the common pitfalls.

Getting the most from your Sports Data - Insights and Best Practice

My name is Jim Howard and I am a freelance Product Manager specialising in
developing products using data and content. After working at the BBC for a
number of years, I have worked on projects for a number of clients including
Government Digital Service, Eurosport and Stats Perform.

I got to know Data Language after working with some of the team on big
publishing and audience-facing Sport products (World Cups, Olympics etc), when I
was at the BBC and The Telegraph.

We have been discussing some of the issues that we have had to deal with over
the years and the opportunities that are still there for companies to make more
of the data that they consume. In this post, I will try and summarise some of
the key areas we have identified and some areas that could still be developed
further.

Sport, data and content in publishing

Now that the restrictions on Sport are gradually being eased, we thought it
would be an opportunity to review some of the areas where Sport-related products
have been developed and business issues have been addressed with a more
innovative approach to the use of data (not analytics) as a core ‘building
block’ to development. 

As always in organisations, there are lots of great ideas around innovation but
the business needs a focus - and sport is an ideal test subject. As we have
found out, you can look at a major sporting event in isolation (Olympics,
Football World Cup etc), review the outcomes, decide on what areas worked and
then integrate into wider product development strategies. 

I will try and break this down into areas that will be relevant to other
products, not just Sport.

Buying in sports data 
First off, you should look at refocusing Sport as a data proposition and not
just a content one.

Most major publishers typically buy in fixtures, results and tables from a
specialist sports data company. Users, and editorial teams expect that.
Depending on the amount and scale of data, this contract is usually paid
annually and is an ongoing cost. So how do you get maximum value from the data
you are buying?

The data usually contains unique IDs for teams, players and competitions, as
well as additional metadata such as dates, times, locations, rounds
(semi-finals, round 13 etc) and venues. These data feeds also contain names of
players, teams and competitions. A whole wealth of information that we could
exploit much better than we were. 

When we started looking in a lot more detail at the data that was being bought,
you could start to see the opportunities to properly join the dots between the
content proposition, the technical challenges, and what we wanted to build as a
Product. 

It wasn’t perfect the first time round we tried this, although we did get to the
stage very quickly of having a process to tag content and create automated pages
for all players, teams and groups at the 2010 World Cup
[https://www.bbc.co.uk/blogs/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html]
.

We then realised that as Sport is so consistent, once you had a basic domain
model in place, then you could approach development and products in a different
way. 

Working up a Sport domain modelUsing the data to help your tagging 
Workflow is always a challenge, whether you are introducing a whole new system
or just tweaking an existing one. 

From a product perspective, I want consistent and accurately tagged content to
avoid issues with SEO, duplication, URLs and historical inconsistency across
different systems within a business. 

Other business aspects to consider were ensuring we could align the editorial
style guide, improving editorial team efficiency, enabling consistent reporting
of traffic and ad revenue, allow for cleaner syndication of content and at
times, manage tags in multiple languages (Bayern Munich and Bayern München are
the same team), and managing alternate names and synonyms for your tags.

When we first introduced manual tagging of content, we had issues with
consistency and accuracy. Even with guidance, individuals approached relevance
and accuracy in slightly different ways. 

Taking feedback from this, our next iteration was a semi-automated tagging
process using NLP techniques, where we focussed tagging only on headlines and
first 4 or 5 paragraphs of the content. The copy was cross referenced with the
sport vocabulary we had created from the bought in data and tags suggested to
journalists. 

This created a much quicker and accurate workflow for the editorial teams. 

There were unexpected challenges too, such as when a new CMS allowed creation of
multiple tags by content teams, with no editorial oversight on tag quality. The
CMS allowed audience-facing pages to be created automatically once a tag had
been used several times - even if they were incorrect. This caused major issues
for the editorial team, the SEO strategy and User Experience. 

We revisited the key terms around sport that the data supplier had delivered,
used that as a starting point for a controlled vocabulary for the editorial
team, and then used machine intelligence to automatically retag the archive of
existing articles, video and other content in the CMS, to ensure consistency. 

Linking a content model and data 
Another key part of this process is understanding the editorial workflow around
fixtures and events and where a level of automation can help save time and
effort. 

There is a considerable amount of consistency around how events are approached
within newsrooms, with editorial teams creating match previews, live coverage,
reports, photo galleries, post-match analysis etc, all of which fit within a 
content model
[https://www.cmswire.com/content-strategy/content-modeling-what-it-is-and-how-to-get-started/] 
and data strategy approach. This thinking equally applies to video and audio,
live and clips. 

As part of a workflow review we could see that the editorial teams were taking
time to manually create pages in the CMS in the days before a big weekend of
sport. 

We suggested using the bought in fixture list to drive automated creation of
pages in the CMS for journalists. We identified what templates needed creating
around a match - which links nicely with the content model work - and spun-up
pre-tagged templates in the CMS when the fixtures were announced. 

Before a match starts, we already know the teams, competition, venue, historical
records etc.  Depending on the sport, we will also know the athletes taking part
in a match (football tends to announce starting line-ups about 45 mins before
kick off) although other sports make announcements further in advance. 

Silver Oliver’s workshop flipchart diagram of editorial workflowScaling Up
This data-driven approach allows you to scale quickly. New competitions or
events can be approached from a ‘we just need some new data’ perspective and
this should fit with your ongoing data management strategy. 

This is especially relevant around major events where you may want to cover new
teams or events as part of the Olympics, Commonwealth Games or research has
shown you that your audience wants more detailed club level NFL or NRL coverage. 

If you want to create a new type of written content, such as a match report,
that is an extension of your existing content model. 

When you start adding in multiple media types used for coverage such as
photographs and social media, you can begin to see where consistency of tagging
and annotation is key to creating product propositions. 

These methods can also be used for content gathering as well - from external
suppliers to content supplied by correspondents and social media sources. 

Data visualisation
One of the challenges every media business has, is how do you stand out from the
crowd around Sport? Everyone has the same fixture list (that you have no control
over) therefore a focus on the user experience and a unique proposition helps
you stand out. 

Depending on the amount of data that you are buying, there is a lot of mileage
in offering a unique insight, and deeper analysis into the events or
competition. 

I worked on the Roboblogger project
[https://www.niemanlab.org/2016/06/the-telegraph-is-trying-to-streamline-soccer-live-blogging-with-an-automated-graphic-system/]
, part funded by the Google Digital News Innovation fund
[https://newsinitiative.withgoogle.com/dnifund/]. We really dug into the details
of what was in the football data that we bought and pushed the automated
publishing of infographics as far as we could.

It helped that we had already put in place a controlled vocabulary of sport
terms, which made the consistency between systems much easier. 

We ensured that any graphics created fitted with the visual style guide of The
Telegraph and were getting them published in under a few minutes after we
received data about the events. 

It’s not all about the website or app
Once you start digging into how a publishing business works, you can start to
see how progress could be made by thinking about an ‘open data strategy’ within
your organisation and allowing different systems to use the same vocabulary to
make life easier, although that does start changing the business focus to manage
data better - still a very overlooked area at the moment.

There are still multiple opportunities to join the data dots in businesses.
Especially linking back end systems and archiving with audience facing products.

 * Unlocking content archives. Always handy for Olympics, World Cups, and major
   events.
 * Ensuring you can map multiple, external suppliers to a single tag or ID.
 * Advertising and commercial consistency for teams and competitions and even
   individuals athletes.
 * Providing consistency between content and tags after mergers between
   companies.
 * Electronic Programme Guides (EPG).
 * Video tagging - enhancing video with structured data - both your own video
   and inbound social media video
 * Analysis and reporting.

Sport - or even an individual sport such as football - is a very consistent
domain which makes it easier to integrate into your business model, or at the
very least, look at Sport, as your first step towards utilising data and content
in a structured and consistent way.

The M word
I have avoided writing metadata all the way through this so far. 

Data suppliers provide very consistent products and can help you solve some
business and product issues. Linking together data and content to create better
products both for internal processes and audience-facing or syndication
purposes.

For example, you shouldn't have to rely on editorial teams to rebuild a section
or feature. We can do this based on the data we have. If google enables a new
rich snippet type we can add this markup etc. 

Most media organisations buy in content feeds, and although the assets are
tagged, they won’t be quite the same as the vocabulary that you use within your
organisation. Clarifying that vocabulary will save time and money between
systems and your own team.

The key to structured data is a well structured content model, a data management
strategy and metadata.

If you want to extend the model, then you can have a list of your contributors
and journalists as well. This would enable me to create a feed or even a
subscription based-product based on a question such as:

> “Give me all match reports (content type) written by Joe Bloggs (Author) about
Melchester Rovers (team) in the past 6 months (date range)”
Are you making the most of your bought data and content?
Data management is a challenge and unfortunately it does tend to get lumped in
with data science and analytics within a business.

I prefer to look at the reference data as a core building block for the Product.
I want to look at the details of the data contract to see what is being bought.
Does it fit with the current editorial strategy or is it just a historical add
on? Is this what the audience is looking for or expecting? Do they come to your
site or buy your paper for a specific type of coverage?

The cost of the data contract varies from company to company and of course also
depends on the amount of data bought. One thing in my experience that is
consistent though, is that usually only a small amount of the data is being used
by the business.

The coronavirus pandemic has caused a raft of fixture cancellations,
rearrangements, postponements and economic challenges across the board for
industries associated with sport. 

Meeting new business challenges is going to be about agility and speed of
response to new opportunities. 

Companies will have to be more agile in the future and the key to this agility
is having scalable and well marked up content that can be repurposed. 

The future
Hopefully 2021 will be a big year for Sport with a still to be defined weekly
sporting calendar, a rescheduled Olympics in Japan, a rescheduled European
football championships, a Lions Tour to South Africa and lots more. 

Companies will want to make the most of these opportunities for engaging
audiences and hopefully these examples can start some processes we have been
involved in:

 * Advertising and commercial consistency - for teams and competitions and even
   individual athletes
 * Unlocking under utilised content archives
 * Providing consistency across content and data after mergers between
   organisations
 * Electronic programme guides - a key syndication product and internal tool for
   some organisations
 * Video clips and live coverage

Around major events, other departments within your organisation that don’t
usually do sport coverage, may well want to start creating content about the
event such as documentaries etc that you may want to leverage.

Many of these ideas are not just relevant to Sport, the patterns I describe can
be applied to many domains and publishing any form of content, whether that is
in news, sport, entertainment, healthcare, science, or finance. Adding
geospatial metadata into the mix adds another dimension, and more avenues for
adding value.

Every domain has its own challenges but Sport is a very nice place to start. If
you would like to know more, please do get in touch.

Data Language is an industry leader in Information Architecture, ontology design and practical knowledge graph implementation. We use Domain Driven Design practises and user-centred design to focus information and data design on core business needs.

Find Information Architecture news and product innovation updates here, including ontology design, vocabulary management, domain & data modelling and more.

Information Architecture - News, Product Innovation and Technology

Information Architecture

Getting the most from your Sports Data

A few weeks ago Cochrane launched their COVID-19 Study Register and I described
the data and technical architecture in detail here
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
.

I described how PICO graphs [https://data.cochrane.org/ontologies/] are used to
augment the Studies with structured linked data that describe the Population,
Intervention, Comparator and Outcome of the trial. Cochrane have now started in
earnest annotating PICO graphs onto the Studies, and these are now being
surfaced in the Study Register application [https://covid-19.cochrane.org/].

Here, we can see the labels of the PICO concepts rendered in the UI, but in the
data itself, these are arranged as a PICO graph structure conforming to the 
PICO
ontology [https://data.cochrane.org/ontologies/pico/index-en.html].

We have also added faceted search by PICO concept into the sidebar. This lets
users quickly filter down their search results using the Cochrane vocabulary
terms.

Once we have a critical mass of PICOs annotated in the register, each vocabulary
concept will be searchable in the search bar at the top of the app, extending
the previously full-text search into a controlled-vocabulary search. A user can
now find studies using a combination of terms chosen from the Cochrane linked
data vocabulary, to deliver extremely accurate results. Moreover we can combine
a metadata driven search with full-text where needed.

Schema.org
Each Study page rendered in the application is also marked up with schema.org
[https://schema.org]structured data. So while internally we are describing our
content and PICO data with the Cochrane bounded context ontologies, at render
time, we are transforming our model to the schema.org MedicalEntity model to
provide machine readable content for search engines and other content consumers.
You can read more about the schema.org medical schema here.
[https://schema.org/docs/meddocs.html]

As an example, for this Studyhttps://covid-19.cochrane.org/studies/crs-13266121 
when viewing the source of the page, you can see we are marking up the clinical
trial as a MedicalStudy in the schema.org ontology, and identifying the 
healthCondition terms, and the interventions, including the Drug Chloroquine.
Each term has associated medical codes mapped to bioinformatics taxonomies
including SNOMED-CT, MeSH, RxNorm and the ATC codes for Drugs where available.
The resulting schema.org JSON-LD for this study is :

{
  "@context": "http://schema.org",
  "@graph": [
    {
      "@type": "MedicalStudy",
      "id": "http://data.cochrane.org/studies/crs-13266121",
      "name": "NCT04333628",
      "identifier": "NCT04333628",
      "description": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
      "mainEntityOfPage": {
        "@type": "WebPage",
        "url": "https://covid-19.cochrane.org/studies/crs-13266121",
        "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
        "description": "Clinical study : Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial"
      },
      "healthCondition": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        }
      ],
      "studySubject": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        },
        {
          "type": "MedicalProcedure",
          "id": "http://data.cochrane.org/concepts/r4hp5z141f1t",
          "url": "https://data.cochrane.org/concepts/r4hp5z141f1t",
          "name": "Usual Care",
          "alternateName": [
            "Routine Prenatal Care",
            "Routine Care",
            "Standard Care",
            "Providing care according to standard",
            "SOC"
          ],
          "code": [
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "372921003"
            }
          ]
        },
        {
          "type": "Drug",
          "id": "http://data.cochrane.org/concepts/r4hp13k8jxft",
          "url": "https://data.cochrane.org/concepts/r4hp13k8jxft",
          "name": "Chloroquine",
          "alternateName": [
            "Chloroquine",
            "Product containing chloroquine"
          ],
          "code": [
            {
              "codingSystem": "MeSH",
              "codeValue": "D002738"
            },
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "14728000"
            },
            {
              "codingSystem": "RxNorm",
              "codeValue": "2393"
            },
            {
              "codingSystem": "ATC",
              "codeValue": "P01BA01"
            }
          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "name": "Pharmacological Interventions"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "name": "Other"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        }
      ],
      "subjectOf": [
        {
          "@type": "MedicalScholarlyArticle",
          "datePublished": "2020-01-01",
          "url": "https://clinicaltrials.gov/show/NCT04333628",
          "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19",
          "abstract": "19 COVID is a deadly viral disease that has been spreading around the world for several months, and is caused by a CORONA family virus (COVID-19). Following IN-VITRO evidence of the antiviral effect of CHLOROQUINE in CORONA viruses, this drug has been used empirically for COVID-19 patients and is currently recommended in Israel for the treatment of intermediate and severity disease. The mechanism of action of chloroquine is in part by inhibiting the virus distribution, and changing the intracellular acidity, the virus distribution site. The intracellular chloroquine concentration is determined by a pump called PGP that removes the drug from the cell and is activated by the drug. In the treatment of malaria, the benefit of low dosage of the drug has been shown to be effective due to the fact that the intracellular concentration of the drug is probably higher, and therefore the logic to examine this issue in COVID-19 treatment. The purpose of this study is to test whether a low dose of Chloroquine will reduce the duration of the viral shedding and prevent the disease from worsening",
          "author": [
            {
              "type": "Person",
              "name": "HaEmek Medical Center"
            }            
          ]
        }
      ]
    }
  ]
}

PICO and Schema.org
The PICO model and process (https://en.wikipedia.org/wiki/PICO_process) is now
widely adopted for framing research questions in evidence based healthcare with
the BMJ citing this model as best practice
[https://bestpractice.bmj.com/info/toolkit/learn-ebm/how-to-clarify-a-clinical-question/]
.

PICO data is applied by clinicaltrials.gov in a semi-structured tabular format
on most of their published trials and PubMed also has a PICO search tool for
querying MEDLINE, the  database from the US National Library of Medicine.

Supporting PICO would be a welcome addition to the schema.org model, allowing
any publisher of medical literature to mark up their content in a standard way
with machine readable structured PICO data as Cochrane and others are now. I can
imagine a future where Google indexes the schema.org PICO markup and provides
the ultimate PICO based search tool across all the medical literature citations
at their disposal.

At Data Language, as co-authors of the PICO Ontology, we would be more than
happy to help define some simple extensions to the schema.org MedicalEntity
model to make this happen.

PICO, Cochrane and Schema.org

A knowledge graph is a collation of information and data typically within the domain of your business context. The information is stored in a database, often a graph database such that it can be queried, maintained, and interacted with by humans or machines. We have exceptional experience in the design, implementation and deployment of knowledge graphs. We have been working with graph databases since our inception and have delivered knowledge graphs solutions for global organisations.

We have exceptional experience in the design and implementation of knowledge graphs and have delivered knowledge graphs solutions for global organisations.

Knowledge Graphs - our Work, Features, News and Updates

Knowledge Graphs

How knowledge graph technology is helping Cochrane respond to COVID-19

Julian Everett

Over the past five years it has been our privilege to be a technology partner of 
Cochrane [https://www.cochrane.org/]. For anyone unfamiliar with their work,
they are globally recognised as the gold standard in evidence-based healthcare.
Historically their primary published output has been systematic reviews, which
evaluate and summarise the current state of human knowledge about specific
clinical questions. These then inform both healthcare policy and clinical
guidelines for point-of-care decision making around the world. For example, over
90% of WHO guidelines are informed by Cochrane reviews. 

The review production process starts with the formalisation of the clinical
question of interest in terms of the Population it relates to (e.g. age, sex,
condition), the Interventions being investigated (e.g. drugs and/or procedures),
the Comparators against which the Interventions are being evaluated (e.g.
placebo), and the Outcomes against which effects are measured. This is then
translated into search protocol and run against trial registries and journal
databases to identify all relevant clinical study information in the public
domain. The matches are then evaluated for risk of bias, and the subset of
results which meet Cochrane’s stringent data quality thresholds are subsequently
aggregated and analysed to determine the cumulative effect estimates. 

In essence, this is about the identification of causal relationships between
interventions and outcomes. To that end, up until now Cochrane has focussed on
double-blind randomised controlled trials. They provide the highest quality
input data: randomisation greatly reduces the likelihood of alternative
explanations for the observed results as the number of participants increases;
blinding of field researchers and participants (i.e. hiding information about
specific interventions) removes the cognitive and behavioural bias risks that
might arise if, for example, placebos are known. 

Finally, review production is not a static process. New information is being
released into the public domain all the time both in the form of additional
journal articles about completed studies as well as information about new
studies. As a result, ongoing evidence surveillance and timely meta-analysis
updates are important additional requirements.

During our time working with Cochrane, we have built a new data platform using
domain-driven design techniques which has an explicit information model encoded
into a knowledge graph at its core. The model is published here
[https://data.cochrane.org/ontologies/]. This is complemented by a loosely
coupled, service-based architecture which ensures each business capability in
the value stream is cohesively encapsulated. In a follow up article, my
colleague Paul Wilton will review how the technical architecture hangs together
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
. The platform design allows services to be very quickly deployed in new
choreographies as the business context changes - something which has proved
invaluable over the last few weeks. 

So, what have been the business function and information flow impacts of the
Covid-19 pandemic on Cochrane? Firstly it has created an unprecedented critical
demand for the most up-to-date, reliable evidence to inform frontline clinical
decision-making around the world in order to save as many lives as possible. To
that end, they have been asked by the WHO to produce a precisely scoped,
authoritative register of Covid-19 studies. This essentially inverts their
business model from one based on a wide coverage of clinical questions using
only the highest quality primary evidence to one based on a single clinical
question which urgently extracts every bit of informational value from every
piece of evidence available. That means using all types of RCTs and
interventional studies, as well as observational studies, modelling studies,
diagnostic/prognostic studies and anything else that might improve outcomes. The
surveillance requirement for updates and new evidence switched from months to
days. The cycle time for systematic review production needed to be cut, and a
new Rapid Review format has been developed as a result to ensure baseline
datasets can be pulled together as quickly as possible and then augmented with
streamlined editorial content to meet clinical need. At the same time, the
feedback loop to frontline workers was closed to ensure that their highest
priority clinical questions are the ones being addressed first.

The Covid-19 Study Register [https://covid-19.cochrane.org/] was launched
internally on Monday 30th March and publicly on Monday 6th April. In all, two
back-end platforms were integrated together and a new front-end web application
was designed, built and shipped in just under 3 weeks. It has necessarily been
our fastest ever delivery by some distance, and was only even remotely possible
because of three core enablers:

 1. A data architecture, technical architecture and continuous integration
    infrastructure which minimised the technical risks and impacts of adapting
    to the changing organisational context.
 2. A small number of individually, amazingly talented developers.
 3. The tacit/emergent team structure and pre-existing relationships between
    those developers, established over a long track record of successful
    delivery on other demanding client projects, which meant that as a delivery
    unit they functioned at a truly optimal level from day one.

Since internal launch, our focus has been on getting as much data as possible
into the information model by any and every means we can think of, as quickly as
possible and at the highest possible quality levels. This is covering expert
crowd annotation, machine learning, and other text analysis and informational
reverse engineering techniques. One of the biggest automation/ML challenges
currently is the lack of training data due to the rapid emergence and rate of
evolution of the Covid-19 domain. We have previous experience of this class of
problem from our work with media newsrooms and automated classification of
breaking news stories. It is likely the crisis will drive extensive further
innovation in this space. 

At the same time, we have started receiving comments from stakeholders and end
users across the organisation. It has been some of the most inspiring and
touching feedback we have ever received both as an organisation and as
individuals, especially in light of the wider situation right now. The high
point so far was one very senior Cochrane staff member who, in wonderfully
memorable terms, said he couldn't overstate how useful the register is to all
aspects of his work and that he's now going to shout about it to everyone who
will listen.

A beautiful moment in the midst of demanding times.

Technical Strategy

Covid-19: Rapid Response Technical Delivery at Cochrane

Steve Liles

React and SVG make a powerful combination: Declarative UI component library
meets declarative graphics language — a match made in front-end developer
heaven.

In this mini-series we’ll be looking at rendering interactive graphical UI’s
using components built with React and SVG.

These days SVG icons are ubiquitous, but those are not the sort of components
we’re talking about.

We’re going to put together the interactive floor-plan UI using simple React
code that renders SVG elements instead of HTML. We’ll take it relatively slowly,
building through the basics into more advanced topics.

 * Part 1 — Declarative Graphics (this post)
   In which we look at the match made in heaven that is React and SVG
 * Part 2 — Arcs, Angles, and Transformations
   Keeping geometry simple with groups and transformations
 * Part 3 — Clip-paths
   An adventure in composite geometry
 * Part 4 — Masks
   When clip-paths don’t cut it: masks to the rescue
 * Part 5 — Animating components 
   Where we animate individual components
 * Part 6 — Animating the viewBox 
   In which we fluidly zoom and pan our entire UI

Let’s dive straight in!

Declarative Graphics
React developers enjoy first-class support for HTML elements in JSX:

const SomeComponent = () => (  
  <div>
    <p>hi!</p>
  </div>
);

But did you know that this applies to inline SVG elements too?

const SomeOtherComponent = () => (
  <svg viewBox="0 0 100 100">
    <circle cx={50} cy={50} r={30} fill="red"/>
  </svg>
);

Out of the box we can write  components that compose SVG primitives to create
complex interactive UI’s, just as you would with more typical HTML based
components.

(Note for React-Native folks: you’ll need to install react-native-svg, which was
looking pretty good last time I worked with it. Here
[https://stackoverflow.com/a/39041997/473338] is a simple but pretty powerful
example I created a few years ago).

Simple composition of declarative graphical components (https://62zqd.csb.app/)
Notice that we did all this drawing without a single line of imperative code,
and the declarative markup of our ghosts is very readable and accessible to
anyone familiar with React — a far cry from what this would look like if we
tried to do it with the Canvas API.

All your elements are belong to us
To draw anything with SVG, we first need an <svg>element to define the drawing
context within which all descendant elements are rendered.

We can have many <svg> elements in our page, each describing an isolated drawing
context — e.g. one <svg> per icon, chart, or other snazzy graphical widget.

Alternatively, we can root many graphical components within the same <svg> and
create an entire UI in a single drawing context.

Viewport and ViewBox
At a minimum an <svg> element should specify how much screen-space it will
occupy — its viewport. This can be done with CSS styles or width and height
attributes.

<svg width="200px" height="200px">...</svg>

We’ll typically want to specify a viewBox too, to explicitly define the
“user-space” portion of our drawing that is made visible in the viewport.

Think of the viewport as a window into the world defined by the SVG
(“user-space”), and the viewBox as zoom and pan settings that control what parts
of the user-space you can see through the window.

The viewBox attribute is a string which specifies, in order, the user-space x
and y coordinates of the top-left visible corner, and the width and height of
the viewBox.

<svg viewBox="x y width height">...</svg>

For example, “-100 -100 200 200” describes a square viewBox 200 units high and
200 units wide, centred around the origin at (0,0).

<svg width="10" height="10" viewBox="-100 -100 200 200">...</svg>

If we don’t specify it explicitly we get a viewBox that maps width and height
1:1 to the viewport, with the user-space origin (0,0) at the top-left corner.

To re-iterate: the viewport describes how much space the <svg> element takes up
within your HTML page. The viewBox describes the portion of the image that is
visible within that viewport. By changing the viewBox we can zoom in or out, or
show entirely different portions of the SVG image.

The only difference between these two SVG’s is the viewBox and the background
colour (https://0oy6p.csb.app/)If the proportions of your SVG viewport in pixels
don’t match the proportions of the viewBox, the default behaviour is to preserve
the aspect ratio and centre and fit the viewBox within the available viewport.
This behaviour can be controlled by varying the preserveAspectRatio
[https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/preserveAspectRatio] 
attribute.

By default the viewBox will be centred and “fit”within the SVG viewport (
https://52fbc.csb.app/)The separation of coordinate systems between the viewBox
and the viewport means that within our SVG we can work with coordinates that
makes sense for our specific drawing.

For example, if we’re drawing an interactive widget we can make the maths easy
for ourselves by working in percentages and specifying a viewBox of “0 0 100
100”.

Or if our widget has radial symmetry and we want to easily draw from the centre,
we can use a viewBox of, say, “-50 -50 100 100” centred at the origin (0,0).

In this series we’ll be drawing floor-plans, so we could choose to work in
metres, or feet-and-inches, or whatever system of measurement our plans use.
We’re actually going to use millimetres, just for the simplicity of working in
whole numbers.

We’ll describe our floor-plan with some simple JSON, specifying the room
geometry as coordinates of corners working clockwise around the room.

Now we can set up our viewport and viewBox, allowing a little padding around the
floor-plan so it isn’t too snug in the viewport.

import data from './floorplan-data.json';
const App = () => (
  <svg 
    width="500px"
    height="500px"
    viewBox="-1000 -1000 14000 11000"
    style={{backgroundColor:'blue'}}
  >
    // ... floorplan components here ...  
  </svg>
);

With our viewport and viewBox sorted, we can get on with drawing.

Declarative Drawing Primitives
SVG includes a number of drawing primitives ranging from the simple <line> and 
<circle> to the more flexible <path>. Mozilla Developer Network has a good 
reference [https://developer.mozilla.org/en-US/docs/Web/SVG/Element] of the
available elements.

Shapes can be filled and/or stroked, where fill is the colour applied within the
bounds defined by the shape, and stroke is the colour applied to the outline of
the shape. The width and other styles of the stroke can be controlled with
attributes or CSS.

We’ll be drawing our initial floor-plans using two simple primitives: <line> and 
<circle>.

<line> describes a line between a pair of (x, y) coordinates.

<line x1={100} y1={100} x2={200} y2={200} stroke="red"/>

<circle> describes a circle of radius r centred at (cx, cy).

<circle cx={50} cy={50} r={20} fill="green" stroke="black"/>

Room with a View
To draw our building we’ll pass the data into a Floorplan component which will
render out the various elements of the floor-plan — initially just the rooms.

const Floorplan = ({ data: { rooms } }) => (
  rooms.map(r => <Room {...r} />)
);

For the Room component to draw the walls we’ll need to map over pairs of
consecutive corner coordinates, linking them together to form walls. For example
a rectangular room with corners a, b, c, and d, will have four walls
(a-b),(b-c), (c-d), (d-a).

We can pull out those pairs with a simple function, perhaps wrapped in a useMemo 
hook for efficient re-renders.

const walls = useMemo(() =>  
  coords.map((_, i) => {
    const a = coords[i];
    const b = coords[(i + 1) % coords.length];
    return [a, b];
  }),
  [coords]
);

Now drawing the walls is as simple as using the SVG <line> element to describe a
line between the pairs of connected corners.

That works, but it isn’t very pretty. Let’s extract a Wall component and a 
Corner component, and get a bit more creative.

In our wall component we’re now drawing two lines per wall — first a thick white
line, then over-drawing the white line with a thinner, dark blue, dashed line.

Corner is very simple — just a circle, stroked white and filled dark blue.

The Room component now simply composes these higher-level Wall and Corner 
components instead of SVG primitives.

https://codesandbox.io/s/react-svg-basics-rk1t3You may have noticed that we
nested our Walls and Corners in an SVG <g> element. This is a logical grouping
element — the <g> on its own doesn’t render anything visible — but it does give
us a neat way of performing transformations such as rotations and translations
on entire sub-hierarchies of our SVG, and a means of capturing events from those
sub-hierarchies by attaching event handlers to the <g> element.

Recap
In this post we saw that:

 1. We can easily draw arbitrary shapes using SVG primitives and
    straight-forward, declarative React code — no plugins, libraries, or
    imperative code required.
 2. Complex graphical UI’s can be built by composing together increasingly
    higher level components, just like any other React application.
 3. The separation of viewBox and viewport lets you draw in a coordinate space
    that suits you, then render the result in a viewport appropriate to the
    final display requirements.

In “Part 2 — Arcs, Angles, and Transformations”, we’ll dig further into those 
<g> elements and look at paths and transformations while adding camera’s to our
floor-plan.

Until next time, happy drawing!

Software Development

Graphical UI's with SVG and React,  part 1 - Declarative Graphics

5 latin characters
Until recently, my understanding was that apart from 5 latin characters GraphQL
has got little or nothing to do with Graph Databases. But things are changing,
Graph Database vendors have started bringing these two technologies together. A
recent project has given me some insights.

For some background, GraphQL is a contemporary API interface standard used as an
alternative to REST. GraphQL was originally developed by Facebook. It provides a
structured query language for querying (and mutating) data, along with a runtime
implementation to invoke those queries upon your data. In pretty much all
implementations it uses JSON as its serialization format.

REST (I suspect most readers know this) is a lightweight low-level API standard
for performing uniform and a predefined set of stateless operations on data
aligned with the HTTP verbs. With REST you have to design and  build your API
interface contract for that set of operations - and while of course there are
common patterns in REST APIs, this is very much arbitrary. REST makes no
assumptions about the high level API contract or the underlying data model. The
developer defines both. With GraphQL you only define the data model (the GraphQL
schema), exposing to the GraphQL runtime and your consumers what the shape of
the data is.

With a GraphQL query you can request only the minimum data required to fulfil
your current need. Of course you can do this with a REST API too - in fact it is
a common pattern with JSON REST APIs to filter response data by JSON fields
using dot notation. But of course you need to build this yourself.

GraphQL does a lot more for you to boot, as well as out of the box query
patterns for mutating data, it also supports subscriptions (pub/sub style) - so
your GraphQL client can subscribe to server side events, and have data pushed to
it.

The developer ecosystem around GraphQL is superb now - The Apollo GraphQL
[https://www.apollographql.com] implementation works beautifully with React -
integrating with React Hooks for query and subscriptions. Apollo also has
in-browser developer tools akin to the redux-devtools for GraphQL debugging, and
you can manage local app state via Apollo using the GraphQL schemas / queries
etc.

Caching and performance
Requests to GraphQL are typically all POST requests - not great for caching you
ask ? Indeed, this is where REST has excelled for years. But the GraphQL server
implementations have come to the rescue. The Apollo GraphQL server has an
internal caching layer that supports full response caching, and fine grained
field-level caching (read more here
[https://www.apollographql.com/docs/apollo-server/performance/caching/] )

Amazon’s AWS AppSync is a fully managed GraphQL service built upon the Apollo
technologies - and can source data out of the box from other services such as
DynamoDB, ElasticSearch, REST APIs etc. With fully managed caching and real-time
data subscriptions, it really does make your developer life easy.

On the surface it still appears GraphQL has little to do with Graph Databases.
Yet increasingly we are hearing about how they should go together like bread and
butter, sausage and mash.

Not all graph databases are equal
I won't go into depth about what a Graph Database is, but am going to explore
some of the options available, and the relationship between Graph Databases and
GraphQL.

Graph databases come in several flavour combinations. First up you have RDF
graphs vs Labelled-Property graphs. The first big choice. Both are very
different [https://en.wikipedia.org/wiki/Graph_database#Graph_models]. Your
primary use case is important here. Essentially if you primarily need to walk,
query and analyse paths through your graph, then Labelled-Property graphs are
probably the way to go. If you are more interested in the nature of the
relationships between things, and creating rich semantic representations of
things, then RDF graphs hit the spot. Both types have their own well-developed
query language. Labelled-Property graphs are typically queried with Cypher or
Gremlin (eg Neo4J), RDF graphs are queried with SPARQL.

While most mainstream RDF based graph databases are fully compliant to the RDF
and SPARQL W3C standards, not all RDF graph databases are equal. Each has its
own implementation nuances and come with their own added extras (decisions,
decisions!):

 * An RDF graph database may or may not support reasoning (to some degree of the
   RDFS and OWL standards).
 * Reasoning may be at query time (backward chaining) or at write-time
   (forward-chaining).
 * The way reasoning is implemented can dramatically impact performance.
 * Add-ons and extensions for GeoSPARQL, Path analysis, Full text search,
   connections to other datastores etc.

To give some examples (DISCLAIMER - apols to those vendors not mentioned, I am
only referencing those I have worked with properly hands-on, and this is not an
exhaustive features list - it’s just a flavour) :

GraphDB [https://www.ontotext.com/products/graphdb/] (Ontotext)

 * Fully compliant with RDF/SPARQL standard;
 * Highly compliant and configurable OWL / RDFS forward-chaining reasoning
   (materialises inferred knowledge at write time);
 * Unparallelled full-text-search capability using lucene, or via bi-drectional
   connectors to external SOLR or ElasticSearch clusters.
 *  Ability to materialise post-inference fully structured JSON or JSON-LD
   objects into ElasticSearch indices or MongoDB
 * Geospatial querying and other extensions..
 * Rich visual query and management workbench UI

Neptune [https://aws.amazon.com/neptune/] (Amazon AWS)

 * Fully compliant with RDF/SPARQL standards;
 * No reasoning capability
 * Fully managed cloud service
 * Highly, highly horizontally scalable (primarily due to lack of reasoning)
 * Full text search capability via connector to ElasticSearch - materialisation
   into elastic is constrained (you cannot materialise user-modelled JSON
   objects into an elastic index), but delivers good full-text search utility.
 * You use this either as a Labelled-Property graph (using Gremlin) or an RDF
   graph (with SPARQL)
 * No query workbench/admin UI - you need to install your own 

Stardog [https://stardog.com]

 * Fully compliant with RDF/SPARQL standards;
 * Highly compliant and configurable OWL / RDFS backward-chaining reasoning
   (materialises inferred knowledge at query time);
 * Extremely fast online write times (due to backward-chaining model)
 * Adequate full-text-search capability using internal indexing
 * An outstanding SPARQL extension for PATH analysis (Pathfinder) - you can
   query for paths, shortest paths with weighted edges, normally only possible
   with labelled-property graphs
 * Geospatial querying and other extensions..
 * Rich visual query and management workbench UI

So you can see that each of these three are fully W3C compliant RDF / SPARQL
graph databases. But each has one or more killer features, that the others do
not. 

GraphDB - the post-inference connectors to ElasticSearch and MongoDB unlock some
outstanding technical and data architecture patterns out of the box.

Stardog - combines the semantics of RDF with sophisticated Path querying and
analysis.

Neptune - fully managed cloud service with unparalleled horizontal scalability,
but no reasoning.

The upshot - choose wisely based on your requirements now, and what is likely to
matter most in the medium term.

..and then enter stage right, GraphQL

Sausage and Mash or Pineapple on Pizza ?
How do GraphQL and Graph Databases fit together? As we learned earlier, GraphQL
is a REST API alternative with a structured query language combined with a
runtime for query processing and serving data to GraphQL clients (and it is
almost always implemented with JSON in mind).

So why are Graph Database vendors getting involved and what do you get for your
money?

The cynical marketer in me says those 5 latin characters definitely count for
something. Which sales manager wouldn’t love the potential energy pressure in
the ready-to-explode singularity of Graph Databases and GraphQL (while humming
‘Don’t stop me now’).

However, GraphQL does only provide another query mechanism for your graph
database alongside SPARQL or Gremlin.  Moreover it is much less expressive, and
most definitely not designed for exploiting the rich semantics of RDF and OWL. 

In fact if you go all in on a michelin-starred Graph Database, and only use
GraphQL to query it, I would argue you really probably don’t need a Graph
Database - a NOSQL JSON store would probably suffice. Of course, if you have
forward-chaining inference in your graph database (as with GraphDB), then your
GraphQL queries will benefit from the RDFS/OWL semantics you have modelled your
data with, which does add weight to the value-proposition.

Still hungry ?
Ontologies are an important part of the picture. In the RDF world an ontology
semantically describes the model and nature of your domain, and ultimately the
data structures stored in your knowledge graphs, and if inference-aware, they
describe the meaning of the data, and how we reason upon it. Typically these
models are communicated up and down our tech stack (and the responsible
developers), and often all the way out to consumers of your APIs (public or
enterprise-internal).  While we typically move data around using JSON the
linked-data aficionados will know well that if we do this using JSON-LD, we can
use the same ontology models from the knowledge graph (or a subset) for
describing the JSON-LD payloads we ship around outside of our graph database.

So do we need GraphQL to take advantage of these ontological graph models ?.  No
we don’t, we can do it with normal REST APIs (querying our graph with SPARQL)
and using JSON-LD payloads - but of course we have to build more API code to do
this.

With GraphQL though we can (we must) formally describe our data structures as
GraphQL schemas, and so we have a great opportunity here to align our GraphQL
schemas with those same ontology models that describe the data in our knowledge
graph -  this could be either a subset of our ontology landscape, or possibly
the superset, if we are using other GraphQL (served) data sources aside from the
data in the graph database. This is starting to look like very nice decoupled
data architecture with well structured cohesive data models, with possibly the
most efficient mechanism for consuming data in our web/mobile applications (a
GraphQL client).

It is looking more tasty
Adding GraphQL client interfaces to a Graph Database does make life easier - if
you are rapidly prototyping straight from the graph, or federating data into an
external GraphQL server there is a nice pattern for making a common set of well
described data models.

The Graph Database vendors have indeed jumped on-board the GraphQL sushi train
(and it’s not all about those 5 latin characters). Stardog and GraphDB (from
Ontotext) both offer GraphQL clients now.

At this point I was going to show an example of how GraphQL might be used to
query a Graph Database, but Jem Rayfield at Ontotext has already done the job in
his superb Star Wars blog post
[https://www.ontotext.com/blog/the-rise-of-the-knowledge-graph/] (if you have
got this far, then this is a must read).

Ontotext has taken the next logical step and created a ‘Semantic Object Service’
 that automatically generates GraphQL schemas bound to the underlying RDF data
model  in GraphDB, moving us towards the ontology ubiquity I described above,
where your domain ontologies are expressed in the knowledge graph, and across
your wider data-estate, including your GraphQL schemas. Nice work.

Jem concludes by saying :

> GraphQL is a very popular graph query language, well used, known and liked by
developers. The Ontotext Platform uses GraphQL to lower the barrier of entry to
knowledge graph data, whilst still providing the full expressivity and power of
SPARQL.
and this is a key message, which equally applies to Stardog and any other graph
database that may yet offer a GraphQL client. While GraphQL lowers the barrier
to entry, GraphQL gives you only as much as it is able to with respect to
querying and graph traversal;  to really unlock the power of the graph database,
you will need to use SPARQL.

But if you never plan to, and you are not interested in reasoning either, then
possibly you should ask yourself if you really need a graph database ?

What if you want to use GraphQL as the primary API technology in your stack, but
want to use SPARQL too to access the graph database (amongst other tech)?

A mixed data technology case study
In a very recent engagement, the Data Language team have been working on a Smart
Cities IoT project. The nature of the domain meant the type of data we needed to
deal with was very ‘graphy’ in nature, and the ability to describe our data
semantically and traverse a graph with SPARQL was of high utility. Inference was
less of a concern. The proof of concept for the project was brought together in
AWS with :

 * AWS MSK - a high performance Kafka message bus to consume, orchestrate, and
   distribute IoT sensor events
 * AWS Neptune to give us our scalable RDF based smart city knowledge graph
 * A set of JSON-LD REST API endpoints invoking SPARQL queries on the graph
   running in Kubernetes (EKS)
 * A suite of Kafka event processor consumers and producers deployed into
   Kubernetes mutating the graph, and emitting new events based on our business
   logic rules.
 * AWS AppSync - the GraphQL engine sourcing events and the JSON-LD REST APIs
   from Neptune, providing the primary developer interface to our client
   applications
 * A ReactJS built UI using GraphQL react-hooks to subscribe to real time events
   data, and to query data from the sensors and our knowledge graph (ultimately
   via expressive SPARQL queries).

A simplified view of the architecture looks essentially like this :


The kicker was modeling data across the entire stack using a suite of
ontologies. These ontology models describe our knowledge graph data, our IoT
message payloads in JSON-LD and were all federated in the primary application
API expressed as GraphQL schemas in AppSync. Building the ReactJS front end
using the Apollo GraphQL client and react-hooks was a very nice developer
experience.

Dessert
While GraphQL technically has little to do with Graph Databases, they are not a
perfect match like sausages and mash, GraphQL is just another API / query
technology, and not as expressive as the native graph database query languages
like SPARQL, but in itself is a high-utlity, very sweet technology.

There are some clear synergies, and do provide some of that warm comfort food
feeling when used together. GraphQL does give you an easily understood and very
developer friendly way into a graph database. Once you add the secret sauce of
domain ontologies describing both your knowledge graph and possibly even your
entire data-estate, expressed through your GraphQL schemas, then you have a very
nice pattern for a decoupled data architecture with well structured, nicely
communicated data models across your technology stack, and out to your consumers
too - making for efficient application development using the GraphQL ecosystem.
Get stuck in.

Graph databases

GraphQL and Graph Databases

Fabio Colasanti

Earlier this week, this tweet
[https://mobile.twitter.com/random_walker/status/1196870349574623232] appeared
on my feed.

> Much of what’s being sold as "AI" today is snake oil. It does not and cannot
work. In a talk at MIT yesterday, I described why this happening, how we can
recognize flawed AI claims, and push back. Here are my annotated slides: 
https://t.co/iCpyFw5urN pic.twitter.com/pmOTI3vq8p [https://t.co/pmOTI3vq8p]

— Arvind Narayanan (@random_walker) November 19, 2019
[https://twitter.com/random_walker/status/1196870349574623232?ref_src=twsrc%5Etfw]

Prof Narayanan's deck
[https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf] has
reached a relatively sizeable audience and received widespread positive
commentary. After all, anyone who has had any involvement with AI in the last
few years can 100% relate with the catchy title of his deck.
Hey - we're just about starting to see a feeble light at the end of the
blockchain tunnel, right?

But, but...
Something doesn't quite sit well with me. So I'm going to stick my neck out and
respectfully reject, if not the conclusions, the way Prof Narayanan has chosen
to make his point. Or rather, the way he has chosen to make his real point.

I find this deck deceptively coy, as the actual message is not "How to recognize
AI snake oil" but rather "Why I don't want AI anywhere near socially sensitive
topics", which would have been a perfectly valid argument, and made for a
fantastic set of slides, but required a slightly different set of arguments and,
most importantly, to be explicit about it.

At the end of the day ML/AI is about identifying patterns in data and draw
inferences, so of course it can "predict the future". Google Maps does it all
the times when it gives you an ETA for your journeys. In Prof Narayanan's own
examples, bail decisions and border control are not performed by machines, but
based on the prediction of machines. It's not about predicting the future: it's
called decision support, performed at speed that is beyond human capabilities.
Is the way banks and insurance companies make decision on mortgages and premiums
snake oil too? 
Prof Narayanan seems exceedingly preoccupied about "massive transfer of power
from [...] workers to unaccountable tech companies" and points at "enormous
commercial interests" that "we must resist" and "push back". He is of course
completely entitled to give the whole argument a political spin, but I'm not
sure this helps his arguments.

The real issue here is that some problems, and especially socially impactful
ones, carry high personal risks if an inference is wrong (problem 1). Therefore,
in order to minimise that, large amounts of potentially sensitive data may be
required (problem 2) and in most, if not all, cases there is a strong argument
for transparency, accountability and human-in-the-loop that makes such
propositions much more expensive and challenging from both a technical (problem
3) and governance (problem 4) perspective. 
Bias poses no doubt massive challenges (problem 5), not just in its
identification and removal, but also in its very definition, as what turns
common sense into bias is a function of our set of social and cultural values,
which greatly varies in time and space. Should history be changed because it
doesn't fit our Western 21st century view of the world? I don't have an answer,
but at least I'm asking the question.
Also, there is the topic of regulation which can't be ignored in these cases:
who establishes the boundaries? Who is it meant to enforce them? Do we need a
universal legislation, or would locally applicable frameworks suffice? Who makes
a call on what is an acceptable risk vs potentially beneficial outcomes -
supranational bodies, governments, individuals, etc.. ? (problem 6)

So the better discussion to have is around the risks involved in deploying AI in
socially sensitive fields, what is the minimum level of benefits it must bring
about to make it worth us facing the challenges above, and how do we go about
having that conversation.

Just branding the whole thing as snake oil is sidestepping the issue and largely
unfair, because it's bypassing the actual conversations to be had and going
straight for the outcome that Prof Narayanan - in arguably a biased opinion -
has decided is what's best for us all.

AI is a core discipline at Data Language. We have an expert team skilled in machine learning and deep learning, and importantly we also have the engineering and deployment skills to deliver AI at scale for your production systems.

AI is a core discipline at Data Language: Our expert team is skilled in machine learning and the engineering skills needed to deliver AI at scale.

Artificial Intelligence delivered at scale - our core discipline.

Artificial Intelligence

We need to talk about AI

It may be argued that customers' satisfaction is the only and ultimate KPI that
really matters when judging the success of a product. In which case Tagmatic
would pass with flying colours, given the fantastic feedback we are receiving
from clients. 

Remarkably enough, we found that sheer inferencing performances are often at the
edges of our clients' considerations: this is partly due to the intrinsically
tailored nature of Tagmatic's models, but mostly attributable to a plethora of
invaluable qualities that go beyond the naked F1 scores: being able to quickly
spin up a model on your own content and vocabulary, and then interact with it
via a minimalistic HTTP interface, zero maintenance, on-the-fly learning of new
classes, low latency, security, scalability and high availability out of the
box, are crucial factors to enterprise-scale deliveries. 

Your Intellectual Property lies in your content, your data, your data models,
and how you use the classifications. It is not in commoditised compute
solutions.
Your resources are thus more effectively spent developing and exploiting your
market differentiating IP, rather than spending way too much time and money on
trying to replicate the engineering complexities of off-the-shelf MLaaS
solutions which may not be your technical team's core competence.

This is where Tagmatic comes in. The hard work and complex engineering has been
done, just bring your content!

The datasets
RCV1 is arguably the most popular dataset for multi-label classification,
comprising of 800,000 newswires by Reuters tagged with 103 classes. Oddly enough
for such a totemic dataset, the breadth of the vocabulary is anachronistic when
compared to some of our clients, modern global publishers who routinely maintain
taxonomies with thousands of topics.
The dataset is available here
[http://www.ai.mit.edu/projects/jmlr/papers/volume5/lewis04a/lyrl2004_rcv1v2_README.htm] 
although unfortunately not in its original XML format with full HTML markup,
which we would normally leverage as part of the classification task. Also the
corpus comes already tokenised and train/test datasets pre-canned. Besides
taking away part of the fun, this has also negative implications on the
vectorisation logic Tagmatic implements, making it largely ineffective
unfortunately. For the purpose of this benchmark, we have used the full train
set (23,000) and a reduced test set (100,000).

AAPD (Arxiv Academic Paper Dataset) is a collection of research papers covering
a number of subjects organised in several classification systems. The dataset 
here [https://www.kaggle.com/neelshah18/arxivdataset] is annotated with a number
of classification schemes, of varying level of quality. We have carved out a
subset corpus in the Computer Science domain, comprising of 32,000 papers and 39
subjects in total.

The benchmark
We have considered a number of models amongst the most widely employed for
multi-label classification tasks. The reference evaluation metric is
weighted-F1.

RCV1 AAPD One-vs-Rest - LinearSVC 0.78 0.71 One-vs-Rest - SGD 0.74 0.62 
One-vs-Rest - XGBoost 0.75 0.64 Classifier Chain - LinearSVC 0.79 0.71 Label
Powerset - MultinomialNB 0.55 0.55 Label Powerset - RandomForest 0.60 0.55 
Tagmatic 0.80 0.74 
Tagmatic offers out-of-the-box improvement over a number of basic
implementations. Granted, a team of Data Scientists might possibly come up with
better optimised flavours of such models, pushing performances up. But at what
cost? And by how much? And what then? There is still the much bigger task of
bringing that model to fruition in a real world environment, requiring
non-trivial development effort. Is that really worth it? 
Unless text classification is either your USP or a strong market differentiator,
then the answer ought to be a resounding no. 

Tagmatic is production-ready, enterprise-grade, and available to you right now
[https://datalanguage.com/tagmatic].

Lastly...
We have also benchmarked Tagmatic against Hansard data: a corpus of ~200,000
parliamentary interventions for the 2005/6 period , classified with ~7200
categories, scoring a whopping 0.83. More on this on the next blog post. 
Make sure you sign up for our newsletter [ https://datalanguage.com/contact-us] 
and follow @Tagmatic [https://twitter.com/tagmatic] on Twitter for all the
latest.

News, Blog Posts and Innovation updates about Tagmatic - our sophisticated AI automated content classification SaaS product. We have taken care of all the engineering and data science for you: Tagmatic learns on the fly, is self-optimising, and language-agnostic.

Tagmatic is an AI automated text classification SaaS product by Data Language. It learns on the fly, is self-optimising, and language-agnostic.

Tagmatic - News and Product Updates about our Text Classification AI

Tagmatic

Benchmarking Tagmatic

One the most important tasks for software architects in the early stages of any
technology delivery program is to reach “architecture conceivability”, or
consensus agreement with senior developers that the technical proposition could
actually conceivably be workable. Starting from a position of defensive
scepticism, it typically entails an outside-in narrative exploration of the
potential end-to-end program execution flows at a level of detail between the
Component and Code levels in the C4 Architecture Model [https://c4model.com/] 
for key use cases.

The granularity of the discussion will switch depending on the level of
technical risk entailed by the component. If it is providing a standard
capability that is self-evident to to those present how to code, then there is
no point doing the more detailed code discussion. If it is encoding anything
with a non-trivial level of technical risk, then dropping down to the next level
will be required until everyone is roughly on the same page about how it needs
to work. In this way it acts a bit like a code preview (as opposed to a code
review), functioning in the same way that a delivery prespective
[https://www.malotaux.eu/nrmc.php?id=prespectives] is more efficient than a
retrospective in terms of mitigating risk.

Once and only once this has been completed, you will have established baseline
viability for the program. The narrative described will be unquestionably wrong,
inefficient, full of flaws – that’s not the point. The point is that it will
represent a first, stuttering, throwaway sketch out which a simple, resilient
system design can eventually be evolved. Gaping holes and pointless layers of
cruft will already be visible. Beyond that, most of the selective pressures
driving that evolution will be created later by writing tests and the feedback
from coding itself. 

That’s all fine. However, until you’ve established architecture conceivability,
in reality you’ve got nothing. It is hard to over-emphasise the importance of
this point. Context and Container-level views are meaningless slideware (or at
very best, vague indicators of business process alignment) until they are backed
up by Component and Code-level views. That is solely how they gain their value,
by proxy as maps to decomposing and navigating the the Code-level views of the
platform. Without the code, they are essentially maps to nothing.