Paul Wilton

Data Language are trusted experts delivering knowledge graph products and services to global brands since 2010.

Knowledge Graph Engineering experts - Data Language

Data Language has exceptional experience in the design, implementation and
deployment of knowledge graphs, linked data, and semantic web technologies. We
have been working with graph databases since our inception, when our founders
worked on possibly the UK’s first enterprise scale knowledge graph for BBC News
and Sport. Since then we have helped global organisations such as Cochrane,
Wellcome, and Euromoney implement them too. Our typical approach to implementing
a knowledge graph solution is:

Domain Driven Design
Through collaborative domain modelling exercises, we work with stakeholders to
gain a broad understanding of the target domain, iteratively refining and
validating ontology models to achieve the ideal level of detail to deliver the
best outcomes, business value and utility.

Technical and Data Architecture
Not all graph databases are equal. We are wholly vendor-agnostic, and our deep
experience in designing scalable, maintainable, and evolvable knowledge graph
solutions, means we will always select the right graph technology to deliver
your use-cases. We understand the difference between label-property graphs and
RDF based graph databases. We have a total grasp on the practical and pragmatic
use of semantics and inference to ensure your solution is not over-engineered
but delivers just the right level of complexity to optimise your total cost of
ownership, and ensure your solution can evolve as your business evolves.

Integration - Data and Software Engineering
We can help you integrate the knowledge graph with your business systems and
workflows, either by helping your own software engineering team with querying
the graph using query languages such as SPARQL or Gremlin, through to developing
microservices and web interfaces that encapsulate your target use-cases. We are
skilled at data engineering, and can help you populate your knowledge graph,
transforming and loading your own business data, and linking to or ingesting
open-data as required.

Automation and Deployment
We can work with and guide your own infrastructure team, or take on the task of
engineering, integration and deployment entirely ourselves. We are skilled at
contemporary best practices for deployment of infrastructure in the cloud or
on-prem. Our devops engineers are skilled in the use of infrastructure-as-code
for fully automating the deployment and maintenance of the entire knowledge
graph solution.

Graph databases

Knowledge Graph Engineering

Matt Shearer

Domain-driven design, or domain modelling is about optimising a team's knowledge and understanding of the domain it works in. Find out how DDD can benefit your project here.

What is Domain Driven Design and why do you need it?

What is Domain-Driven Design?
Domain-Driven Design (DDD) is a method for developing a team's understanding of
a problem space. It emphasises placing the primary focus of a project on the
core area of the business (the core domain). This often takes the form of
facilitated workshops with domain experts and the development of a shared set of
conceptual models. DDD has the additional benefit of creating a shared
understanding (ubiquitous language) between teams when designing technical
solutions.

Skilled facilitation of domain modelling sessions helps our clients get to grips
with their domain. Through collaborative domain modelling exercises, we work
with stakeholders to gain a broad understanding of the target domain. We
typically focus on the high value elements of the domain, iteratively refining
and validating models to achieve the right level of detail to deliver the most
business value.

Would my project benefit from Domain-Driven design?
Common reasons that you would look to Domain Driven development would be
projects that need to distill the knowledge of domain experts into a product.
These types of projects can often suffer from a disconnect in domain
understanding. Discovering this disconnect in code or in data models is
expensive to fix. DDD can address this by tackling differences in domain
understanding before implementation begins. It can create a perfect bridge of
understanding between the business stakeholders, the problem-space, and the
technical teams.

Another application of DDD is in projects that require strong information
architecture. Where the intention is to organise a product around the entities
and relationships of the domain. This is often seen in approaches that build on
taxonomies or knowledge graphs.

We see Domain-Driven Design as a critical starting point for all our knowledge
graph implementations, and on the majority of our project engagements, including
development of our own products.

News, features and updates about our product Data Graphs. Find out how this intuitive knowledge graph product is easy to setup and connect. Brought to you from the team that was foundational in the earliest large scale knowledge graph projects in the UK.

Data Graphs - News, features and product updates from Data Language

Data Graphs

Domain-Driven Design

Resolute Square launches, using Data Graphs as a knowledge graph platform, combining CMS, VOD, DAM, SCV & MDM, complete with powerful APIs.

Resolute Square - launches with our knowledge graph SaaS

We are pleased to announce the launch of Resolute Square
[https://resolutesquare.com/?utm_source=datalanguage&utm_medium=newsPost&utm_campaign=launchPost]
, a pro-democracy media organization using Data Graphs [https://datagraphs.com/] 
to provide its knowledge graph -powered Linked Media Platform.

We will publish more detail about the technology in future posts.

Why did Resolute Square choose Data Graphs?
Firstly, Data Graphs enabled the team to combine CMS, VOD, DAM, SCV, MDM, live
streaming, and a knowledge graph to connect the media, all in one place. This
enabled the project to launch quickly, with an enterprise media publishing
platform, complete with powerful APIs, without the need to integrate 4+ vendor
solutions for this central purpose. Speed to Market!

Secondly, this approach enables the Resolute Square team to interlink its media
in order to deliver rich user experiences, and to innovate and pivot rapidly on
an API-first platform. This kind of capability is not available anywhere else in
the market in an integrated SaaS; projects normally need to pin together several
services to achieve this. 

The Knowledge Graph combines Single Customer View with Single Subject View
The domain model joins Creative Works with People, Places, Organisations, and
the Single Customer View "User" model.Another innovation in this project - Video
Streaming
The Resolute Square platform also includes an innovative ultra-low-latency
participatory live video streaming experience, provided by our friends at 
Perfogram
[https://perfogram.com/?utm_source=datalanguage&utm_medium=newsPost&utm_campaign=launchPost]
.

More soon!

Resolute Square Launches, powered by Data Graphs for speed to market

Alexi Buckingham

Taken from our newsletter which you can subscribe to here
[https://datalanguage.com/newsletter].

Last week we presented our knowledge graph product, Data Graphs
[https://datagraphs.com/?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=intro] 
at The Knowledge Graph Conference [https://www.knowledgegraph.tech] at Cornell
Tech on Roosevelt Island, New York.

Based on the presentation topics and workshops at the conference, and backed up
by the "2nd Annual Knowledge Graph Survey", here are our 3 takeaway messages for
you:

Takeaway 1: Organisations are struggling to get started
We heard a lot about the challenges of skills gaps, organisational inertia, and
governance gaps, and that when trying to adopt knowledge graphs, many
organisations just don’t know where to start. This is reflected in the 2nd
Knowledge Graph Annual Survey with 86% of responses saying that 'Organisational
Inertia' was an important inhibitor to their adoption of knowledge graphs.

Organisations can break through the inertia by using a reference implementation
Our preferred approach to breaking through this inertia is to build a reference
implementation rapidly. Using Data Graphs, you can have a working reference
implementation ready within a day - we did it in 13 minutes here
[https://datalanguage.com/news/13-mins-to-set-up-a-knowledge-graph-with-netflix-disney-amazonprimevideo-and-hulu]
.

A working reference implementation has 3 main benefits:

 1. It helps stakeholders grasp the value of data meaning and interconnection,
    hands-on.
 2. Seeing the knowledge graph visually, and being able to explore it by simply
    clicking from node to node sparks new ideas for all disciplines.
 3. A working version gives teams a real basis on which to iterate and answer
    key questions, including “What KG features do we need?”, “What model works
    well?”, “What kind of inference and path analysis do we need?”, and more.

Have a look at the Knowledge Graph Explorer
[https://media-graph.datagraphs.io/media-demo/people/concepts/urn:media-demo:Person:3Vn1Bxu4W6yzLNL54eB4b5/explore?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=reference-implementation] 
feature of our product Data Graphs below. A reference implementation like this
helps organisations to break through the inertia, and save time spent wallowing
in indecision.

Knowledge Graph Explorer
[https://media-graph.datagraphs.io/media-demo/people/concepts/urn:media-demo:Person:3Vn1Bxu4W6yzLNL54eB4b5/explore?utm_source=email&utm_medium=newsletter&utm_campaign=internal-inertia]
Click here to try out the Knowledge Graph Explorer
[https://media-graph.datagraphs.io/media-demo/people/concepts/urn:media-demo:Person:3Vn1Bxu4W6yzLNL54eB4b5/explore?utm_source=email&utm_medium=newsletter&utm_campaign=internal-inertia]
Takeaway 2: Organisations are looking for key features to enable their plans
In this slide from the KG Annual Survey, you can see the areas here that reflect
what knowledge graph users are interested in, for example 'Search', 'Better
Ontology Management', 'Integration' and 'No-Code/Low-Code'.

Features of interest to KG Survey Respondents

Search
Data Graphs' Advanced Search feature allows you to build complex queries by
chaining and grouping a series of boolean and numeric expressions.

Try Data Graphs' Advanced Search here
[https://media-graph.datagraphs.io/media-demo/_all/concepts?aq=c[director:Person:2mdyLPMIMopbPB0kVUvghw:Quentin%20Tarantino].(c[cast:Person:3ENPqEkRhLUCxj3W3RG1Hp:Samuel%20L.%20Jackson])&pn=1&ps=15&sf=_label&sd=asc&layout=list&viewport=s000000000,1&utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=search]

Better Ontology Management
This area has always been complex and somewhat expert-dependent. "No code"
schema development is now in demand. In our demo, you can see the domain model
(the output of our no-code custom schema tools) for shows on Netflix, Disney+,
Amazon Prime Video, and Hulu.

Try Data Graphs' Domain Model here
[https://media-graph.datagraphs.io/media-demo/_domain-model?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=ontology-management]

Integration
Using APIs and webhooks, how can we help developers integrate with minimum
friction? This step-by-step guide shows how to set up your knowledge graph
quickly and effectively.
[https://support.datagraphs.io/knowledge-base/get-started/building-a-knowledge-graph?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=integration]

Building a Knowledge Graph: Step by Step Guide
[https://support.datagraphs.io/knowledge-base/get-started/building-a-knowledge-graph?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=integration]

No-Code/Low-Code
How might businesses get their knowledge graph up and running without having to
dedicate engineers to coding? The tools in this space are improving, and you can
see how we set up a knowledge graph with Data Graphs using data from Netflix,
with APIs and fully featured management UI production-ready, in 12 minutes here.

Creating a Knowledge Graph in 12 Minutes
[https://datalanguage.com/blog/creating-a-knowledge-graph-in-minutes&utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=integration]

Takeaway 3: Organisations want to reduce 'Time to Value'
In order to get to market rapidly, and be able to iterate based on real customer
needs, organisations are aiming to get their knowledge graph platforms launched
faster. From inception to commercial operation, how do we do this? Whether it is
for a reference implementation, or for commercial use, tools to enable rapid
model building, data ingest, and ready-to-go APIs are front of mind.

This video shows how a knowledge graph with Netflix, Disney+, Amazon Prime Video
and Hulu data can be built, with APIs and powerful UI, in just 13 minutes.

Watch: 13 minutes to setup the media graph
[https://datalanguage.com/news/13-mins-to-set-up-a-knowledge-graph-with-netflix-disney-amazonprimevideo-and-hulu&utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=time-to-value]

But Remember... Preparation is Key
One thing we consistently emphasise is that good preparation is essential. The
speed and relative simplicity of the setup of Data Graphs is only possible with
solid information preparation - data that was already clearly structured and
well organised. Designing your domain model is an essential step in establishing
the shape of your data before embarking on your first implementation. It will
change, no doubt, but do avoid rushing in without a plan. Spend time thinking
about the specifics of your data, the core entities in your domain, the key
relationships between them, and this will enable you to make an effective start.

More on why good preparation is key
[https://datalanguage.com/blog/creating-a-knowledge-graph-in-minutes?utm_source=website&utm_medium=kgc-newsletter-blog&utm_campaign=preparation]

A knowledge graph is a collation of information and data typically within the domain of your business context. The information is stored in a database, often a graph database such that it can be queried, maintained, and interacted with by humans or machines. We have exceptional experience in the design, implementation and deployment of knowledge graphs. We have been working with graph databases since our inception and have delivered knowledge graphs solutions for global organisations.

We have exceptional experience in the design and implementation of knowledge graphs and have delivered knowledge graphs solutions for global organisations.

Knowledge Graphs - our Work, Features, News and Updates

Knowledge Graphs

Reflections and Insights from Knowledge Graph Conference 2022

This video was aired on 6th May at the Knowledge Graph Conference 2022 in NYC, as part of the "Knowledge Graph Tools Track".

The 13-minute knowledge graph: Netflix, Disney, Amazon, Hulu

This video was aired today at the Knowledge Graph Conference 2022 in NYC, as
part of the "Tools Track".

This video was aired live at KGC2022We showed participants how they can create a
Knowledge Graph in 13 minutes using open Netflix, Disney+, AmazonPrimeVideo, and
Hulu data from Kaggle.

It ends with a graph path analysis of how Kevin Bacon connects with Sandra Oh,
via shows and casts, using the graph that we implemented in Data Graphs in 13
minutes.

NB - we had made some careful model design and data preparation before setting
up the knowledge graph.

We took our Knowledge Graphs as a Service product "Data Graphs" to The Knowledge Graphs Conference 2022, loaded with Netflix, AmazonPrimeVideo, Hulu and Disney+ data.

Knowledge Graph Conference 2022

The 13-minute knowledge graph, with Netflix, Disney+, AmazonPrimeVideo, and Hulu data

Find out about our feature at KGC2022, and explore this Netflix, Hulu, and Amazon Prime knowledge graph, using our product "Data Graphs".

Data Graphs at the Knowledge Graph Conference 2022

We've launched a read-only version of Data Graphs for participants at the
Knowledge Graph Conference 2022 to have a go, hands-on.

Data Graphs is our Knowledge Graphs as a Service product. It is used by global
brands to provide a knowledge graph platform to accelerate product
time-to-value.

Lowering the barrier to entry for Knowledge Graph Platforms
Often knowledge graphs are seen as a high-cost and high-risk strategy that only
the largest of companies can implement, over a very long time. 

With Data Graphs, we've ensured that businesses can see value rapidly.
Specifically, this means speeding up the ability to do proof of concepts, and
simple but informative demos that allow organizations to realise early value at
low cost and low risk. Here's an example of one of those demos right now.

13 mins to set up:
Netflix, Disney+, AmazonPrime, and Hulu
When developing Data Graphs, we wanted to make the setup and creation of a
knowledge graph, simple and quick. 

This video, prepared for KGC2022, shows how we created a knowledge graph,
complete with secure API and powerful UI tools, ready for production, in 13
minutes:

Try it yourself! - play with the Data Graphs demo here
[https://media-graph.datagraphs.io/media-demo/_all/concepts/?q=&pn=1&ps=15&sf=_label&sd=asc&layout=list&viewport=s000000000,1&utm_source=website&utm_medium=kgc-post&utm_campaign=faceted-search]
This demo version was created using Netflix, Disney+, AmazonPrime, and Hulu
data, and gives you a good view of just some of the key Data Graphs features.

1. Create your Domain Model for your Knowledge Graph
After you have designed and created your domain model with the business entities
and the relationships that link them together, Data Graphs generates a visual
representation of the model so that you can see clearly how the knowledge of
your business interconnects. 

Check out the Domain Model for Shows, Cast, Genres, Ratings here
[https://media-graph.datagraphs.io/media-demo/_domain-model/?utm_source=website&utm_medium=kgc-post&utm_campaign=domain-model]
.

2. Use our powerful Faceted Search over your knowledge graph
The search capabilities of Data Graphs are stellar and allow you to find and
explore your data precisely and most importantly, quickly. With the Advanced
Search feature, you can build complex queries by chaining and grouping a series
of boolean and numeric expressions, to pinpoint the targets of your search. 

Test the search capabilities of Data Graphs here, on 115k concepts including
shows, cast, directors, genres, countries, and more
[https://media-graph.datagraphs.io/media-demo/_all/concepts/?utm_source=website&utm_medium=kgc-post&utm_campaign=faceted_search]
.

As well as an intuitive visual search tool, all the Data Graphs search features
are available via API so you can integrate them with your business systems and
apps.

Explore the Data Graphs API here [https://support.datagraphs.io/].

3. Use the Graph Explorer to visualise your knowledge graph
The graph explorer is one of the star features of Data Graphs. You can now
visualise your knowledge graph in an interactive representation of your data,
that you can then explore and dive into, using the nodes. By visualising your
entire knowledge graph, from any concept in your dataset, we open up a wealth of
possibilities for your structured data. 

Interact with the knowledge graph explorer here, starting from the "Stranger
Things" node
[https://media-graph.datagraphs.io/media-demo/shows/concepts/urn:media-demo:TvShow:netflix-s3686/explore/?utm_source=website&utm_medium=kgc-post&utm_campaign=knowledge-graph-explorer]
.

4. Use the Graph Path Analysis to gain insights
Data Graphs lets you visually explore paths between any entities in the graph,
in order to find the shortest paths, and to specify constraints on both nodes
and properties that make up these paths. A key feature of knowledge graphs, this
type of analysis is typically used in compliance, fraud detection, and supply
chain analytics. 

You can try out the graph path analysis here
[https://media-graph.datagraphs.io/media-demo/_all/_graph-search/?utm_source=website&utm_medium=kgc-post&utm_campaign=graph-path-analysis]
.

About the Data Graphs demo
The Data Graphs demo is a read-only view of the full version of Data Graphs
[https://datagraphs.com]. The complete product provides a full suite of tools
for curating, ingesting, editing, and maintaining data in a knowledge graph,
along with APIs for integration of your graph data with apps and business
systems.

About the Media Graph data
The media graph data has been sourced from the open data provided by Netflix,
Hulu, Amazon Prime, and Disney+ via kaggle.com here
[https://www.kaggle.com/datasets/shivamb/netflix-shows]. The data sets from
these 4 publishers have been cleaned, loaded, and linked together in our graph
to join 72k actors and directors across 21k  TV shows and movies, all faceted by
genre, rating, country, and year of production.

Data Graphs at The Knowledge Graph Conference 2022

We are holding a team hack event this week with themes around blockchain, AI,
and knowledge graphs. We're solving some problems, and having fun with new tech.

A "Hack event" is a useful way of rapidly exploring, and testing new ideas. The
basic format uses a framed challenge and a time constraint.  Participants
usually "hack" in teams, and the emphasis is on creating working prototypes to
test hypotheses.

In stark contrast to a formal product delivery process, hack events participants
get to try any route without the need for production-grade code, or the
restriction of "business as usual" concerns. 

More often than not the prototypes created are not "the next unicorn"-type
projects, but this is ok! The point is to explore the problem spaces, try novel
approaches, and test new ideas efficiently. 

The ideas are tested in the context of our products, and concepts may make it
into our core product set.

Data Language Hack Event

Charlie Furniss

We have been hard at work improving our Knowledge Graph as a Service product 
Data Graphs [https://datagraphs.com] - in fact it is in a state of continuous
improvement. We have rolled out a number of new features in the first quarter of
2022, and here is a brief roundup on what they are:

Advanced search
Data Graphs Advanced Search feature allows you to build complex queries by
chaining and grouping a series of boolean and numeric expressions. For example
for finding a set of concepts in the graph where they are of Type A and Type B
or Type C, with a date property after a specified date, and a property equal to
some specified value. These searches are available in both the Data Graphs API
and now in the app too.

Saved searches
Saved search enables users to label, store and recall search criteria – great
when you are regularly using complex queries, particularly with Advanced Search
where building up items and groups can be time consuming and you want to return
to the same search results later.

Custom validation rules
When creating a property, custom validation allows you to choose from a set of
configurable rules that can be applied to certain data types and used to control
the property’s data, for example an age limit, an expiry date or a control on
input text. These rules include min/max characters (for text, keyword types),
regular expressions, before/after date (for date, dateTime) and min/max value
(for integer, decimal, latitude, longitude). You can also define cardinalities
on arrays and relationships in the graph.

Enum data types
The new enum data type allows you to create a property with a predetermined list
of options, from which users choose when setting its value. This will be useful
for any field whose value you want to limit to a small controlled vocabulary
that you may not want in your graph as entities in their own right. In the Data
Graphs Concept Editor the enum options are presented as a drop-down list on the
property.

Pre-commit web hooks
Data Graphs pre-commit web hooks allow you to perform a remote impact assessment
of a given event, for example, merging two Concepts, or deleting a Concept when
integrated with some target business system. When one of these actions occurs,
Data Graphs makes an HTTP request with the event’s details in the payload to an
endpoint in your environment, allowing you to analyse the downstream impact of
this change and respond by either accepting or declining the event or by
providing a warning (as illustrated). A full integration of these at Cochrane
has been discussed in this case study here
[https://datalanguage.com/blog/data-graphs-at-cochrane-a-case-study].

Improved geospatial visualisations
As well as the default map view of geospatial data, you can now view concepts
geospatially on dark, light, outdoors, and satellite views with automated entity
clustering based on the zoom-level of the map:

New Features in Data Graphs - Q1 2022

How Cochrane has integrated Data Graphs [https://datagraphs.com] with their own
enterprise graph database of evidence-based healthcare as a complete linked data
management solution.

Population
Data Language has been working with Cochrane as a technical partner for a number
of years now, and we have blogged extensively
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19] 
on the digital transformation at Cochrane, including how linked data and an RDF
based knowledge graph have become a core information backbone for their business
processes.

The Cochrane vocabularies of some 300k concepts provide the linked reference
data that describes the PICO domain of evidence-based healthcare, describing the
Populations (and their conditions, ages, sexes, social context), the
Interventions (drugs, devices, procedures, materials etc), Comparisons, and the
Outcomes. These concepts are annotated onto clinical evidence as PICO
micrograph
data structures [https://data.cochrane.org/ontologies/pico/index-en.html] to
describe the questions being answered in clinical evidence. All of Cochrane’s
vocabularies, content metadata and PICO annotations are stored in an enterprise
graph database, and exposed by a cohesive set of microservice APIs that can be
consumed and recombined to allow rapid innovation.

Data Language has provided the technical architecture, design and implementation
of the knowledge graph, microservices, APIs, and tooling that allows the
Cochrane subject matter experts to curate and QA these PICO data structures as
part of their daily business processes. This linked data platform has allowed
Cochrane to rapidly innovate: enhancing the Cochrane Library with linked data
query capability; and build at lightning speed, the largest Covid-19 database of
clinical evidence in the world, comprising more than 100k studies, described
with well-structured PICO metadata,  serving high volumes of requests and
queries to researchers globally.

Condition
While tooling and processes for Cochrane’s SMEs to create and maintain PICO
graphs describing the Cochrane systematic reviews and studies is state of the
art, the tooling around the management of the reference data vocabularies is
less sophisticated. The Cochane vocabularies consist of a highly curated set of
linked data concepts that form a graph in their own right with class and
property semantics, transitivity, and multi-parent taxonomic structures.
Although Cochrane has an existing visual editor tool to perform operations on
individual concepts in the reference data, it is limited in functionality, and
does not allow for bulk ingest and updates, rich search and discovery operations
for quality assurance, data cleansing, and other operations such as merge,
migrate, and workflow.

To solve the vocabulary management problem, we have integrated our product Data
Graphs [https://datagraphs.com] into the Cochrane linked data ecosystem.

What is Data Graphs ?
Data Graphs is a SaaS product designed to lower the barrier of entry to
Knowledge Graphs. While it can be used purely as a Knowledge Graph in its own
right, it can also be used as a linked data vocabulary management tool, and
integrated with an enterprise graph database, or other business systems using
its API and Web-hook event triggers.  It has an intuitive user experience for
managing linked data, and powerful search, discovery and visualisation
capability, effectively making an ideal vocabulary management tool for Cochrane.

Data Graphs allows you to design or replicate any domain model or ontology, and
populate it with linked data, so it was a straightforward task to replicate the
existing Cochane concept model
[https://data.cochrane.org/ontologies/core/index-en.html] and ingest their
vocabularies. The screenshot below shows the cochrane model recreated in Data
Graphs. 


Cochrane Core Concept Model
Data Graphs provides a number of benefits to Cochrane for managing their linked
data vocabularies :

 * Fully decoupling the workflow and processes around managing their linked
   reference data from their business critical systems.
 * Being able to flexibly and easily modify their core concept model without
   impacting their downstream processes.
 * An intuitive user experience for search, discovery, match, merge and
   migration of data.
 * Graph visualisation for exploring their linked data, and identifying
   anomalies and circular relationships.
 * Easy identification of duplicates
 * A full audit history of all changes made to the vocabularies
 * Injection workflow states into the linked data concepts - for managing
   pre-publication, publication and deprecation of terms.

Cochrane’s Data Architecture
Cochrane’s linked data architecture is centred around an enterprise graph
database (Ontotext’s RDF-based GraphDB
[https://www.ontotext.com/products/graphdb/]), a set of microservices and online
tools for annotating systematic reviews and clinical studies with creating well
formed PICO graphs that describe the evidence. A slice of the logical
architecture  is shown below and is discussed in more detail here
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
:

Cochrane Linked Data ArchitectureIntervention
Integrating Data Graphs
In order to integrate Data Graphs we needed an integration pattern that is
robust and also allows Cochrane SME’s to understand the impact of changes they
may make to the vocabularies, such that they can make an informed decision
whether or not they should proceed with the updates.

For integration with an organisation’s business systems, Data Graphs comes with
an expressive, contemporary REST API, serving JSON-LD payloads (pull), and a
suite of webhooks that can be used for event-driven integrations (push) that can
be configured for different core operations, Create, Update, Delete, Merge etc.
For Cochrane’s purposes the webhooks are ideal. 

Pre-Commit Hooks
Furthermore, for each web-hook, a pre-commit remote endpoint can be configured,
which will be invoked before a write operation is committed in Data Graphs, on
which the target business system can respond with a payload of information that
Data Graphs will render to the user as an impact-assessment if the transaction
is completed. This gives the user an opportunity to back out. These pre-commit
hooks allows the target system to respond for a variety of scenarios :

 1. Silent commit - no impact, just commit the transaction silently, the user is
    not blocked.
 2. Force reject - in which the target system determines the transaction should
    not proceed, and the user is not allowed to commit the change.
 3. An impact assessment, with a risk score, an optional summary of the impact,
    and an optional table of data, that the Data Graphs user will be able to
    make an informed decision upon.

Resilience
When consuming web-hooks (essentially an HTTP endpoint or API in a target
business system, that is configured and invoked on some trigger condition), we
need to ensure to all reasonable endeavours that the event fired will not be
lost due to unforeseen circumstances (eg target system failures, connectivity
issues etc.).  To achieve this, the target system endpoint needs to be as robust
as possible and ideally decoupled from other downstream systems. 

The best way of achieving this in the target system is to build an extremely
lightweight microservice that simply receives the HTTP event payload and places
it on a topic or queue. This queue can then be consumed at the target system's
leisure and can cope with failure downstream (the message will not be lost if
the consumer fails temporarily). The target system can take this further with
dead-letter queues and retries to cope with a variety of challenging
circumstances. This is the pattern adopted at Cochrane and is a pattern we would
generally recommend for consuming webhooks between other products and platforms.
You can see the logical architecture for this below :

Data Graphs Integration Architecture at CochraneThe key integration points
(numbered in red)  within the Cochrane technical stack are :

 1. The Pre-commit web-hook handler. This is a simple webservice that is invoked
    by Data Graphs for a specific transaction event before the transaction is
    committed (for example a Concept Delete). The service makes a query against
    the Cochrane evidence graph to ascertain the impact the change will have.
    For example if a concept is used on thousands of PICO graphs, it is likely
    the user should not be deleting this concept, and the service can respond
    with a straight reject, or return a table of the PICOs affected.  This
    information is presented back to the user to make an informed decision.
 2. The Web-hook handler. This is a lightweight service that is invoked by Data
    Graphs for a specific transaction type (Concept Delete, Merge, Update etc)
     after the change is committed. This service simply places the data event on
    a queue, ready to be processed.
 3. Concept Event Handler. This service does the heavy lifting, taking events
    off the queue, and writing updates to concepts and PICOs in Cochrane’s Graph
    Database using SPARQL. If this fails for whatever reason, the message is
    retained on the queue, and can be tried later. For Cochrane much of this
    code already existed in the legacy LDGraphs API, which was performing a
    similar function for their original linked data editor, so was a case of
    porting this into the event handler.

The webhooks in Data Graphs are easy to configure, as can be seen on the
following screenshots.

First, choose an event type to trigger a web-hook, then configure the endpoint
in the target system, an optional pre-commit hook, and any headers / security
tokens that need to be sent with the request :

Webhook Configuration in Data GraphsMigrating the vocabularies into Data Graphs
The final step was to load the Cochrane linked data vocabularies into Data
Graphs, conforming the domain model we created earlier. This can be done in
different ways

 * Using the Data Graphs REST API
 * Using CSV upload using the CSV to Ontology mapping tool within Data Graphs

In this case, the API was the easiest approach as we could read pages of JSON-LD
concepts from Cochrane’s concepts API, and pipe them into the Data Graphs write
API. As the ontology model’s in both were identical, this was a relatively
trivial migration script.

Outcome
With this in place, Cochrane now has a state of the art linked data management
toolkit. The fully whitelabelled user experience engenders user-ownership.
Cochrane’s information experts can organise their core entities in logical
datasets for rapid navigation:

Cochrane Datasets
Drilling into each dataset, a SME can search for concepts, and easily find
potential duplicates through a complex query or use the label/property facets :

Faceted SearchAnd merge those duplicates with an intuitive UI :

Merging Concepts in Data Graphs
The resulting merge transaction will be handled by the webhook event, not just
to update and merge the concepts in the Cochrane Graph Database, but also update
all PICO graphs in which the source concept was previously used to reference the
target concept instead.  Similarly, deletes will remove the concept reference
from all content and PICO annotations.

Making any changes, such as deleting a concept results in the pre-commit hooks
requesting an impact assessment, so the user can make an informed decision to
continue or cancel:

Pre-commit Impact AssessmentsThe knowledge graph can be explored visually to
help find and correct anomalies in the vocabulary, such as circular
relationships :

Graph Explorer in Data GraphsThe Cochrane bioinformatics SMEs now have many more
capabilities at their disposal taking their ability to deliver the next
generation of evidence-based healthcare innovations to a new level.

.

Data Graphs at Cochrane: a Case Study

Fabio Colasanti

We have recently completed some new "Inline Entity Extraction" benchmarks. This
is a sophisticated "Named Entity Recognition" (NER) product that we launched as
part of our Text AI Services suite in 2020
[https://datalanguage.com/news/inline-entity-extraction-text-ai-ner]. It is
optimized for Knowledge Graph platforms in high-value knowledge organizations.
It identifies entities inline within text media and attributes rich metadata.

The Problem: Automated Tagging in Custom Domains
Our customers are high-value media publishers, knowledge organizations, and DaaS
providers. 

 * A huge portion of their business asset value is locked in their unstructured
   mixed media assets.
 * To liberate this value, accurate inline text classification of 100s of 1,000s
   of entities is required to enable structured connection with bespoke
   information architecture models and/or knowledge graphs.
 * Generic off-the-shelf NLP SaaS solutions do not recognize these custom
   entities, so a mapping exercise is required, and is often a too low
   resolution to provide value.
 * Many NLP solutions do not provide the rich, domain model-aware metadata,
   because they are not trained for the custom domain.
 * Other custom NLP solutions require larger training sets and golden corpora
   that cover the entire entity set.
 * "Build it yourself" becomes very expensive, with a large data scientist and
   engineering overheads, while organizations find themselves re-solving the
   same challenges that we have already solved.
 * "Manage it yourself" has a surprisingly large data science and engineering
   overhead, to overcome production challenges that we have already solved.

Testing methodology
We trained our Inline Entity Extraction SaaS to understand around 180,000
entities from a custom domain.  We then benchmarked it in a controlled
environment where two teams of annotators were asked to annotate 500 documents:-

 * The first team used the old-fashioned method of manually annotating a
   document.
 * The second team moderated the output of the model that had already scanned
   the documents and picked up entities, only needing to confirm or reject
   pre-determined results.

Results: High F1 Score
Here’s how our Data Language Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] Inline
Entity Extraction SaaS performed:

 * Precision: 0.94
 * Recall: 0.90
 * F1: 0.92

We have proven we can train a highly accurate model, specialized for a custom
domain, that can cope with hundreds of thousands of entities with very
high-performing metrics, using a golden corpus that covers only a small subset
of the entities. 

Achieving such high results on the classification of such a high number of
classes is very rare for a model like this in the wild.

Results: Huge Workflow Efficiency Gains
The findings of this exercise were that the error rate of the team that used the
Data Language AI model to pre-annotate the documents was substantially lower
than the team that worked without the model. 

 * The team using "Data Language AI" demonstrated a 57% drop in error rate.
 * The team using "Data Language AI" demonstrated a 53% increase in annotation
   coverage.

The measurable benefits were clearly efficiency and accuracy and added to the
coverage of entities being substantially larger, it’s clear that using our model
was a great success.

Should we keep Humans in the loop, or not?
We were also analyzing whether there is benefit in keeping humans in the loop
and this result proved surprising. While it was clear that helping humans out in
entity extractions tasks by pre-running the content through a model could
significantly assist their operation, we actually found that the error rate of
the model was actually slightly higher when humans were involved than when the
results were totally automated. In this case, humans reviewing results actually
makes things slightly worse!

There are some cases where perfect results are not mission-critical - e.g.
upstream automated tagging to simplify downstream workflow, where "the machines
do the heavy lifting". In these cases a certain amount of error is acceptable
and a totally automated process will work. 

In other cases, where errors are less acceptable, the confidence and evidence
metadata in our SaaS can be used to allocate "less certain" predictions to SMEs
and expert analysts for decisions and feedback loop optimizations.

Comparison to other Text Analytics NER Approaches
So what does our competition offer to compete with our results in the Named
Entity Recognition (NER) space? 

Well, it’s difficult to compare apples to apples in this scenario, as our
competitors suffer from the same problem outlined above: Complex custom domains
with small training corpora.

This is the exact advantage of our SaaS that these benchmarks emphasize. 

To put these results into the context of standardized testing of NER performance
elsewhere in the wider world and industry, one usually finds that most tests of
F1 approach the value of 1, only where these models work by identifying a small
handful of classes. Have a look - you will likely see "People, Places,
Organizations, Events"... the usual suspects, and this problem itself has been
solved daily for the last 12 years by almost every computer science student.

The Benefits of Data Language Text AI Services
The fact that our model can cope with a far greater number of classes than most
other providers, and still see such impressive results, is a strong
differentiator.

Another advantage of our Data Language AI Services
[https://datalanguage.com/products/datalanguageai] that we offer is that they
work out of the box at scale for your production platforms.

Our Inline Entity Extraction SaaS comes with all these benefits immediately: 

 * Ability to classify text content to 100s of 1,000s of entities in a custom
   domain.
 * A rich API for interacting with the machine learning service metadata.
 * Automatic management of machine learning model ‘fallout’.
 * Self-optimisation of the machine learning models, on the fly.
 * Capability to cope with an extremely high number of entities.
 * The ability to scale up indefinitely.
 * Production-ready engineering, out of the box.

All the engineering is done, and the security & maintenance are on us!

AI Relationship Extraction for Knowledge Graph Platforms
Later in 2022, we will launch a new commercial Knowledge Graph Relationship
Extraction service as part of Text AI Services - early testing has impressive
benchmarks for high-value knowledge organizations. More to come soon.

For more information on Data Language AI Services, check out the links below.

Data Langauge AI [https://datalanguage.com/products/datalanguageai] | Text AI
Services [https://datalanguage.com/lp/text-ai-services]

New Benchmarks from our Named Entity Recognition (NER) Text AI SaaS Product

Silver Oliver

Data Graphs [https://datagraphsa.com] is a product we have designed to lower the
barrier of entry for organizations to launch a knowledge graph platform. We have
shortened the time to launch from months down to hours (in a recent event, we
built a Netflix knowledge graph from scratch with Data Graphs in just 12
minutes).

In Gartner's recent report
[https://www.gartner.com/en/articles/the-4-trends-that-prevail-on-the-gartner-hype-cycle-for-ai-2021] 
on the significance of knowledge graphs, they listed some obstacles that many
companies encounter when trying to implement and maintain them successfully. In
this piece, we'll be explaining how our new knowledge graph product, Data Graphs
handles each of these obstacles.

Gartner's Obstacle 1:

Data Graphs Solution:

Lowering the barrier of entry to knowledge graphs will make it easier for
businesses to see value quickly, and therefore pushing them higher up the
priority list. Specifically, this means speeding up the ability to do proof of
concepts, and do simple but informative demos that allow organizations to
realise early value at low cost and low risk. 

Gartner's Obstacle 2:

Data Graphs Solution:

Data Graphs is an out-of-the-box product. All the engineering and system
integration expertise, that Gartner points out as an obstacle to most companies,
is already taken care of. Data Language has already addressed the main
engineering challenge and initial setup within the product, and engineered the
knowledge graph capability. Integrating with Data Graphs is light, easy, and
based on common web standards and well constructed, robust APIs which saves you
having to wrestle with complicated graph data standards.

Gartner's Obstacle 3:

Data Graphs Solution:

This is a slightly more nuanced obstacle that in summary deals with the fact
that there are many different types of knowledge graph. Any one product,
including Data Graphs, is not a one-size-fits-all solution so companies must be
careful to choose the right knowledge graph for their use-case. 

Data Graphs, for example, is not trying to be an RDF Triple Store, which will
provide the infrastructure for a great knowledge graph for some use-cases, but
less suitable for others. 

In Data Graphs we’ve distilled-down what we’ve seen as the most useful
attributes of a knowledge graph and vastly simplified their use for businesses
that don’t want/don’t have the resources to engineer a full RDF triple store or
a property-label  graph solution.

Over our 10-15 years experience in implementing knowledge graph solutions in the
enterprise, we find that a large percentage of solutions don’t need RDF based
platforms. Instead, we have tried to create the perfect cocktail of enough
semantics to deal with the use cases that most businesses have, but at the same
time keeping it simple enough so as not to get bogged down in complexities that
the majority will never need. Data Graphs does provide RDF interoperability
through its JSON-LD APIs, and Data Graphs does not preclude you from using an
enterprise-scale graph database too. Data Graphs can also be used as a graph
entity editing tool, that can be integrated with an RDF or property graph
database, and this is a soluiton pattern used by some of our customers. 

Gartner's Obstacle 4:

Data Graphs Solution:

To tackle this obstacle we have to go back to the main lesson we always try to
impress on our knowledge graph clients: preparation is key. You have to
understand your data and design it well, solid domain modelling is always the
imperative first step. We often see projects get distracted by the technology
when in reality some of the most important work when creating a knowledge graph
is the preparation, making sure your data is described, structured and
understood clearly. Desgining a quality domain model/ontology for your own
use-cases is critical to unlock the value in sharing linked-data. Data Graphs
allows you to construct your domain model and visulize it within the tool, and
also allows you to export the domain model as an RDF/OWL ontology. Data Graphs
uses linked-data at its heart, to enable interoperability through JSON-LD and
JSON-LD APIs.

Find out more about Data Graphs and our knowledge graph experience below.

Data Graphs [https://datagraphs.com/] | Knowledge Graph Engineering
[https://datalanguage.com/knowledge-graph-engineering]

How Data Graphs addresses Gartner’s obstacles to Knowledge Graphs

After a long history of working together on industry standards, Data Language
has become an official member of the IPTC.

The IPTC is the global standards body for news & media. They provide the
technical foundation for the news ecosystem and involved in setting schemas or
models for the exchange of content in news media. For example, this could be
deciding how the wires are delivered, working on metadata values, controlled
lists of values, and many other functions.

15 Years of Collaboration
Collectively at Data Language we've been working with IPTC on their standards
for the last 15 years, and have been actively involved at many stages. We were
involved in rNews [https://iptc.org/standards/rnews/] which was the schema.org
for news standards, and more recently we've been contributing to their sports
models. Silver Oliver, Data Language's Head of Information Architecture, has
been an active member of the sports model working group, developing those models
to discover what the next generation might be. 

Metadata Innovation in the UK publishing industry
So why did we decide to formalise our involvement with the IPTC? 

Their work is highly valuable in the industry and we want to solidify our
contribution to the innovation, maintenance, support and development of their
standards. We want to help define standards from a Data Language point of view,
and with that be able to encourage the adoption of those standards. As we are
one of the leading agencies supporting metadata efforts in the UK publishing
industry, and given our client expertise, this new relationship with the IPTC is
a great fit.

The opportunity to show our knowledge graph product Data Graphs
[https://datagraphs.com] as an example of good industry standards to other
members of the IPTC is another reason why we wanted to partner even more closely
with them. Having the prestige and quality reputation of the IPTC associated
with Data Language and our push to deliver brilliant knowledge graphs offers
great benefits. We are also always looking at new patterns and tooling in this
space so this membership is an opportunity to validate some of our thinking and
our toolsets against what the IPTC is identifying as the direction that they
think the industry should be going. It's  a really positive way for us to keep
in touch with how the industry itself talks about its needs and what trends are
likely to arise.

Find out more at the links below:

IPTC [https://iptc.org/]| Data Graphs [https://datagraphs.com/] | Knowledge
Graph Engineering [https://datalanguage.com/knowledge-graph-engineering]

Data Language Joins the IPTC

Janes is the latest in a group of high-value, knowledge-driven organisations to
develop their information backbone using rapid innovation in data management
with Data Language.

18th November 2021, London:  Data Language, the data innovation consultancy and
solution provider, today announces it is supporting the interconnection of Janes
120+ years of open-source defence intelligence to create Janes Intara, which
enables the integration and normalisation of disparate data sources, allowing
Janes customers to exploit all intelligence sources in an actionable, shareable
manner.

Data Language has worked as part of a joint team with Janes data and technology
experts to launch a new knowledge graph platform to power the Janes Intara
solution. The platform creates the foundations for ongoing and rapid innovation
of Janes foundational interconnected intelligence.

Ben Conklin, Chief Product Officer at Janes, comments: 

> “Our partnership with Data Language enabled us to augment our development of
Janes Intara, a solution that interconnects millions of assured data points
across Janes foundational intelligence, delivering contextualised insight to our
customers. Working with Data Language on our Knowledge Graph has been imperative
to accelerating the development of our first generation product, further
supporting our integration into customer workflows worldwide.”
Paul Wilton, CEO, Data Language, adds: 

> “The impact that Knowledge Graph and AI technologies can have on high-value
information organisations is significant in terms of both enterprise efficiency
and rapid innovation, enabling Janes to deliver state-of-the-art intelligence
data capability to their customers. Data Language is a global leader in
Knowledge Graph solution delivery, and we're delighted to be working with Janes
to develop their Intara platform.”
Data Language has unique expertise and years of experience working with large
organisations to devise and develop world-class data management and innovative
data products, including Knowledge Graph and AI-based tools.

> Pleased that our data and technology experts worked with Data Language to build
the knowledge graph for Janes Intara! Learn more here: https://t.co/RhAtUiVNdN 
https://t.co/ecWCtOQgVh

— Janes (@JanesINTEL) November 18, 2021
[https://twitter.com/JanesINTEL/status/1461351890243833859?ref_src=twsrc%5Etfw]
About Data Language

Data Language is a UK-based data innovation consultancy and solution provider
that has worked with organisations such as News UK, Syngenta and Cochrane to
deliver digital transformation solutions using knowledge graphs and AI. Learn
more at www.datalanguage.com [http://www.datalanguage.com/].

About Janes

The Janes stamp of trust means supplying our customers with timely, validated,
unbiased and relevant intelligence in a world crowded with increasingly
unreliable information. With an unmatched legacy of more than 120 years, Janes
has adapted, expanded, and developed its unique tradecraft while transitioning
from a traditional military publisher to the leading global agency for
open-source defence intelligence. Janes Intara interconnects millions of assured
data points across Janes foundational intelligence, with the ability to
integrate and contextualise multiple sources, delivering the single source of
truth. Learn more: www.janes.com/whatwedo

To find out more visir Data Language Knowledge Graphs
[https://datalanguage.com/knowledge-graph-engineering] | Janes Intara
[https://www.janes.com/intara-interconnected-intelligence]

Data Language & Janes: A Knowledge Graph Partnership

Last week we presented a talk at the online Knowledge Graph Forum
[https://www.ontotext.com/company/event/knowledge-graph-forum-2021/] hosted by 
Ontotext [https://www.ontotext.com/]. Here’s an overview of our objectives for
this talk, and a summary of some of the things we spoke about. 

Our Aim
We assumed that our audience were people who were either interested or actively
working on knowledge graphs and feeling their way through the problem space.

Our main aim therefore with this presentation was to show the amazing
capabilities of our own knowledge graph product Data Graphs
[https://datagraphs.com/], and how quickly it helps you create a knowledge graph
for a domain.

Common Knowledge Graph Challenges
Before we started with the process of building the knowledge graph, we wanted to
highlight three main challenges that previous clients had raised time and time
again when faced with this task:

1. Semantics in your Knowledge Graph

If you start using an enterprise graph database and RDF, the semantics can get
complicated quickly and it can be easy to get bogged down in a lot more
complexity than you actually need. The core question then is: ‘How much
semantics is enough?’. One of our main objectives for this talk was to show that
we believe Data Graphs has just the right amount of semantics to demonstrate how
we can lower the barrier of entry for knowledge graphs.

2. Technologies

There's a whole suite of technologies which are generally new to most
organisations, they don't have this expertise internally around RDF and
semantics.

3. Insufficient Information Management

Many of the knowledge graph initiatives that we see are technology-driven. Many
companies buy the knowledge graph technology, overlooking the essential
information management side of things. Your knowledge graph is only as good as
your domain model and this must be made up of well curated data. Often companies
that have a knowledge graph will have neglected their information management and
the data quality is poor. We'll return to this later.

The Main Act
We took attendees through a step by step process of creating a knowledge graph
in just 12 minutes with Data Graphs, using the open source data that Netflix has
made available on Kaggle. We started by showing the domain model that we wanted
to replicate and then began the process of building the knowledge graph. 

The first step was to create datasets and concept classes, which is effectively
our project canvas. Then using Data Graphs’ schema builder we can replicate our
domain model. We built out the classes and their relationships and then finally
we have a visual view of our domain model, replicated within Data Graphs. The
next step was to load in the data. Taking individual CSV files prepared from the
Kaggle Netflix data we populated our domain model one concept class at a time,
and let Data Graphs stitch them together with identifiers that we had
predefined. Once all the data was loaded we had a ‘big reveal’, through the Data
Graphs explorer which allows you to navigate and visualise your knowledge graph. 

The audience was able to see all Netflix’ data on movies, shows, actors,
directors, countries etc. presented in an interconnected, structured, and
manageable way.

The Rewards
The rewards can be reaped immediately. Directly within the UI for Data Graphs
you can start giving people access to it to explore, search, query and facet the
data conforming to your domain model. You can also use immediately use the Data
Graphs APIs to build your own applications on top of the data. 

Key Message? Preparation Before Creation
The key message we were trying to impress on our audience was that for knowledge
graphs, good preparation is essential. The speed and relative simplicity of the
setup that we had demonstrated was only possible with solid information
preparation - data that was already clearly structured and well organised.
Pre-defining your domain model is an essential step in establishing the shape of
your data before embarking on a knowledge graph project. Many people jump into
ontologies well too quickly without spending enough time thinking about the
specifics of their data, the core entities in your domain, the key relationships
between them that will enable to you to answer the right questions of your data.

For example, in our case, before this talk we had gone through several questions
that we knew we would need to know the answers to when creating the knowledge
graph: 

 * What will the shape of the data be?
 * What ubiquitous language should we use to describe our model?
 * What identifiers will each of the data points have?
 * What sort of questions will be want our knowledge graph to answer?
 * What will the pros and cons of different approaches be?

And so on...

Making those decisions upfront and doing the work outside of the technology was
essential in allowing that knowledge graph creation to run smoothly. This
preparation is agnostic to the technology solution so once that is aligned, if
you have the right tool, which in this case was Data Graphs, it becomes simple
to put it all together. When that thinking doesn’t happen before the process of
creating a knowledge graph starts, you end up having to work out many imperative
and foundational design decisions within your already-chosen technology and thus
implementation becomes more complex.

For more information of knowledge graphs and how to implement them correctly,
visit the links below.

Data Graphs [https://datagraphs.com/] | Knowledge Graph Engineering
[https://datalanguage.com/knowledge-graph-engineering]

Creating a Knowledge Graph in 12 minutes at Ontotext’s Knowledge Graph Forum

Get hands-on and try the Graph Explorer yourself. Explore the data and interconnections, and help everyone understand your knowledge graph.

Graph Explorer - To demonstrate your Knowledge Graph

The Graph Explorer feature on Data Graphs [https://datagraphs.com/] allows you
to navigate through your knowledge graph visually and interact with your data in
a new light. Being able to present your information visually in the Graph
Explorer helps provide the context, meaning, and interconnection of your data,
an essential asset to your digital transformation.

Demonstrate your Knowledge Graph
A clickable graph user interface enables a visual way of explaining how your
knowledge graph can work for your stakeholders and users. This approach enables
a clickable journey around your business data so that it brings the key
information to life. In our experience this approach generates that "lightbulb
moment" that you have been looking for!

Try it yourself on our Media Demo [https://media-graph.datagraphs.io/media-demo]
Click here [https://media-graph.datagraphs.io/media-demo] to try out our Media
Demo, a read-only version of our product made solely for exploration purposes so
that you can try out the amazing capabilities of the product with no strings
attached. This knowledge graph is populated by thousands of data points from
Netflix, Disney, Hulu, and Amazon and has a plethora of features available to
try out including Search
[https://media-graph.datagraphs.io/media-demo/_all/concepts], Advanced Search
[https://media-graph.datagraphs.io/media-demo/_all/concepts?aq=], Path Analysis
[https://media-graph.datagraphs.io/media-demo/_all/_graph-search], and Domain
Model [https://media-graph.datagraphs.io/media-demo/_domain-model].

Kevin Bacon on Graph Explorer
Click here
[https://media-graph.datagraphs.io/media-demo/people/concepts/urn:media-demo:Person:3Vn1Bxu4W6yzLNL54eB4b5/explore] 
to try out the exact same action being completed in the gif below - exploring
the all media data that links to Kevin Bacon... With Graph Explorer you can
navigate visually through his films, the countries of those films, other actors
he's worked with and much more.

[https://media-graph.datagraphs.io/media-demo/people/concepts/urn:media-demo:Person:3Vn1Bxu4W6yzLNL54eB4b5/explore]
What's the shortest path between Sandra Oh and Ricky Gervais?
Click here
[https://media-graph.datagraphs.io/media-demo/_all/_graph-search/?from=Sandra+Oh%28urn%3Amedia-demo%3APerson%3A6XCMzX4fttIARGWlEPynkN%29&to=Ricky+Gervais%28urn%3Amedia-demo%3APerson%3A2AafOAewdEpIBHxVTKAOHg%29&constraints=0%7Bi%5Byear%3Ar%3A1990%3A2020%5D-%3Ecast-%3E*%7D&view=graph?utm_source=socials&utm_medium=twitter&utm_campaign=shortest-path] 
to complete the action below - finding the shortest path between Sandra Oh and
Ricky Gervais, with path constraints on other actors and shows between 1990 and
2020, and then clicking the Graph Explorer button to visualise the path.

[https://media-graph.datagraphs.io/media-demo/_all/_graph-search/?from=Sandra+Oh%28urn%3Amedia-demo%3APerson%3A6XCMzX4fttIARGWlEPynkN%29&to=Ricky+Gervais%28urn%3Amedia-demo%3APerson%3A2AafOAewdEpIBHxVTKAOHg%29&constraints=0%7Bi%5Byear%3Ar%3A1990%3A2020%5D-%3Ecast-%3E*%7D&view=graph?utm_source=socials&utm_medium=twitter&utm_campaign=shortest-path]
And this entire knowledge graph can be created in just 13 minutes in Data
Graphs... Watch this video to find out how.

Find out more about Data Graphs here [https://datagraphs.com/].

Graph Explorer - Demonstrate your Knowledge Graph

Data Language announces promising results from this Innovate UK -funded AI
project, building on their long-running technology partnership with Cochrane. 

Innovate UK Project
Title: “AI Classification of COVID-19 studies to ensure clinical guidance and
healthcare policy is informed by current meta-analysis of all available evidence
”

This Innovate UK grant-funded project ran from August 2020 to April 2021.

Data Language has worked with Cochrane since 2014, designing and delivering a
microservice-based linked data platform that encodes an authoritative knowledge
graph of healthcare clinical evidence. This has included work to refine, iterate
and scale the platform whilst creating a suite of applications to publish
evidence in healthcare as structured data. The platform enabled Cochrane to
rapidly deploy their Covid 19 Study Register in response to the coronavirus
pandemic in 3 weeks last March.

This Innovate UK -funded AI Project has delivered a new capability in Data
Language’s Text AI services to enable automated creation (“clinical coding”) of
PICO Graphs, using values from Cochrane vocabularies and standard medical
vocabularies assigned to Population, Intervention, Comparison and Outcome
structures for clinical analyses.

The specific outcomes from the initial phase of project work can be summarised
as:

 1. We created a working Machine Learning approach.
 2. We have a clear set of next steps.

Following the successful conclusion of the project work to develop the AI
components, the next steps are to focus on key optimizations to move it into
productionization and commercialization. Timeframes are yet TBC.

Results from our Text AI Services Covid-19 Project with Cochrane

In my last blog post, I introduced our new SaaS product Data Graphs
[https://datalanguage.com/blog/data-graphs-fully-managed-knowledge-graphs] - a
cloud hosted Knowledge Graphs as a Service platform - effectively lowering the
barriers to entry into knowledge graph adoption. Data Graphs
[https://datagraphs.com] not only makes it trivially easy to spin up a knowledge
graph for a domain or project, but provides a very intuitive user interface for
curating and managing your data within, along with search, discovery and
integration using a richly featured API.

Since then we have been busy rolling out new features, the most striking of
which are the new visualisations. Data Graphs now lets you automatically
visualize your domain model, and lets you navigate your graph data visually too.

The Domain Model
This visualization is a superb way to verify the domain (or ontology) model you
have created in Data Graphs accurately reflects the true nature of your domain
and the semantic relationships between classes of concepts are as you intended.
This visualisation lets you see the structure of your schema, including all
primitive data properties, relationship properties, and any subclasses you have
defined. 

A typical pattern is to draw your domain model first in some external tool, or
on paper or whiteboard, then construct the schema in Data Graphs. Now you can
visually verify if the models align.

The domain model is visualized using a force-directed graph. If the automatic
render is not perfect,  you can use the controls to adjust the layout. Each node
can be pinned and moved simply by dragging it to the position you prefer.  You
can pin all nodes using one of the control bar buttons on the left of the
diagram. Subclass relationships can be toggle on / off, and you can also
remove/prune classes using the sidebar that lists all the concept types in your
project. When happy with the layout, you can save it, so next time you view it,
it will retain the visual changes you have made.

Graph Explorer
You can explore your knowledge graph visually starting from any concept in your
datasets.

There is an option in the concept view screen to “explore graph”. Clicking this
will open the data visualization using the concept selected as the starting
node.

The graph opens and you then have the ability to explore the graph. Clicking on
a node in the graph opens all its inbound and outbound relationships. It also
renders the data about the node selected in the sidebar on the right.

If a concept has a preview image defined, these will be rendered as the
canonical image for each node.

To navigate at a more granular level, right clicking on a node will open a
context menu where you have the option to open or close specific relationships,
or prune the node itself from the graph visualization.

Like in the domain model, holding and dragging a node pins it the place you move
it to.

I hope these are helpful. We have plans to continually improve these
visualization features over the coming months.

Knowledge Graph Visualization in Data Graphs

Our knowledge graph product Data Graphs [https://datagraphs.com/] has a brand
new feature. You can now view your entire knowledge graph in the 'Domain Model'
view. This allows you to see your data presented as a spider diagram and explore
it interactively by moving each data point to wherever you like. 

Being able to present your data visually is an essential facet in your company's
digitisation journey and this new Data Graphs feature makes this simple.

Find out more about Data Graphs and Data Visualisation here
[https://datagraphs.com/].

Domain Modelling

New Data Graphs Feature: 'Domain Model' View

Data Graphs is the latest Data Language baby. Data Graphs
[https://datagraphs.com] was born out of need, a need for really easy structured
data management. Our thoughts on this evolved to eventually produce a SaaS
product that allows a team to curate a knowledge graph for any domain, from
simple flat concept collections to complex structured domain data. This is all
wrapped up in an extremely intuitive user experience, combined with a fully
comprehensive API and web-hooks to give you extremely efficient integration with
applications and business systems. You can check it out here : 
https://datagraphs.com

What does it do ?
Put simply it provides a dead simple way to create knowledge graph data models,
populate them with data, and interrogate (query) and integrate that data with
your applications using a flexible and easy API.  In essense, it is Knowledege
Graphs as a Service (KGaaS).

Building your own knowledge graph platform can take months, requites highly
specialised expertise and lot of $$, and that is before you start populating it
with data. Even when you build your own KG platform, the information management
tools dont come for free, and are typically sorely lacking in utlity.

We have short-circuited that complex engineering process, and done all the hard
yards for you. Futhermore we have added a stunning information management user
experience. With Data Graphs you can be up and running in minutes, start
creating your graph data models, and managing your data.

You don’t have to worry about scaling, engineering, backups, it’s all handled
for you. But best of all is the super-intuitive user interface for managing,
searching, and maintaining data over time. The API lets you search, query,
filter, and facet on your data - slice and dice it. The API is rocket fast and
easy to use, and also geospatially aware.

We think Data Graphs addresses a lot of use-cases, from straight-forward data
catalogs or vocabulary management, to easy to setup and maintain knowledge
graphs. You could use it for managing structured tagging data for a publishing
system, integrating it with your CMS, or even for maintaining time series
structured data powering charts in WordPress for data journalism. It is ideal as
a central metadata or reference data repository for your enterprise business
systems. You could even use it for maintaining a nicely categorised and faceted
product catalog to integrate with your e-commerce store.

Schemas, Concepts, and Datasets
The Data Graphs model is based around three fundamental ideas - 

Datasets - these are collections of Concepts - organised into groupings of your
choice. Datasets can hold concepts of different types, or concepts all of the
same type. Datasets can also be public or private -

 * a public dataset means the data it holds is accessible via API using only an
   API key, so it could be directly accessible via anyone on the internet, who
   has access to your API key.
 * a private dataset means the data it holds can only be accessed using Open ID
   / oAuth security credentials

Schemas - this is your data model, definitions of the things (or concepts) in
your domain, their properties, and how they are linked together, the
relationships between the things. Data Graphs has an intuitive user interface
that lets you quickly design a data structure that reflects your own domain
model.  Each Concept type in your domain has its own schema definition where you
define the properties of a concept and the data types of those properties
(dates, integers, text etc). For example, a simple schema of a Person might have
a givenName (text), familyName (text), and a dateOfBirth (date), and a headShot 
(image).  Rather than explain it in depth here, if you need to know more, we
have made a short video
[https://datagraphs.com/demos/how-to-setup-your-data-sets] that shows how
quickly you can create a schema for your domain model.

Concepts - the data itself. Concepts are the data instances that you load into
Data Graphs conforming to your schemas. You can create and edit concepts via the
user interface, bulk import them from for example CSV files or external sources,
or load them via the Data Graphs UI. Once created, you can search , discover,
and maintain the data via the UI.

SaaSification
Data Graphs is fully SaaSified - you can sign up online [https://datagraphs.com]
, frictionless, and get started immediately with a free trial. You can manage
your own users, roles, your own backups, your own API keys and oAuth
credentials. 

The User Experience
Delivering a great data management experience was our primary goal when creating
Data Graphs. In almost all the knowledge graph and information management
projects we have worked on, data management and curation tooling was always an
area that was a challenge. We set out to build something extremely intuitive to
use, with a UI that would be self-aware of the data types, the relationship
semantics, and make it super easy for users to explore, find, and navigate data.

Dog Food
We are great fans of the “Eat your own dog food” approach to product
development. If our APIs are not good enough for ourselves to build an
application upon, how can we expect them to be good enough for others? With this
in mind, all the features in the Data Graphs Application are built upon the same
APIs exposed to the Data Graphs client, and with that in mind, let’s talk about
integration..

Integration
There is no point having a fabulous user experience for ingesting and managing
your data if you cannot integrate it. As with most SaaS products Data Graphs
comes with an API. A set of REST endpoints for searching, filtering, querying,
faceting, retrieving and creating data. It is fast, it is flexible.

With the API you can access data only in specified datasets, or you can federate
a query across all your data.

If used with only your API key, you access only the data in public dataset. If
used with your oAuth credentials, you can access both public or private
datasets.

Web Hooks - Triggered
The API gives us pull integration, web hooks provide us with push. You can
configure web hooks for Concept create, update, delete, and also candidate
(concept) approval. You define optional endpoints (typically in your  own
platform) that are invoked whenever a specified Data Graphs event is triggered.
For example you could trigger some downstream system when a Concept of a
specific type is created or updated. For example if an Organization concept is
updated, trigger some workflow in an existing business system. The web-hook
events supply the entire concept data in its payload, which means you could use
this to push data into another database or application.

Querying data - The Data Graphs Filter Expression Language
As well as features such as sort and facet (by concept property) the API lets
you query your data using a flexible filter expression language (often called
NQL).  In simple terms, this lets us filter data like this 
filter=property1:value1,property2:value2

A filter expression is a string which provides a (Concept)  property, operator
and value in the form where property is a JSON path representing the field to
filter on (using JSON dot notation) and operator is optional , and tells the API
how to compare values. value is the value to match against.

Operators can be -(not), >, >=,<<=, and also a test if a value is in a group of
values [value,value,...]

Boolean expressions are also supported including parentheses for grouping
expressions together, and overriding operator precedence. 

Full text search is supported using the traditional q parameter. We index all
the text fields in concepts, and boost the canonical labels.

Combining all these together lets us build extremely high utility data queries,
and very flexible integrations.

You can query within a specified dataset, or you can federate across all your
data, and using the filter expression language it means you can ask extremely
useful questions of your data.

The API is fast - you can expect all requests to be returned in less than a few
hundred milliseconds.

Geospatial
We also support geospatial data. We have primitive datatypes for latitude, 
longitude and geohash - so both our API and UI can be geospatially aware for
concepts with location information. As well as being able to explore geospatial
data in the UI on maps, you can also query geospatially using point-radius,
bounding-box, and bounding -polygon queries. Any geospatial query can be
combined with other API filters and full text search.

This means you can ask questions of your data such as “Find me all the concepts
within 50km of New York, classified by X, with property P greater than Y, and
the word “hello” in their label”. Nice.

For example, given a geospatial data set of Airports with relationship to
airlines , get all airports within a 50km radius of London, serviced by EasyJet,
we might use an API request like :

/airports?filter=_near((51.51, -0.12), 50km),airlines:urn:demo:airlines:easyjet

Client Library
For developers we have published a Javascript client library for Data Graphs to
make integration even easier. The client library is particularly useful for
consuming the API using OpenID credentials, as the machine-machine OpenID
authorization flow is all taken care of.  The client library documentation is
published in NPM here : 
https://www.npmjs.com/package/@datalanguage/datagraphs-client

Sub-classing, Inheritance, and Transitive Inference
While Data Graphs is not a full blown graph database, it is a fully managed
structured data solution that can be used to build a high-utility highly
performant knowledge graph for your organization without having to manage any
infrastructure.  There are some key features in knowledge graphs that add
significant value. 

Subclassing and class inheritance is one of those features. In our (extensive)
experience of working with graph databases, inference is a feature that is
widely misunderstood. The set of potential inference rules available in the full
RDF and OWL specification is considerable. However in most practical
implementations of knowledge graphs (outside of academia), there are just one or
two core inferences that add significant value in most business applications.
Subclassing and transitivity is top of that list. Data Graphs has full support
for this.

When you create a new Concept schema definition, you have the option of
declaring a parent type (class) and inheriting its properties. 

Create a Concept with a parent typeOnce you have created your concept schemas,
we have a neat class structure management tool like this :

Concept relationships (and the User Experience) then become class structure
aware, with transitive inference.  When a concept property defines a
relationship to another concept type/class the UI allows you to select concept
instances of that type and all its subclasses.

Transitive relationship propertiesAPI queries also obey the usual semantics of
transitive inference across a class hierarchy. 

Linked Data and Graph Databases
Data Graphs has support for JSON-LD. API responses can be requested with an 
application/ld+json mime-type such that the data is returned in JSON-LD with a
context dynamically constructed from your schema.

This lets you transform data directly to RDF for loading into your own graph
database, or another RDF compliant system. 

So while functioning capably as a standlone fully managed knowledge graph, Data
Graphs could thus also be used as a tool-suite for editing data in a downstream
enterprise graph database / triple store. Using the web-hooks, it is fairly
trivial to build a service that updates entities in a graph database when the
web-hook is triggered from a concept create/update operation in Data Graphs.

The Road Ahead
Data Graphs already has a plethora of great features
[https://datagraphs.com/features], but we have some great new ones lined up in
our road map. Not least is data and model visualisation.

Building on the class structure visualisation we plan to render detailed domain
model / ontology diagrams dynamically from your schema. We also plan a dynamic,
navigable graph visualisation of the instance data itself. On the back-end, we
plan to offer regional data storage options.

We think we have produced something really special with Data Graphs, something
that really delivers utility and value for information management in your
organisation - but don’t take my word for it, go ahead try it out for free
[https://datagraphs.com].

Data Graphs - Fully Managed Knowledge Graphs

Data Language is proud to be featured in today's 'AI for Business' supplement in
The Times (10th December 2020)

Created by Raconteur, this supplement covers a range of AI solutions across a
spectrum of enterprise use cases. In it we offer our view on how to implement AI
successfully: 

 1. You need a consistent source of reliable data to feed the AI systems.
 2. You need leadership within the organisation who understand the need to make
    faster, data-driven decisions.

The Times : "AI for Business" - 10th December 2020.Read more:
 1. Raconteur AI Section
    [https://www.raconteur.net/topic/technology/artificial-intelligence/]
 2. Data Language Explainable AI Platform
    [https://datalanguage.com/products/datalanguageai/explainable-ai-platform]
 3. Data Language Text AI Services
    [https://datalanguage.com/products/datalanguageai/text-ai-services]

Data Language featured in The Times' AI for Business Supplement

Data Graphs [https://datagraphs.com/] is a knowledge graph SaaS product by Data
Language that provides rapid knowledge graph setup and API provisions. It
enables you to set-up, manage and build apps on your knowledge graph in minutes;
an essential tool in Digital Transformation
[https://datalanguage.com/about/digital-transformation].

What is Data Graphs?

Data Graphs is our brand new a structured data solution that makes creating, managing and organising your data easy. Start your free trial now.

Launching Data Graphs | Our new knowledge graph product

We are very proud to announce the launch of our latest SaaS product - Data
Graphs! We wanted to create a data solution that could provide really easy
structured data management and we believe we've done that with this new
knowledge graph product.

As a fully managed SaaS solution, Data Graphs [https://datagraphs.com/] combines
an intuitive user experience with flexible integration. It allows a team to
curate a knowledge graph for any domain, from simple flat concept collections to
complex structured domain data. This is all wrapped up in an extremely intuitive
user experience, combined with a fully comprehensive API and web-hooks to give
you extremely efficient integration with applications and business systems. 

After working for years with many companies on structured data solutions, we are
thrilled to have now produced a product that can provide businesses with the
structured data they need to embark on their digitisation journey.

Here's how you can explore Data Graphs [https://datagraphs.com/]:

Read more [https://datagraphs.com/] | Sign up for a free trial
[https://datagraphs.com/] | Book a 15-minute discovery call
[https://calendly.com/datalanguage/data-graphs-intro-call]

Launching Data Graphs

Tolga Ergin

Datagraphs Logo on Dark

Data Graphs provides effortless knowledge graphs. Set up, manage and build apps on your knowledge graph in one easy-to-use product. Ideal for MVP launches.

Data Graphs - Effortless Knowledge Graphs

Data Graphs provides you with ‘effortless knowledge graphs’. Whether you are in
publishing, agriculture, finance, law, government or any other sector, this is a
foundational knowledge graph tool for your digital transformation.

You can find out more, and sign up for a free trial of Data Graphs at 
https://datagraphs.com

Setup is Easy
Setup takes minutes, and this easy-to-use SaaS product can be shared with your
team to easily integrate with whichever systems, apps and services you use. 

Intuitive User Interface
As a web application, you can import, create, organize, edit, or consolidate
your data effortlessly with its intuitive user experience.

Powerful API
Connect your data via its powerful and fast API, define your business critical
data and enable your team to explore new opportunities, all managed through the
elegant user interface.

The best way to deploy a Knowledge Graph rapidly
From our years of experience building knowledge graphs for global brands, Data
Graphs enables you to get your knowledge graphs launched, build apps, and
connect services in minutes and hours, rather than days and months!

Data Graphs Strapline

We are developing our Text AI Media Classification Products to help Cochrane automatically classify Clinical Studies onto Snomed-CT and PICO structured data

AI Classification of Clinical Studies with Text AI to fight Covid-19

We've just received the green light on this public-funded innovation project,
bringing two areas of significant Data Language expertise into play to help
combat Covid-19.

The mission combines two of our favourite things:
1. Clinical Evidence Knowledge Graphs
We've been working with Cochrane on their knowledge graph and data strategy
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19] 
since we started the project with them in 2014. As part of this, we recently
helped them launch their Covid-19 Study Register
[https://datalanguage.com/blog/covid-19-rapid-response-technical-delivery-at-cochrane]
, which summarises the current state of Covid-19 clinical evidence from around
the world that has been released into the public domain via authoritative trial
registries and journal databases.

2. Our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] for
automated text classification
Over that time, we have also cemented our position as industry leaders in
machine learning text classification and analytics, and have developed
particular expertise in high volatility domains (e.g. academic research,
breaking news, R&D) and complex ML engineering challenges. This led us to
creating our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services].
[https://datalanguage.com/tagmatic-ai-document-classification]

Mission: to ensure Covid-19 clinical decision making is informed by the latest,
most comprehensive reliable evidence possible.
With the help of the UKRI [https://www.ukri.org/], Digital Catapult's Machine
Intelligence Garage
[https://www.digicatapult.org.uk/for-startups/acceleration-programmes/machine-intelligence-garage]
, and Innovate UK [https://www.gov.uk/government/organisations/innovate-uk], we
are now bringing our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] into play
with our subject matter expert partner Cochrane to help the fight against
Covid-19.

We will be launching this new HealthTech AI beta before December. 

Get in touch if you want to know more.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common specialist and scaling problems.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common digital transformation challenges.

Data Language AI - News and AI Product Updates

Data Language AI

Automated Classification of Healthcare Evidence

Datagraphs Logo

Data Graphs website

We are proud to announce the new member of our Text AI Services
[https://datalanguage.ghost.io/products/datalanguageai/text-ai-services] family:
Inline Entity Extraction. This new AI NLP service was created for high-value
knowledge organizations who want to accurately classify their text media at a
granular, inline-level.

The Problem: Automated Tagging in Custom Domains
We created this service to solve the problems that other Named Entity
Recognition (NER) NLP SaaS - and "build it yourself" approaches - could not
solve.

Our customers are high-value media and knowledge organizations. A huge portion
of their business asset value is locked in their unstructured media assets,
which they must leverage to add value to their subscription, DaaS, targeted
advertising, and other business models. This unstructured content must be
connected to their custom business domain via knowledge graphs and other
structured data systems:-

 * Complexity: To liberate this value, accurate inline text classification of 
   100s of 1,000s of entities is required to enable structured connection with
   bespoke information architecture models and/or knowledge graphs.
 * Custom Domain: Generic off-the-shelf NLP SaaS solutions do not recognize
   these custom entities, so a mapping exercise is required, and is often a too
   low resolution to provide value.
 * Rich Metadata: Other NLP solutions do not provide the rich, domain
   model-aware metadata, because they are not trained for the custom domain.
 * Small Training Sets: Other custom NLP solutions require larger training sets
   and golden corpora that cover the entire target entity set.
 * Cost of Build: "Build it yourself" becomes very expensive, with a large data
   scientist and engineering overheads, while organizations find themselves
   re-solving the same challenges that we have already solved.
 * Cost of Maintenance: "Manage it yourself" has a surprisingly large data
   science and engineering overhead, to overcome production challenges that we
   have already solved.

Our Solution: Text AI Services Inline Entity Extraction
Our Inline Entity Extraction SaaS handles a far greater number of classes than
other NER NLP solutions, while delivering impressive performance results. It
also works out of the box at scale for your production platforms.

Our Inline Entity Extraction SaaS comes with all these benefits immediately: 

 * Custom Domains at Scale: It classifies text content to 100s of 1,000s of
   entities in a custom domain.
 * Powerful API: It provides a rich API for interacting with the machine
   learning service metadata.
 * Does its own ML housekeeping: It automatically manages machine learning model
   ‘fallout’.
 * Self-Optimizing: It self-optimizes its machine learning models, on the fly.
 * Scalable: It is able to scale up indefinitely.
 * Production-Ready: It is Production-ready, out of the box.

All the engineering is done, and the security & maintenance are on us!

For more information on Data Language AI Services, check out the links below.

Data Langauge AI [https://datalanguage.com/products/datalanguageai] | Text AI
Services [https://datalanguage.com/lp/text-ai-services]

Product Launch: Inline Entity Extraction SaaS - AI Custom Domain Auto-Tagging

Jim Howard

Get top tips and insights from Jim Howard on valuable use cases for your Sports Data, and avoid the common pitfalls.

Getting the most from your Sports Data - Insights and Best Practice

My name is Jim Howard and I am a freelance Product Manager specialising in
developing products using data and content. After working at the BBC for a
number of years, I have worked on projects for a number of clients including
Government Digital Service, Eurosport and Stats Perform.

I got to know Data Language after working with some of the team on big
publishing and audience-facing Sport products (World Cups, Olympics etc), when I
was at the BBC and The Telegraph.

We have been discussing some of the issues that we have had to deal with over
the years and the opportunities that are still there for companies to make more
of the data that they consume. In this post, I will try and summarise some of
the key areas we have identified and some areas that could still be developed
further.

Sport, data and content in publishing

Now that the restrictions on Sport are gradually being eased, we thought it
would be an opportunity to review some of the areas where Sport-related products
have been developed and business issues have been addressed with a more
innovative approach to the use of data (not analytics) as a core ‘building
block’ to development. 

As always in organisations, there are lots of great ideas around innovation but
the business needs a focus - and sport is an ideal test subject. As we have
found out, you can look at a major sporting event in isolation (Olympics,
Football World Cup etc), review the outcomes, decide on what areas worked and
then integrate into wider product development strategies. 

I will try and break this down into areas that will be relevant to other
products, not just Sport.

Buying in sports data 
First off, you should look at refocusing Sport as a data proposition and not
just a content one.

Most major publishers typically buy in fixtures, results and tables from a
specialist sports data company. Users, and editorial teams expect that.
Depending on the amount and scale of data, this contract is usually paid
annually and is an ongoing cost. So how do you get maximum value from the data
you are buying?

The data usually contains unique IDs for teams, players and competitions, as
well as additional metadata such as dates, times, locations, rounds
(semi-finals, round 13 etc) and venues. These data feeds also contain names of
players, teams and competitions. A whole wealth of information that we could
exploit much better than we were. 

When we started looking in a lot more detail at the data that was being bought,
you could start to see the opportunities to properly join the dots between the
content proposition, the technical challenges, and what we wanted to build as a
Product. 

It wasn’t perfect the first time round we tried this, although we did get to the
stage very quickly of having a process to tag content and create automated pages
for all players, teams and groups at the 2010 World Cup
[https://www.bbc.co.uk/blogs/bbcinternet/2010/07/bbc_world_cup_2010_dynamic_sem.html]
.

We then realised that as Sport is so consistent, once you had a basic domain
model in place, then you could approach development and products in a different
way. 

Working up a Sport domain modelUsing the data to help your tagging 
Workflow is always a challenge, whether you are introducing a whole new system
or just tweaking an existing one. 

From a product perspective, I want consistent and accurately tagged content to
avoid issues with SEO, duplication, URLs and historical inconsistency across
different systems within a business. 

Other business aspects to consider were ensuring we could align the editorial
style guide, improving editorial team efficiency, enabling consistent reporting
of traffic and ad revenue, allow for cleaner syndication of content and at
times, manage tags in multiple languages (Bayern Munich and Bayern München are
the same team), and managing alternate names and synonyms for your tags.

When we first introduced manual tagging of content, we had issues with
consistency and accuracy. Even with guidance, individuals approached relevance
and accuracy in slightly different ways. 

Taking feedback from this, our next iteration was a semi-automated tagging
process using NLP techniques, where we focussed tagging only on headlines and
first 4 or 5 paragraphs of the content. The copy was cross referenced with the
sport vocabulary we had created from the bought in data and tags suggested to
journalists. 

This created a much quicker and accurate workflow for the editorial teams. 

There were unexpected challenges too, such as when a new CMS allowed creation of
multiple tags by content teams, with no editorial oversight on tag quality. The
CMS allowed audience-facing pages to be created automatically once a tag had
been used several times - even if they were incorrect. This caused major issues
for the editorial team, the SEO strategy and User Experience. 

We revisited the key terms around sport that the data supplier had delivered,
used that as a starting point for a controlled vocabulary for the editorial
team, and then used machine intelligence to automatically retag the archive of
existing articles, video and other content in the CMS, to ensure consistency. 

Linking a content model and data 
Another key part of this process is understanding the editorial workflow around
fixtures and events and where a level of automation can help save time and
effort. 

There is a considerable amount of consistency around how events are approached
within newsrooms, with editorial teams creating match previews, live coverage,
reports, photo galleries, post-match analysis etc, all of which fit within a 
content model
[https://www.cmswire.com/content-strategy/content-modeling-what-it-is-and-how-to-get-started/] 
and data strategy approach. This thinking equally applies to video and audio,
live and clips. 

As part of a workflow review we could see that the editorial teams were taking
time to manually create pages in the CMS in the days before a big weekend of
sport. 

We suggested using the bought in fixture list to drive automated creation of
pages in the CMS for journalists. We identified what templates needed creating
around a match - which links nicely with the content model work - and spun-up
pre-tagged templates in the CMS when the fixtures were announced. 

Before a match starts, we already know the teams, competition, venue, historical
records etc.  Depending on the sport, we will also know the athletes taking part
in a match (football tends to announce starting line-ups about 45 mins before
kick off) although other sports make announcements further in advance. 

Silver Oliver’s workshop flipchart diagram of editorial workflowScaling Up
This data-driven approach allows you to scale quickly. New competitions or
events can be approached from a ‘we just need some new data’ perspective and
this should fit with your ongoing data management strategy. 

This is especially relevant around major events where you may want to cover new
teams or events as part of the Olympics, Commonwealth Games or research has
shown you that your audience wants more detailed club level NFL or NRL coverage. 

If you want to create a new type of written content, such as a match report,
that is an extension of your existing content model. 

When you start adding in multiple media types used for coverage such as
photographs and social media, you can begin to see where consistency of tagging
and annotation is key to creating product propositions. 

These methods can also be used for content gathering as well - from external
suppliers to content supplied by correspondents and social media sources. 

Data visualisation
One of the challenges every media business has, is how do you stand out from the
crowd around Sport? Everyone has the same fixture list (that you have no control
over) therefore a focus on the user experience and a unique proposition helps
you stand out. 

Depending on the amount of data that you are buying, there is a lot of mileage
in offering a unique insight, and deeper analysis into the events or
competition. 

I worked on the Roboblogger project
[https://www.niemanlab.org/2016/06/the-telegraph-is-trying-to-streamline-soccer-live-blogging-with-an-automated-graphic-system/]
, part funded by the Google Digital News Innovation fund
[https://newsinitiative.withgoogle.com/dnifund/]. We really dug into the details
of what was in the football data that we bought and pushed the automated
publishing of infographics as far as we could.

It helped that we had already put in place a controlled vocabulary of sport
terms, which made the consistency between systems much easier. 

We ensured that any graphics created fitted with the visual style guide of The
Telegraph and were getting them published in under a few minutes after we
received data about the events. 

It’s not all about the website or app
Once you start digging into how a publishing business works, you can start to
see how progress could be made by thinking about an ‘open data strategy’ within
your organisation and allowing different systems to use the same vocabulary to
make life easier, although that does start changing the business focus to manage
data better - still a very overlooked area at the moment.

There are still multiple opportunities to join the data dots in businesses.
Especially linking back end systems and archiving with audience facing products.

 * Unlocking content archives. Always handy for Olympics, World Cups, and major
   events.
 * Ensuring you can map multiple, external suppliers to a single tag or ID.
 * Advertising and commercial consistency for teams and competitions and even
   individuals athletes.
 * Providing consistency between content and tags after mergers between
   companies.
 * Electronic Programme Guides (EPG).
 * Video tagging - enhancing video with structured data - both your own video
   and inbound social media video
 * Analysis and reporting.

Sport - or even an individual sport such as football - is a very consistent
domain which makes it easier to integrate into your business model, or at the
very least, look at Sport, as your first step towards utilising data and content
in a structured and consistent way.

The M word
I have avoided writing metadata all the way through this so far. 

Data suppliers provide very consistent products and can help you solve some
business and product issues. Linking together data and content to create better
products both for internal processes and audience-facing or syndication
purposes.

For example, you shouldn't have to rely on editorial teams to rebuild a section
or feature. We can do this based on the data we have. If google enables a new
rich snippet type we can add this markup etc. 

Most media organisations buy in content feeds, and although the assets are
tagged, they won’t be quite the same as the vocabulary that you use within your
organisation. Clarifying that vocabulary will save time and money between
systems and your own team.

The key to structured data is a well structured content model, a data management
strategy and metadata.

If you want to extend the model, then you can have a list of your contributors
and journalists as well. This would enable me to create a feed or even a
subscription based-product based on a question such as:

> “Give me all match reports (content type) written by Joe Bloggs (Author) about
Melchester Rovers (team) in the past 6 months (date range)”
Are you making the most of your bought data and content?
Data management is a challenge and unfortunately it does tend to get lumped in
with data science and analytics within a business.

I prefer to look at the reference data as a core building block for the Product.
I want to look at the details of the data contract to see what is being bought.
Does it fit with the current editorial strategy or is it just a historical add
on? Is this what the audience is looking for or expecting? Do they come to your
site or buy your paper for a specific type of coverage?

The cost of the data contract varies from company to company and of course also
depends on the amount of data bought. One thing in my experience that is
consistent though, is that usually only a small amount of the data is being used
by the business.

The coronavirus pandemic has caused a raft of fixture cancellations,
rearrangements, postponements and economic challenges across the board for
industries associated with sport. 

Meeting new business challenges is going to be about agility and speed of
response to new opportunities. 

Companies will have to be more agile in the future and the key to this agility
is having scalable and well marked up content that can be repurposed. 

The future
Hopefully 2021 will be a big year for Sport with a still to be defined weekly
sporting calendar, a rescheduled Olympics in Japan, a rescheduled European
football championships, a Lions Tour to South Africa and lots more. 

Companies will want to make the most of these opportunities for engaging
audiences and hopefully these examples can start some processes we have been
involved in:

 * Advertising and commercial consistency - for teams and competitions and even
   individual athletes
 * Unlocking under utilised content archives
 * Providing consistency across content and data after mergers between
   organisations
 * Electronic programme guides - a key syndication product and internal tool for
   some organisations
 * Video clips and live coverage

Around major events, other departments within your organisation that don’t
usually do sport coverage, may well want to start creating content about the
event such as documentaries etc that you may want to leverage.

Many of these ideas are not just relevant to Sport, the patterns I describe can
be applied to many domains and publishing any form of content, whether that is
in news, sport, entertainment, healthcare, science, or finance. Adding
geospatial metadata into the mix adds another dimension, and more avenues for
adding value.

Every domain has its own challenges but Sport is a very nice place to start. If
you would like to know more, please do get in touch.

Data Language is an industry leader in Information Architecture, ontology design and practical knowledge graph implementation. We use Domain Driven Design practises and user-centred design to focus information and data design on core business needs.

Find Information Architecture news and product innovation updates here, including ontology design, vocabulary management, domain & data modelling and more.

Information Architecture - News, Product Innovation and Technology

Information Architecture

Getting the most from your Sports Data

A few weeks ago Cochrane launched their COVID-19 Study Register and I described
the data and technical architecture in detail here
[https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
.

I described how PICO graphs [https://data.cochrane.org/ontologies/] are used to
augment the Studies with structured linked data that describe the Population,
Intervention, Comparator and Outcome of the trial. Cochrane have now started in
earnest annotating PICO graphs onto the Studies, and these are now being
surfaced in the Study Register application [https://covid-19.cochrane.org/].

Here, we can see the labels of the PICO concepts rendered in the UI, but in the
data itself, these are arranged as a PICO graph structure conforming to the 
PICO
ontology [https://data.cochrane.org/ontologies/pico/index-en.html].

We have also added faceted search by PICO concept into the sidebar. This lets
users quickly filter down their search results using the Cochrane vocabulary
terms.

Once we have a critical mass of PICOs annotated in the register, each vocabulary
concept will be searchable in the search bar at the top of the app, extending
the previously full-text search into a controlled-vocabulary search. A user can
now find studies using a combination of terms chosen from the Cochrane linked
data vocabulary, to deliver extremely accurate results. Moreover we can combine
a metadata driven search with full-text where needed.

Schema.org
Each Study page rendered in the application is also marked up with schema.org
[https://schema.org]structured data. So while internally we are describing our
content and PICO data with the Cochrane bounded context ontologies, at render
time, we are transforming our model to the schema.org MedicalEntity model to
provide machine readable content for search engines and other content consumers.
You can read more about the schema.org medical schema here.
[https://schema.org/docs/meddocs.html]

As an example, for this Studyhttps://covid-19.cochrane.org/studies/crs-13266121 
when viewing the source of the page, you can see we are marking up the clinical
trial as a MedicalStudy in the schema.org ontology, and identifying the 
healthCondition terms, and the interventions, including the Drug Chloroquine.
Each term has associated medical codes mapped to bioinformatics taxonomies
including SNOMED-CT, MeSH, RxNorm and the ATC codes for Drugs where available.
The resulting schema.org JSON-LD for this study is :

{
  "@context": "http://schema.org",
  "@graph": [
    {
      "@type": "MedicalStudy",
      "id": "http://data.cochrane.org/studies/crs-13266121",
      "name": "NCT04333628",
      "identifier": "NCT04333628",
      "description": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
      "mainEntityOfPage": {
        "@type": "WebPage",
        "url": "https://covid-19.cochrane.org/studies/crs-13266121",
        "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial",
        "description": "Clinical study : Chloroquine for Mild Symptomatic and Asymptomatic COVID-19 in A Two Staged, Multicenter, Open Label and Randomized Trial"
      },
      "healthCondition": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        }
      ],
      "studySubject": [
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "url": "https://data.cochrane.org/concepts/NrO30O5ZnYIkjE",
          "name": "COVID-19",
          "alternateName": [
            "COVID-19",
            "Coronavirus disease 2019",
            "2019-nCoV acute respiratory disease",
            "Wuhan flu",
            "HCoV-19",
            "SARS-CoV2",
            "SARS CoV2 infection"
          ]
        },
        {
          "type": "MedicalCondition",
          "id": "http://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "url": "https://data.cochrane.org/concepts/B5JrjqYxGmF8me",
          "name": "Viral load"
        },
        {
          "type": "MedicalProcedure",
          "id": "http://data.cochrane.org/concepts/r4hp5z141f1t",
          "url": "https://data.cochrane.org/concepts/r4hp5z141f1t",
          "name": "Usual Care",
          "alternateName": [
            "Routine Prenatal Care",
            "Routine Care",
            "Standard Care",
            "Providing care according to standard",
            "SOC"
          ],
          "code": [
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "372921003"
            }
          ]
        },
        {
          "type": "Drug",
          "id": "http://data.cochrane.org/concepts/r4hp13k8jxft",
          "url": "https://data.cochrane.org/concepts/r4hp13k8jxft",
          "name": "Chloroquine",
          "alternateName": [
            "Chloroquine",
            "Product containing chloroquine"
          ],
          "code": [
            {
              "codingSystem": "MeSH",
              "codeValue": "D002738"
            },
            {
              "codingSystem": "SNOMED-CT",
              "codeValue": "14728000"
            },
            {
              "codingSystem": "RxNorm",
              "codeValue": "2393"
            },
            {
              "codingSystem": "ATC",
              "codeValue": "P01BA01"
            }
          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6lz",
          "name": "Pharmacological Interventions"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "url": "https://data.cochrane.org/concepts/kn3ptfq7c6m7",
          "name": "Other"
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        },
        {
          "type": "MedicalEntity",
          "id": "http://data.cochrane.org/concepts/q25g9q497cwj",
          "url": "https://data.cochrane.org/concepts/q25g9q497cwj",
          "name": "Physiological or clinical",
          "code": [

          ]
        }
      ],
      "subjectOf": [
        {
          "@type": "MedicalScholarlyArticle",
          "datePublished": "2020-01-01",
          "url": "https://clinicaltrials.gov/show/NCT04333628",
          "headline": "Chloroquine for Mild Symptomatic and Asymptomatic COVID-19",
          "abstract": "19 COVID is a deadly viral disease that has been spreading around the world for several months, and is caused by a CORONA family virus (COVID-19). Following IN-VITRO evidence of the antiviral effect of CHLOROQUINE in CORONA viruses, this drug has been used empirically for COVID-19 patients and is currently recommended in Israel for the treatment of intermediate and severity disease. The mechanism of action of chloroquine is in part by inhibiting the virus distribution, and changing the intracellular acidity, the virus distribution site. The intracellular chloroquine concentration is determined by a pump called PGP that removes the drug from the cell and is activated by the drug. In the treatment of malaria, the benefit of low dosage of the drug has been shown to be effective due to the fact that the intracellular concentration of the drug is probably higher, and therefore the logic to examine this issue in COVID-19 treatment. The purpose of this study is to test whether a low dose of Chloroquine will reduce the duration of the viral shedding and prevent the disease from worsening",
          "author": [
            {
              "type": "Person",
              "name": "HaEmek Medical Center"
            }            
          ]
        }
      ]
    }
  ]
}

PICO and Schema.org
The PICO model and process (https://en.wikipedia.org/wiki/PICO_process) is now
widely adopted for framing research questions in evidence based healthcare with
the BMJ citing this model as best practice
[https://bestpractice.bmj.com/info/toolkit/learn-ebm/how-to-clarify-a-clinical-question/]
.

PICO data is applied by clinicaltrials.gov in a semi-structured tabular format
on most of their published trials and PubMed also has a PICO search tool for
querying MEDLINE, the  database from the US National Library of Medicine.

Supporting PICO would be a welcome addition to the schema.org model, allowing
any publisher of medical literature to mark up their content in a standard way
with machine readable structured PICO data as Cochrane and others are now. I can
imagine a future where Google indexes the schema.org PICO markup and provides
the ultimate PICO based search tool across all the medical literature citations
at their disposal.

At Data Language, as co-authors of the PICO Ontology, we would be more than
happy to help define some simple extensions to the schema.org MedicalEntity
model to make this happen.

PICO, Cochrane and Schema.org

Analysis of 1 month of video stats from Jamie Oliver Christmas videos shows 10x CTR on Data Moments, and extremely high engagement rates.

Data Moments shows benchmark-beating stats on Jamie Oliver videos

Data Moments is a new type of video experience, where users can interact with
moments in video. For Jamie Oliver, this means getting more from each step in a
recipe video, and highlighting products that are used in the video so that
viewers can buy them directly.

This image shows a "Making thyme flavoured oil" video moment, that had an
interaction rate of 62%.From a data sample of 1 month of Data Moments on Jamie Oliver video
[https://datalanguage.com/news/jamie-oliver-christmas-videos-enhanced-with-luminery]
, here are some highlights:

Video Moment Engagement
 * Challenge: Video experiences haven't changed for years. Can we get a new,
   user-centred experience that dramatically improves how viewers use Jamie
   Oliver videos?
 * Solution: Identify steps in the recipe, and the techniques, products and
   ingredients in those steps. Activate the steps as structured "How To" moments
   using Luminery. Add important products so that users can find them directly.
 * Results: 92 new "moment interactions" per 100 video views. 9.3% Interaction
   Rate with HowTo steps (compared to the current ZERO).

Video Marketing Conversion Performance
 * Challenge: Jamie Oliver produces a range of products, and creates incredible
   recipe videos. How do we drive better product sales from these videos?
 * Solution: Identify products in the recipe and activate the moments to help
   users directly reach and buy the products at the right moments using Data
   Moments (saving huge conversion fall-off due to searching and uncertainty
   barriers).
 * Results: 5.4% CTR (clicks as % of video views) generated by the "product
   moments" (compared to 0.5% of standard pre-roll Ads). 9.6% conversion rate at
   the Ecommerce vendor destination for Jamie Oliver products.

What does this tell us?
This demonstrates, again, the benefit of using structured video moments to
activate new value in Video. We've demonstrated new engagement and revenue
-driving opportunities, and a fresh model for video publishing.

News, features and updates about our video moment product Data Moments. Find out how this innovative product harnesses structured data in video. Brought to you from the team that was foundational in the earliest large scale linked data projects in the UK.

Read news, features and updates about our video moment product Data Moments. Find out how this innovative product harnesses structured data in video.

Data Moments - News and Video Product Updates

Data Moments

Data Moments - Get the latest on this Video Data Moment Product

Data Moment video Stats from Jamie Oliver Christmas videos

Who is Cochrane?
Cochrane [https://www.cochrane.org/] is the global not-for-profit organisation
whose core mission is to put evidence at the heart of health decision-making all
over the world. Over 90% of WHO guidelines are informed by Cochrane’s systematic
reviews.

What is the Covid-19 Study Register?
The Register is a precisely scoped, authoritative source of the latest clinical
evidence from around the world concerning the Covid-19 pandemic, that is being
used to inform point-of-care clinical decision making and evolving healthcare
policy internationally.

Healthcare professionals can use it to locate all relevant, reliable clinical
evidence about specific healthcare questions from the latest Randomised
Controlled Trial (RCT), observational, modelling and other studies to support
the fight against Covid-19.

This crucial, trusted resource is essential at a time when the evidence base is
developing rapidly, as healthcare professionals require structured and reliable
up-to-date information.

Partners with Cochrane since 2014
We first worked with Cochrane in 2014 on a digital transformation project,
designing and delivering a radical new platform architecture centred around
linked-data and knowledge graph technologies. Since then we have worked to
refine, iterate and scale their platform to create a suite of applications to
publish evidence in healthcare using structured data.

Solid data architecture makes rapid innovation possible
This Covid-19 Studies Register is the latest example of how Cochrane is able to
innovate rapidly on the scalable knowledge graph platform architecture delivered
by Data Language. The structure and flexibility means that new use cases can be
delivered rapidly and efficiently.

See the Covid-19 Study Register here [https://covid-19.cochrane.org/].

Find out more
 * Cochrane's Covid-19 Resources and News
   [https://www.cochrane.org/coronavirus-covid-19-cochrane-resources-and-news] 
   (page updated regularly).
 * More about Cochrane and the rapid delivery of the Covid-19 Study Register
   project
   [https://datalanguage.com/blog/covid-19-rapid-response-technical-delivery-at-cochrane]
   .
 * Our work with Cochrane since 2014 - Information Architecture, PICO Ontology
   and more
   [https://datalanguage.com/blog/how-knowledge-graph-technology-is-helping-cochrane-respond-to-covid-19]
   .
 * More about our Knowledge Graph Engineering
   [https://datalanguage.com/knowledge-graph-engineering].

Launching the Cochrane Covid-19 Study Register

Our global expertise in knowledge graph engineering are helping Cochrane respond to COVID-19. We are using linked data and semantic technologies in this mission.

Knowledge Graph Technology vs. COVID-19 | Cochrane & Data Language

How knowledge graph technology is helping Cochrane respond to COVID-19

Our pitch included using our Explainable AI Platform to provide London's analysts with explainable AI (XAI) decision support.

Explainable AI Platform at London Civic Innovation Challenge

We've just been at City Hall for London's Civic Innovation Challenge Awards
Ceremony. Sadly we didn't win on this occasion, but we were proud to be in the
finalists for the Counter Violent Extremism Social Tech Challenge.

Explainable AI Social Tech to help Londoners
Our pitch included using our Explainable AI Platform
[https://datalanguage.com/products/datalanguageai/explainable-ai-platform] to
provide London's analysts with explainable AI (XAI) decision support.

Explainable AI decision support - in this case for violent extremism specialists
Thanks to the GLA Team and Social Tech Trust!
The finalists with the GLA team and Social Tech Trust.

News and Product Updates about our Explainable AI Platform (XAI)

Explainable AI (XAI) means that the process, reasoning and decisions made by the AI are transparent and clear to the users and teams involved. Even before this became a trending technical jargon term, we were delivering Explainable AI products and services.

Updates and news about our Explainable AI Platform (XAI). We are specialised in moving new AI services to production rapidly, avoiding pain points in XAI

News and Product Updates about our Explainable AI Platform (XAI)

Explainable AI Platform

Proud Finalists in London's 2020 Civic Innovation Challenge

GraphQL is a contemporary API technology developed by Facebook. Typically it is
used for querying, manipulating and serializing JSON data structures. Until
recently (as of Jan 2020) it was not normally associated with the querying of
graph databases, but a number of graph database vendors are now providing
GraphQL API interfaces alongside the native query languages like SPARQL and
Gremlin. While it is a very high utility, and extremely developer-friendly API
technology, it is not as expressive or as fully featured as the native graph
query languages.

What is GraphQL, how does it relate to Graph Databases ?

The method of querying a knowledge graph depends on its implementation, they are
typically queried using one or more dedicated query languages. A knowledge graph
built upon a labelled-property architecture (e.g. Neo4J) is typically queried
with either Cypher or Gremlin. A RDF based graph (e.g. GraphDB or StarDog) is
queried using SPARQL. Graph database vendors are also starting to provide
GraphQL interfaces as a query mechanism, but this does not provide the
expressiveness and feature-capability of the native query languages. These query
languages are typically aimed at developers and data specialists, for business
users, the ability to navigate and ask questions of the graph is typically
abstracted into microservices, search interfaces, visual graph explorers, or
tailored user-experiences to meet specific use-cases.

How do you query a knowledge graph ?

Graph Databases come in two main archetypes: Labelled-Property Graphs (LPGs),
and Resource Description Framework (RDF) based graphs.

With a labelled-property graph, nodes and edges both can have metadata
(properties) assigned to them in the form of key-value pairs. They are tuned for
querying and analyzing Paths through the graph, and do not typically support
semantic reasoning.

RDF based graph databases are based upon the W3C RDF standard, and store data in
the form of triple-statements (subject-predicate-object). The predicates
(relationships) that join nodes together confer semantic meaning upon the data.
RDF graph databases can be reasoned upon, and can infer new assertions
(statements) from the semantic data model.

What types of graph databases are there ?

A graph database is a NOSQL database based on graph-theory that stores data in
indexed collections of nodes and edges, where the nodes represent domain
entities, and the edges represent the semantic relationships between the
entities. Node-Edge-Node relationships (statements) can be chained to create
navigable arbitrary-length paths through the data. Graph databases can be
queried using one of a number of query languages depending on the database
implementation type. They can typically store many billions of assertion
statements, and are often used as an implementation technology for knowledge
graphs. Graph databases typically form the foundation of a knowledge graph - but
not a necessary requirement.

What is a Graph Database ?

Knowledge graphs are becoming an important and integral part of an
organisation's data landscape. They provide a human and machine readable
database of all the things of interest to the enterprise in their domain. The
knowledge graph typically describes the domain entities and the semantic
relationships between them. It provides interfaces to query the graph as
structured data to drive products and user-experiences both internal and
external to the organisation. It is evolvable, and the knowledge stored can grow
and mutate as the business itself grows and changes. Its value lies in its
ability to allow an organisation to ask questions of the things it cares most
about.

What is a Knowledge Graph ?

Data Language have helped Cochrane transform from a document centric
organisation to a data centric one. Their knowledge of linked data and data
strategy is world class. They deliver.

Cochrane

The Google knowledge graph contains over 1 billion entities, and 70 billion
facts (assertions) about those entities. The majority of Graph Databases today
will comfortable scale to many billions of statements.

The Linked Open Data Cloud [https://lod-cloud.net/] contains some 1,239 datasets
such as Wikidata, DBpedia, Geonames, IMDB, BBC Music, data.gov.uk, across
domains including media, life-sciences, government, social and geography.

1,239

We helped Cochrane develop a knowledge graph for evidence-based health care with
some 350k concepts covering populations, interventions and outcomes, linking to
industry biomedical vocabularies such as MeSH and SNOMED-CT.

350k

A discussion on the synergies between GraphQL and Graph Databases, data architecture and how ontology models provide the special ingredients.

The synergies between GraphQL and Graph Databases

5 latin characters
Until recently, my understanding was that apart from 5 latin characters GraphQL
has got little or nothing to do with Graph Databases. But things are changing,
Graph Database vendors have started bringing these two technologies together. A
recent project has given me some insights.

For some background, GraphQL is a contemporary API interface standard used as an
alternative to REST. GraphQL was originally developed by Facebook. It provides a
structured query language for querying (and mutating) data, along with a runtime
implementation to invoke those queries upon your data. In pretty much all
implementations it uses JSON as its serialization format.

REST (I suspect most readers know this) is a lightweight low-level API standard
for performing uniform and a predefined set of stateless operations on data
aligned with the HTTP verbs. With REST you have to design and  build your API
interface contract for that set of operations - and while of course there are
common patterns in REST APIs, this is very much arbitrary. REST makes no
assumptions about the high level API contract or the underlying data model. The
developer defines both. With GraphQL you only define the data model (the GraphQL
schema), exposing to the GraphQL runtime and your consumers what the shape of
the data is.

With a GraphQL query you can request only the minimum data required to fulfil
your current need. Of course you can do this with a REST API too - in fact it is
a common pattern with JSON REST APIs to filter response data by JSON fields
using dot notation. But of course you need to build this yourself.

GraphQL does a lot more for you to boot, as well as out of the box query
patterns for mutating data, it also supports subscriptions (pub/sub style) - so
your GraphQL client can subscribe to server side events, and have data pushed to
it.

The developer ecosystem around GraphQL is superb now - The Apollo GraphQL
[https://www.apollographql.com] implementation works beautifully with React -
integrating with React Hooks for query and subscriptions. Apollo also has
in-browser developer tools akin to the redux-devtools for GraphQL debugging, and
you can manage local app state via Apollo using the GraphQL schemas / queries
etc.

Caching and performance
Requests to GraphQL are typically all POST requests - not great for caching you
ask ? Indeed, this is where REST has excelled for years. But the GraphQL server
implementations have come to the rescue. The Apollo GraphQL server has an
internal caching layer that supports full response caching, and fine grained
field-level caching (read more here
[https://www.apollographql.com/docs/apollo-server/performance/caching/] )

Amazon’s AWS AppSync is a fully managed GraphQL service built upon the Apollo
technologies - and can source data out of the box from other services such as
DynamoDB, ElasticSearch, REST APIs etc. With fully managed caching and real-time
data subscriptions, it really does make your developer life easy.

On the surface it still appears GraphQL has little to do with Graph Databases.
Yet increasingly we are hearing about how they should go together like bread and
butter, sausage and mash.

Not all graph databases are equal
I won't go into depth about what a Graph Database is, but am going to explore
some of the options available, and the relationship between Graph Databases and
GraphQL.

Graph databases come in several flavour combinations. First up you have RDF
graphs vs Labelled-Property graphs. The first big choice. Both are very
different [https://en.wikipedia.org/wiki/Graph_database#Graph_models]. Your
primary use case is important here. Essentially if you primarily need to walk,
query and analyse paths through your graph, then Labelled-Property graphs are
probably the way to go. If you are more interested in the nature of the
relationships between things, and creating rich semantic representations of
things, then RDF graphs hit the spot. Both types have their own well-developed
query language. Labelled-Property graphs are typically queried with Cypher or
Gremlin (eg Neo4J), RDF graphs are queried with SPARQL.

While most mainstream RDF based graph databases are fully compliant to the RDF
and SPARQL W3C standards, not all RDF graph databases are equal. Each has its
own implementation nuances and come with their own added extras (decisions,
decisions!):

 * An RDF graph database may or may not support reasoning (to some degree of the
   RDFS and OWL standards).
 * Reasoning may be at query time (backward chaining) or at write-time
   (forward-chaining).
 * The way reasoning is implemented can dramatically impact performance.
 * Add-ons and extensions for GeoSPARQL, Path analysis, Full text search,
   connections to other datastores etc.

To give some examples (DISCLAIMER - apols to those vendors not mentioned, I am
only referencing those I have worked with properly hands-on, and this is not an
exhaustive features list - it’s just a flavour) :

GraphDB [https://www.ontotext.com/products/graphdb/] (Ontotext)

 * Fully compliant with RDF/SPARQL standard;
 * Highly compliant and configurable OWL / RDFS forward-chaining reasoning
   (materialises inferred knowledge at write time);
 * Unparallelled full-text-search capability using lucene, or via bi-drectional
   connectors to external SOLR or ElasticSearch clusters.
 *  Ability to materialise post-inference fully structured JSON or JSON-LD
   objects into ElasticSearch indices or MongoDB
 * Geospatial querying and other extensions..
 * Rich visual query and management workbench UI
 * UPDATED May 2022 - Path query capability is now avaialble using a SPARQL
   extension plugin

Neptune [https://aws.amazon.com/neptune/] (Amazon AWS)

 * Fully compliant with RDF/SPARQL standards;
 * No reasoning capability
 * Fully managed cloud service
 * Highly, highly horizontally scalable (primarily due to lack of reasoning)
 * Full text search capability via connector to ElasticSearch - materialisation
   into elastic is constrained (you cannot materialise user-modelled JSON
   objects into an elastic index), but delivers good full-text search utility.
 * You use this either as a Labelled-Property graph (using Gremlin) or an RDF
   graph (with SPARQL)
 * No query workbench/admin UI - you need to install your own 

Stardog [https://stardog.com]

 * Fully compliant with RDF/SPARQL standards;
 * Highly compliant and configurable OWL / RDFS backward-chaining reasoning
   (materialises inferred knowledge at query time);
 * Extremely fast online write times (due to backward-chaining model)
 * Adequate full-text-search capability using internal indexing
 * An outstanding SPARQL extension for PATH analysis (Pathfinder) - you can
   query for paths, shortest paths with weighted edges, normally only possible
   with labelled-property graphs
 * Geospatial querying and other extensions..
 * Rich visual query and management workbench UI

So you can see that each of these three are fully W3C compliant RDF / SPARQL
graph databases. But each has one or more killer features, that the others do
not. 

GraphDB - the post-inference connectors to ElasticSearch and MongoDB unlock some
outstanding technical and data architecture patterns out of the box -UPDATED May
2022 - GraphDB now also has comprehensive Path query capability using sparql
extension plugin

Stardog - combines the semantics of RDF with sophisticated Path querying and
analysis.

Neptune - fully managed cloud service with unparalleled horizontal scalability,
but no reasoning.

The upshot - choose wisely based on your requirements now, and what is likely to
matter most in the medium term.

..and then enter stage right, GraphQL

Sausage and Mash or Pineapple on Pizza ?
How do GraphQL and Graph Databases fit together? As we learned earlier, GraphQL
is a REST API alternative with a structured query language combined with a
runtime for query processing and serving data to GraphQL clients (and it is
almost always implemented with JSON in mind).

So why are Graph Database vendors getting involved and what do you get for your
money?

The cynical marketer in me says those 5 latin characters definitely count for
something. Which sales manager wouldn’t love the potential energy pressure in
the ready-to-explode singularity of Graph Databases and GraphQL (while humming
‘Don’t stop me now’).

However, GraphQL does only provide another query mechanism for your graph
database alongside SPARQL or Gremlin.  Moreover it is much less expressive, and
most definitely not designed for exploiting the rich semantics of RDF and OWL. 

In fact if you go all in on a michelin-starred Graph Database, and only use
GraphQL to query it, I would argue you really probably don’t need a Graph
Database - a NOSQL JSON store would probably suffice. Of course, if you have
forward-chaining inference in your graph database (as with GraphDB), then your
GraphQL queries will benefit from the RDFS/OWL semantics you have modelled your
data with, which does add weight to the value-proposition.

Still hungry ?
Ontologies are an important part of the picture. In the RDF world an ontology
semantically describes the model and nature of your domain, and ultimately the
data structures stored in your knowledge graphs, and if inference-aware, they
describe the meaning of the data, and how we reason upon it. Typically these
models are communicated up and down our tech stack (and the responsible
developers), and often all the way out to consumers of your APIs (public or
enterprise-internal).  While we typically move data around using JSON the
linked-data aficionados will know well that if we do this using JSON-LD, we can
use the same ontology models from the knowledge graph (or a subset) for
describing the JSON-LD payloads we ship around outside of our graph database.

So do we need GraphQL to take advantage of these ontological graph models ?.  No
we don’t, we can do it with normal REST APIs (querying our graph with SPARQL)
and using JSON-LD payloads - but of course we have to build more API code to do
this.

With GraphQL though we can (we must) formally describe our data structures as
GraphQL schemas, and so we have a great opportunity here to align our GraphQL
schemas with those same ontology models that describe the data in our knowledge
graph -  this could be either a subset of our ontology landscape, or possibly
the superset, if we are using other GraphQL (served) data sources aside from the
data in the graph database. This is starting to look like very nice decoupled
data architecture with well structured cohesive data models, with possibly the
most efficient mechanism for consuming data in our web/mobile applications (a
GraphQL client).

It is looking more tasty
Adding GraphQL client interfaces to a Graph Database does make life easier - if
you are rapidly prototyping straight from the graph, or federating data into an
external GraphQL server there is a nice pattern for making a common set of well
described data models.

The Graph Database vendors have indeed jumped on-board the GraphQL sushi train
(and it’s not all about those 5 latin characters). Stardog and GraphDB (from
Ontotext) both offer GraphQL clients now.

At this point I was going to show an example of how GraphQL might be used to
query a Graph Database, but Jem Rayfield at Ontotext has already done the job in
his superb Star Wars blog post
[https://www.ontotext.com/blog/the-rise-of-the-knowledge-graph/] (if you have
got this far, then this is a must read).

Ontotext has taken the next logical step and created a ‘Semantic Object Service’
 that automatically generates GraphQL schemas bound to the underlying RDF data
model  in GraphDB, moving us towards the ontology ubiquity I described above,
where your domain ontologies are expressed in the knowledge graph, and across
your wider data-estate, including your GraphQL schemas. Nice work.

Jem concludes by saying :

> GraphQL is a very popular graph query language, well used, known and liked by
developers. The Ontotext Platform uses GraphQL to lower the barrier of entry to
knowledge graph data, whilst still providing the full expressivity and power of
SPARQL.
and this is a key message, which equally applies to Stardog and any other graph
database that may yet offer a GraphQL client. While GraphQL lowers the barrier
to entry, GraphQL gives you only as much as it is able to with respect to
querying and graph traversal;  to really unlock the power of the graph database,
you will need to use SPARQL.

But if you never plan to, and you are not interested in reasoning either, then
possibly you should ask yourself if you really need a graph database ?

What if you want to use GraphQL as the primary API technology in your stack, but
want to use SPARQL too to access the graph database (amongst other tech)?

A mixed data technology case study
In a very recent engagement, the Data Language team have been working on a Smart
Cities IoT project. The nature of the domain meant the type of data we needed to
deal with was very ‘graphy’ in nature, and the ability to describe our data
semantically and traverse a graph with SPARQL was of high utility. Inference was
less of a concern. The proof of concept for the project was brought together in
AWS with :

 * AWS MSK - a high performance Kafka message bus to consume, orchestrate, and
   distribute IoT sensor events
 * AWS Neptune to give us our scalable RDF based smart city knowledge graph
 * A set of JSON-LD REST API endpoints invoking SPARQL queries on the graph
   running in Kubernetes (EKS)
 * A suite of Kafka event processor consumers and producers deployed into
   Kubernetes mutating the graph, and emitting new events based on our business
   logic rules.
 * AWS AppSync - the GraphQL engine sourcing events and the JSON-LD REST APIs
   from Neptune, providing the primary developer interface to our client
   applications
 * A ReactJS built UI using GraphQL react-hooks to subscribe to real time events
   data, and to query data from the sensors and our knowledge graph (ultimately
   via expressive SPARQL queries).

A simplified view of the architecture looks essentially like this :


The kicker was modeling data across the entire stack using a suite of
ontologies. These ontology models describe our knowledge graph data, our IoT
message payloads in JSON-LD and were all federated in the primary application
API expressed as GraphQL schemas in AppSync. Building the ReactJS front end
using the Apollo GraphQL client and react-hooks was a very nice developer
experience.

Dessert
While GraphQL technically has little to do with Graph Databases, they are not a
perfect match like sausages and mash, GraphQL is just another API / query
technology, and not as expressive as the native graph database query languages
like SPARQL, but in itself is a high-utlity, very sweet technology.

There are some clear synergies, and do provide some of that warm comfort food
feeling when used together. GraphQL does give you an easily understood and very
developer friendly way into a graph database. Once you add the secret sauce of
domain ontologies describing both your knowledge graph and possibly even your
entire data-estate, expressed through your GraphQL schemas, then you have a very
nice pattern for a decoupled data architecture with well structured, nicely
communicated data models across your technology stack, and out to your consumers
too - making for efficient application development using the GraphQL ecosystem.
Get stuck in.

GraphQL and Graph Databases

Data Language chosen for the Scaleup Programme.
We are delighted to announce that we have been chosen for the Innovate UK and
EEN Scaleup Programme. This programme supports a select group of companies on a
journey to exponential growth, providing them with countless opportunities with
bespoke, fully-funded one-to-one support.

We are immensely proud of this achievement as admission onto the Scaleup
Programme is a highly selective process. For the first cohort 2,800 companies
made up the initial long list of applications with only 30 selected, so to be
chosen as one of the successful companies in the new cohort has given us even
greater confidence in our ambition.

We can’t wait to get to work with the Scaleup team to start bursting through
barriers to growth and taking Data Language to its full potential.

Scaling Up

The Jamie Oliver team is using Data Moments - our structured data video moment SaaS - to create new "How To", promotion and Product promotion video moments.

Jamie Oliver uses Data Moments for Christmas Videos

We're really pleased to announce the launch of the Jamie Oliver Christmas "How
To" video feature, powered by Data Moments
[https://datalanguage.com/products/datamoments], our SaaS structured data video
moments platform.

Jamie Oliver is using our platform to activate and enhance the valuable moments
within his Christmas videos.

Data Moments contextual commerce "Product Moments"
A product promotion "moment" on Jamie Oliver's Christmas Video Feature.Data
Moments "How To" video moments.
The Jamie Oliver team are interested in whether users used videos "to cook
along" - we are using our HowTo Video Moments to help users navigate between
important steps.

Jamie Oliver uses Data Moments

In September 2019, we enhanced Rugby World Cup video using our Data Moments product to activate their video. 50% of their key moments were in Google top 10

Video Micro-Moment Performance in Rugby World Cup 2019

Data Moments [https://datalanguage.com/products/datamoments] is a new type of
video experience, where users can interact with moments in video. These are
unique opportunities we call Video Micro-Moments, where we can use Data Moments
to meet user needs
[https://datalanguage.com/video-micro-moments-meet-user-needs-at-their-strongest] 
in the actual moment that they occur.

For the Rugby World Cup in 2019, this means enabling rugby fans to discover,
navigate and enjoy the great sporting video moments with ease.

Video Moment Reach
 * Challenge: J SPORTS have unique sporting moments within their videos, that
   cannot be discovered by search engines or users with normal video publishing.
 * Solution: Markup the key sporting events, teams and stadiums with Data
   Moments [https://datalanguage.com/products/datamoments] structured data video
   key moments that are optimised for SERPs (Video SEO).
 * Results: 45% of video moments created for J SPORTS Rugby World Cup ranked in
   the first page of Google SERPs (Search Engine Results Pages). This created a
   completely new and significant SEO surface area, growing reach by 37%, and
   creating Google Discover opportunities too (with 4.2% CTR), 

Video Marketing Conversion Performance
 * Challenge: J SPORTS run an E-Commerce store selling sporting goods, plus a
    Sports SVOD product, and want to increase the traffic from video highlights
   through to conversions.
 * Solution: Create "Buy in the moment" contextual commerce moments, linking
   directly to products at key video moments, meeting the user's "I want to buy"
   video micro-moment need.
 * Results: 1.1% CTR - significantly beating the 0.5% standard for conventional
   video advertising.

Read the latest news and updates about Data Moments video key moments
[https://datalanguage.com/about/datamoments].

Data Moments Stats - Video Key Moments in the Rugby World Cup

J SPORTS is using Data Moments - our video key moment SaaS product - to activate video key sporting moments and brand moments for the Rugby World Cup.

We're really pleased to announce that J SPORTS is using Data Moments
[https://datalanguage.com/products/datamoments] (our SaaS Video Enhancement
Platform) to enhance video moments for their Rugby World Cup highlights.

We have been working on this project with our innovation partners at EAI
[https://www.entertainmentai.co.uk/] to help J SPORTS use our innovative video
moment enhancement technology for SVOD, E-commerce and Sport Event micro-moment
promotions.

Why is J SPORTS using Luminery?
J SPORTS were particularly excited about the Data Moments
[https://datalanguage.com/products/datamoments] activation of video moments that
effectively meet user needs exactly when they arise. Google's marketing
evangelists call these moments "Micro moments
[https://www.thinkwithgoogle.com/marketing-resources/micro-moments/micro-moments-understand-new-consumer-behavior/]
".

Sporting Event Micro-Moments - "I want to know"
With Data Moments [https://datalanguage.com/products/datamoments], each sporting
moment - tries, conversions, line-outs, penalties etc - has a new opportunity
listed on SERPS, shareable, and brand activated.E-Commerce Micro-Moments - "I
want to buy"
With Data Moments [https://datalanguage.com/products/datamoments], each brand
activated moment can feature Contextual Commerce activated promotions for
products, when they are actually relevant to the user!J SPORTS Enhanced Video
Homepage
At: video.jsports.co.jp [https://video.jsports.co.jp] - the videos are only
available for users in Japan, due to RWC rights restrictions.

The J SPORTS With Data Moments [https://datalanguage.com/products/datamoments] 
Video HomepageGet in touch [https://datalanguage.com/contact-us] if you want to find out more.

Data Moments activates video moments for The Rugby World Cup

Tom Fox from the "Hi Growth Unit" at KPMG awarded us with the KPMG Prize at
Venturefest in Oxford for our contextual video moment innovation pitch.

(Update: Luminery has rebranded as "Data Moments
[https://datalanguage.com/products/datamoments]")

Selected from an impressive spread of startups 
The pitching participants.Luminery Demo
The team demonstrate Luminery enhanced video moments at VenturefestBranding
Update @ 2020
"Luminery" has been re-branded as "Data Moments
[https://datalanguage.com/products/datamoments]".

News and Product updates about Luminery, our state-of-the-art Video Micro-Moment SaaS product. Luminery enables Publishers, Brands and Creators to leverage the value of video micro-moments for increased reach, revenue and engagement.

Read News and Product updates about Luminery - the Video Micro-moment SaaS Platform that increases video reach, engagement, revenue and social capabilities.

Luminery - The Video Micro-Moment Platform - News & Product Updates

Luminery

Luminery wins KPMG Prize at VentureFest

Data Language has been accepted onto the Digital Catapult Machine Intelligence
Garage [https://www.digicatapult.org.uk/projects/machine-intelligence-garage] 
program.

This is fantastic news for us. Through MI Garage, Digital Catapult is going to
provide us with plenty of high performance infrastructure and technical support
to power our machine learning and AI product R&D throughout 2019.

This will allow us to rapidly advance innovation and development across all our
products, running our ML models at scale, and on GPU kit. It will enable us to
expand our Tagmatic [https://datalanguage.com/tagmatic] ML product suite for
automated content classification; our deep learning image classification
platform, SCOPA; and help with automation on our Contextual Commerce platform, 
Luminery
[https://datalanguage.ghost.io/2019/02/06/digital-catapult-machine-intelligence-garage/luminery.com]
.

2019 is going to be a great year for Data Language!

AI is a core discipline at Data Language. We have an expert team skilled in machine learning and deep learning, and importantly we also have the engineering and deployment skills to deliver AI at scale for your production systems.

AI is a core discipline at Data Language: Our expert team is skilled in machine learning and the engineering skills needed to deliver AI at scale.

Artificial Intelligence delivered at scale - our core discipline.

Artificial Intelligence

Digital Catapult - Machine Intelligence Garage

We had some more great news. Innovate UK
[https://www.gov.uk/government/organisations/innovate-uk] has awarded Data
Language a new innovation grant for further funding experimental development and
proving the market of our Luminery [https://luminery.com] platform to deliver
crowd and machine sourced contextual commerce- and brand enhanced- video
moments. 

Our aims are to drastically improve the advertising revenue publishers can make
from video, to improve discovery and reach at a video moment -level, and also to
provide a fair reward mechanism for crowd workers.

You can see a working prototype [https://luminery.com/food-example] here
demonstrating how our contextual commerce works.

Innovate UK

Innovate UK funding for Luminery (Data Moments)

As part of our Text AI Services SaaS product range
[https://datalanguage.ghost.io/products/datalanguageai/text-ai-services], we are
proud to announce the launch of our cutting-edge Text Classification service.

The Problem: Automated Tagging in Custom Domains
We created this service to solve the problems that other NLP Text Classification
products could not solve.

Our customers are high-value media organizations. A huge portion of their
business asset value is locked in their unstructured text content:-

 * Aboutness: To liberate this value, an accurate classification of content
   "Aboutness" is required to classify content to the organization's bespoke
   information architecture models and/or knowledge graphs.
 * Custom Domain: Generic off-the-shelf NLP SaaS solutions do not recognize
   these custom aboutness classifications.
 * Rich Metadata: Other NLP solutions do not provide the rich metadata, because
   they are not trained for the custom domain.
 * Rapid Learning: Other NLP solutions can't learn on the fly, and require
   expensive and time-consuming training cycles to learn new concepts.
 * Cost of Build: "Build it yourself" becomes surprisingly expensive, with a
   large data scientist and engineering overheads, and organizations find
   themselves re-solving the same unforeseen challenges that we have already
   solved.
 * Cost of Maintenance: "Manage it yourself" has a surprisingly large data
   science and engineering overhead, to overcome production challenges that we
   have already solved.

Our Solution: Text Classification AI
Our Text Classification SaaS trains itself and learns your team's in-house style
while delivering impressive performance results. It also works out of the box at
scale for your production platforms.

Our Text Classification SaaS comes with all these benefits immediately: 

 * Customized to your Domain: It classifies text content to 100s of 1,000s of
   entities in a custom domain.
 * Powerful API: It provides a rich API for interacting with the machine
   learning service metadata.
 * Does its own ML housekeeping: It automatically manages the machine learning
   model ‘fallout’.
 * Self-Optimizing: It self-optimizes its machine learning models, on the fly.
 * Scalable: It is able to scale up indefinitely.
 * Follows your in-house style: It learns the tagging style of your SME team.
 * Production-Ready: It is Production-ready, out of the box.

All the engineering is done, and the security & maintenance are on us!

Data Language are specialists in AI Text Classification. This includes automated tagging, machine learning, language agnostic systems, self-optimisation, and specialist text classification areas.

Read about products and projects in AI Text Classification at Data Language. This includes automated tagging, machine learning, language agnostic and specialist text classification areas.

AI Text Classification - News, Technology and Product Updates

AI Text Classification

Data Language AI: Text Classification SaaS

Tagmatic is a service for automatically tagging your content.

It uses sophisticated machine learning techniques to learn how you tag your
content on-the-fly and then will tag new content automatically.

Tagmatic uses your own tag dictionaries. Whether you tag with simple text terms,
identifiers, taxonomies, or linked data, Tagmatic will tag with those. 

There is no data mapping exercise needed, just send your content & tags to
Tagmatic and it gets to work. Tagmatic quickly learns new tags too.

It is incredibly easy to integrate into your CMS or editorial workflows via two
simple web service interfaces.

If you are looking for an automated tagging solution, then look no further.

If you are developing your own automated tagging solution, then pause, give
Tagmatic a try - and be up and running quickly.

Update @ 2020: 
Tagmatic has been renamed as "Text AI Services: Text Classification
[https://datalanguage.com/products/datalanguageai/text-ai-services]" as part of
our "Data Language AI" product suite
[https://datalanguage.com/products/datalanguageai].

News, Blog Posts and Innovation updates about Tagmatic - our sophisticated AI automated content classification SaaS product. We have taken care of all the engineering and data science for you: Tagmatic learns on the fly, is self-optimising, and language-agnostic.

Tagmatic is an AI automated text classification SaaS product by Data Language. It learns on the fly, is self-optimising, and language-agnostic.

Tagmatic - News and Product Updates about our Text Classification AI

Tagmatic

Introducing Tagmatic

Text AI Services combine machine learning and AI to provide world-class text classiciation. Self-optimising, language agnostic and full managed.

Launching Text AI Services | Efficient Text Classification

We have just launched our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services], a fully
managed, SaaS-based set of text services using AI and Machine Learning to
automate key processes. As essential tools in any digital transformation
journey, our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] are the next
generation in NLP and text analytics.

With decades of experience in understanding and solving pain points in
structured data, we have now developed a world-class service for ai text
classification. Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] are perfect
for out of the box machine learning to make your text tlassification and
recommendations more logical, accurate and efficient.

Find out more about Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] | Book an
intro call with the Text AI Team
[https://calendly.com/datalanguage/data-language-ai-intro-call]

Launching Text AI Services

JSON-LD has been among us for several years now, its adoption is increasing, but
I suspect not as fast as it probably should. JSON-LD arose out of the RDF
community, subsequently adopted as a W3C standard, as a pattern for representing
RDF and linked data (hence 'LD') as JSON, with a key aim of making RDF much
easier to consume for developers.

While Google has raised awareness of JSON-LD considerably as it’s preferred
pattern for marking up web pages with schema.org [https://schema.org] structured
data, JSON-LD however has only made relatively small inroads into mainstream web
development. There are number of reasons for this, not least that the W3C and
JSON-LD working group did not do a great job of communicating its worth and
explaining how to use it effectively, although this has improved a lot more
recently, the excellent (and long overdue) JSON-LD API best practices guide
[https://json-ld.org/spec/latest/json-ld-api-best-practices/] published in
February 2018 is a very good example.

I am going to attempt to help the cause too, and make a convincing case of why
JSON-LD should be used in mainstream software development, regardless of whether
you are working with RDF.

A Very Brief Overview
I want to avoid making this post a full-on lesson about JSON-LD and RDF, so if
you are not entirely sure what JSON-LD is, here is super brief 101.
Alternatively get stuck into the full JSON-LD 1.0 specification here
[https://www.w3.org/TR/json-ld], and the draft 1.1 spec here
[https://json-ld.org/spec/latest/json-ld/].

Roots in RDF :
JSON-LD is a W3C standard for representing RDF [https://www.w3.org/RDF/] in JSON
format.
It is a standard pattern (schema) for representing the key elements of RDF
resources and the statements about them, such as the resource identifier (the
IRI) of the object being represented and the rdf type (class) of object being
described. For example, you will often see snippets of JSON-LD looking like this
:

{
  "@id": "http://data.example.com/organizations/1",
  "@type": "Organization",
  "name": "Apple"
}


If you are familiar with RDF, you will be aware that most RDF formats typically
involve a proliferation of IRIs/URIs everywhere. JSON-LD however alleviates this
by abstracting base URIs into a context. For the example above, if this was
written without a context in valid JSON-LD it would look more like this :

{
  "@id": "http://data.example.com/organizations/1",
  "@type": "http://data.example.com/some-model/Organization",
  "http://data.example.com/some-model/name": "Apple"
}


While verbose and ugly, in RDF this can be represented as, or transformed into
long form statements (triples) like this :


<http://data.example.com/organizations/1> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://data.example.com/some-model/Organization> .
<http://data.example.com/organizations/1> <http://data.example.com/some-model/name> "Apple" .



You can test the validity and see how JSON-LD can be expanded into RDF using the 
JSON-LD playground here
[https://json-ld.org/playground/#startTab=tab-nquads&json-ld=%7B%22%40id%22%3A%22http%3A%2F%2Fdata.example.com%2Forganizations%2F1%22%2C%22%40type%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2FOrganization%22%2C%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Fname%22%3A%22Apple%22%7D]
.

The Context :
Introducing the ‘context’ into the JSON-LD makes this much more palatable. The
context defines the fields in the data payload :

{
  "@context": {
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name"
  },
  "@id": "http://data.example.com/organizations/1",
  "@type": "Organization",
  "name": "Apple"
}


In this example, the original data from my first snippet, the type and name
properties of the resource are explicitly identified in some schema or ontology,
so as a consumer of this JSON you are aware of the model that this data was
described with. Without the context the JSON cannot be transformed to RDF
statements. As a consumer of course you can choose whether to ignore the
context, or use it depending on your own need.

This same JSON-LD is often written such that that the context and payload are
separated using the @graph construct as follows. This is in fact identical data
to that above, it just identifies the data payload and the context that
describes it as separate JSON objects.

{
  "@context": {
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name"
  },
  "@graph" : {
    "@id": "http://data.example.com/organizations/1",
    "@type": "Organization",
    "name": "Apple"
  }
}


Back in RDF world, this also lets us create a named-graph which would then allow
us to generate RDF quad statements instead of triples through the addition of an
identifier for the graph :

{
  "@context": {
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name"
  },
  "@id" : "urn:my:organizations:1",
  "@graph" : {
    "@id": "http://data.example.com/organizations/1",
    "@type": "Organization",
    "name": "Apple"
  }
}


Which if you try this
[https://json-ld.org/playground/#startTab=tab-nquads&json-ld=%7B%22%40context%22%3A%7B%22Organization%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2FOrganization%22%2C%22name%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Fname%22%7D%2C%22%40id%22%3A%22urn%3Amy%3Aorganizations%3A1%22%2C%22%40graph%22%3A%7B%22%40id%22%3A%22http%3A%2F%2Fdata.example.com%2Forganizations%2F1%22%2C%22%40type%22%3A%22Organization%22%2C%22name%22%3A%22Apple%22%7D%7D] 
in the json-ld playground will give you RDF statements :

<http://data.example.com/organizations/1> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://data.example.com/some-model/Organization> <urn:my:organizations:1> .
<http://data.example.com/organizations/1> <http://data.example.com/some-model/name> "Apple" <urn:my:organizations:1> .


To make this more concise the context does not need to be explicit in each
payload, it can be remote, so a consumer can choose whether to dereference it or
not :

{
  "@context": "https://data.example.com/contexts/organization",
  "@id" : "urn:my:organizations:1",
  "@graph" : {
    "@id": "http://data.example.com/organizations/1",
    "@type": "Organization",
    "name": "Apple"
  }
}


This also means contexts can be cached for higher performance.
So there you have it, RDF in JSON form. JSON-LD has brought RDF out of the labs,
and made it presentable with a highly consumable JSON representation for
developers.
Or has it...

Mythbusters
Myth 1 - frequently spoken by developers : “it’s not like normal JSON”
While the introduction of JSON-LD is undoubtedly a massive deal for the RDF
development community, it has not been widely embraced by software developers.
Most developers will not have heard of RDF, and even if they are consuming
JSON-LD from some RDF or Graph Database backend, they won’t (and probably don’t
need to be) concerned with it.

A key design goal
[https://json-ld.org/spec/latest/json-ld/#design-goals-and-rationale] in the
JSON-LD specification is :

> Zero Edits, most of the time - “JSON-LD ensures a smooth and simple transition
from existing JSON-based systems. In many cases, zero edits to the JSON document
and the addition of one line to the HTTP response should suffice“


However, when developers first come across JSON-LD it is rarely like this. The
majority of published JSON-LD is littered with JSON-LD nuance. Developers do not
like consuming many of the JSON-LD constructs, and when you hear the complaint
“it’s not like normal JSON” it is typically for good reason. For example, two of
the most common complaints are :

The "@" prefixes :
The “@” prefix on the JSON properties is not a valid ES5 or ES6 prefix for
javascript variables, meaning when dereferencing JSON as a JS object, you cannot
do things like

const id = myObject.@id ;

Instead you need to code

const id = myObject['@id']

Arrays vs Singletons :
JSON-LD output from some SPARQL query or RDF transformation typically does not
confer any indication if a property is an array or a singleton. The result of
one request to some back-end API serving JSON-LD might give :

{
  "@context": "https://data.example.com/contexts/organization",
  "@id": "http://data.example.com/organizations/1",
  "@type": "Organization",
  "name": "Apple",
  "employs": {
    "@id" : "http://data.example.com/people/1",
    "@type": "Person",
    "name" : "Tim Cook"
  }
}


A future request may return an array for the employs property :

{
  "@context": "https://data.example.com/contexts/organization",
  "@id": "http://data.example.com/organizations/1",
  "@type": "Organization",
  "name": "Apple",
  "employs": [
    {
    "@id" : "http://data.example.com/people/1",
    "@type": "Person",
    "name" : "Tim Cook"
  	},
    {
    "@id" : "http://data.example.com/people/2",
    "@type": "Person",
    "name" : "Jony Ive"
  	}
  ]
}


The software developer consuming this output then has to code around this when
evaluating the value of the employs property, and check whether the property
contains an array or a singleton.

I imagine these are possibly the two most common JSON-LD complaints from devs.
Even Google publish their schema.org JSON-LD with @id and @type field names on
the Google Knowledge Graph API [https://developers.google.com/knowledge-graph/].
This doesn't help, and yet it doesn’t have to be like this. Instead of telling
developers to “get over it”, as publishers of JSON-LD we can make JSON-LD look
and behave like normal JSON using the context, meeting the design goal of the
specification. However, it is not always obvious how to make this happen, here
are some simple examples on how to address the two common issues above.

Addressing the “@” issue
Use context aliasing to abstract the “@” symbol away from the data payload. By
creating aliases in the context for those properties prefixed with @, they can
be appear in the data payload fully JS compatible. The context does not just
allow us to provide definitions of our domain-specific JSON properties, but we
can also alias the JSON-LD standard fields. Try this
[https://json-ld.org/playground/#startTab=tab-nquads&json-ld=%7B%22%40context%22%3A%7B%22type%22%3A%7B%22%40id%22%3A%22%40type%22%7D%2C%22id%22%3A%7B%22%40id%22%3A%22%40id%22%7D%2C%22Organization%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2FOrganization%22%2C%22name%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Fname%22%2C%22employs%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Femploys%22%7D%2C%22id%22%3A%22http%3A%2F%2Fdata.example.com%2Forganizations%2F1%22%2C%22type%22%3A%22Organization%22%2C%22name%22%3A%22Apple%22%2C%22employs%22%3A%5B%7B%22id%22%3A%22http%3A%2F%2Fdata.example.com%2Fpeople%2F1%22%2C%22type%22%3A%22Person%22%2C%22name%22%3A%22Tim%20Cook%22%7D%2C%7B%22id%22%3A%22http%3A%2F%2Fdata.example.com%2Fpeople%2F2%22%2C%22type%22%3A%22Person%22%2C%22name%22%3A%22Jony%20Ive%22%7D%5D%7D] 
in the JSON-LD playground :

{
  "@context": {
    "type" : {
      "@id" : "@type"
    },
    "id" : {
      "@id" : "@id"
    },
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name",
    "employs" : "http://data.example.com/some-model/employs"
  },
  "id": "http://data.example.com/organizations/1",
  "type": "Organization",
  "name": "Apple",
  "employs": [
    {
    "id" : "http://data.example.com/people/1",
    "type": "Person",
    "name" : "Tim Cook"
  	},
    {
    "id" : "http://data.example.com/people/2",
    "type": "Person",
    "name" : "Jony Ive"
  	}
  ]
}


Handling collections :
Use the context to define which fields are arrays, and output them as such :

{
  "@context": {
    "type" : {
      "@id" : "@type"
    },
    "id" : {
      "@id" : "@id"
    },
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name",
    "employs" : {
      "@id" : "http://data.example.com/some-model/employs",
      "@container" : "@set"
    }
  },
  "id": "http://data.example.com/organizations/1",
  "type": "Organization",
  "name": "Apple",
  "employs": [
    {
    "id" : "http://data.example.com/people/1",
    "type": "Person",
    "name" : "Tim Cook"
  	},
    {
    "id" : "http://data.example.com/people/2",
    "type": "Person",
    "name" : "Jony Ive"
  	}
  ]
}


Forcing the set (or list) property to be marshalled as an array in JSON even
when it is a singleton. See this in action
[https://json-ld.org/playground/#startTab=tab-compacted&json-ld=%7B%22%40context%22%3A%7B%22type%22%3A%7B%22%40id%22%3A%22%40type%22%7D%2C%22id%22%3A%7B%22%40id%22%3A%22%40id%22%7D%2C%22Organization%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2FOrganization%22%2C%22name%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Fname%22%2C%22employs%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Femploys%22%7D%2C%22id%22%3A%22http%3A%2F%2Fdata.example.com%2Forganizations%2F1%22%2C%22type%22%3A%22Organization%22%2C%22name%22%3A%22Apple%22%2C%22employs%22%3A%7B%22id%22%3A%22http%3A%2F%2Fdata.example.com%2Fpeople%2F1%22%2C%22type%22%3A%22Person%22%2C%22name%22%3A%22Tim%20Cook%22%7D%7D&frame=%7B%7D&context=%7B%22type%22%3A%7B%22%40id%22%3A%22%40type%22%7D%2C%22id%22%3A%7B%22%40id%22%3A%22%40id%22%7D%2C%22Organization%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2FOrganization%22%2C%22name%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Fname%22%2C%22employs%22%3A%7B%22%40id%22%3A%22http%3A%2F%2Fdata.example.com%2Fsome-model%2Femploys%22%2C%22%40container%22%3A%22%40set%22%7D%7D] 
in the JSON-LD playground :

{
  "@context": {
    "type" : {
      "@id" : "@type"
    },
    "id" : {
      "@id" : "@id"
    },
    "Organization" : "http://data.example.com/some-model/Organization", 
    "name" : "http://data.example.com/some-model/name",
    "employs" : {
      "@id": "http://data.example.com/some-model/employs",
      "@container": "@set"
    }
  },
  "id": "http://data.example.com/organizations/1",
  "type": "Organization",
  "name": "Apple",
  "employs": [
    {
    "id" : "http://data.example.com/people/1",
    "type": "Person",
    "name" : "Tim Cook"
    }
  ]
}


Using a remote context (that most web developers can safely ignore), we now we
have JSON-LD that looks just the way your developers would prefer, and meeting
the design goal of the JSON-LD specification :

{
  "@context": "https://data.example.com/contexts/organization", 
  "id": "http://data.example.com/organizations/1",
  "type": "Organization",
  "name": "Apple",
  "employs": [
    {
      "id" : "http://data.example.com/people/1",
      "type": "Person",
      "name" : "Tim Cook"
    }
  ]
}


Myth 2 - “JSON-LD is only needed if you are working with RDF or Graph Databases”
If you are building a web application or micro-service which is using RDF, then
JSON-LD is clearly a no-brainer. However if you don’t care about RDF why bother
?

I argue that you should bother. If you are publishing an API you may not know
how your consumers are using your data. They may well be using a graph database,
or dealing with RDF or linked data and be very keen to consume your API in an
RDF compatible format. Furthermore, publishing JSON-LD requires you to publish a
context. You are thus constraining your JSON to a schema. This is a good thing.
It communicates a model of your domain, provides a payload contract for your
consumers (or a contract between your own micro-service APIs and applications),
and a validation mechanism.

Why we should all adopt JSON-LD
JSON Entropy
Being inherently a schemaless message format, applications built with components
that use JSON to serialize/deserialize data moving between them have a tendency
over time to create JSON property soup. As requirements evolve, and developers
are left to their own devices to modify features and adapt the codebase,
properties are introduced into JSON on an ad-hoc basis. JSON entropy increases.
What may have started out as a simple, clean JSON model, tends towards one of
mess and complexity.
Adopting JSON-LD as your message format reduces this entropy escalation, as :

a) it makes you as a developer think about the schema (structure) of the JSON
from the perspective of your consumers, and the wider application as a whole.
b) once we start publishing schemas for our data, we start to think like a data
architect rather than as a developer.
c) we are more likely to collaborate on the schema design, and at the very
minimum, ask for a peer review of a proposed change.

While this may seem like an overhead, it is a relatively small investment into
your data architecture that pays dividends over the long term.

Schema
The JSON-LD context is a schema for your data, not only defining the property
datatypes but also the classes of json resources and, via referenced ontologies,
the semantics and relationships between properties and classes in your data.
The json-schema [http://json-schema.org/] project has attempted to fill this
space for JSON, but has not really gained too much traction. While it is a
little richer in schema-definition features than what a JSON-LD context
provides, JSON-LD is an official W3C standard, and offers a lot more than
json-schema can with respect to linked data and RDF compatibility.

We all can publish linked data
If you are using linked data or RDF, then you will no doubt already be familiar
with JSON-LD, and hopefully using JSON-LD as a message format in your own
applications. If you are publishing linked data then it is very likely you are
publishing it as JSON-LD. If you are serving public APIs as regular JSON, then
publishing JSON-LD instead makes a lot of sense. As discussed, you should be
able to transition with little or zero change to your data payload. All it takes
to make the transition is to publish a context. For little additional overhead,
you and your consumers benefit. It is essential however, that we, as publishers
of JSON-LD make it as consumer friendly as possible (and this includes Google).

JSON-LD

Publishing JSON-LD for Developers

For those of you that have decoded the title, then the Q1415055
[https://www.wikidata.org/entity/Q1415055] is probably self evident. Having
spent several weeks building a practical application upon Wikidata
[http://wikidata.org], I thought it was time for an honest assessment. There are
indeed, good, bad and ugly facets to Wikidata. I have my tin hat on.

The good
Always best to start with the positives. Wikidata does was it says on the tin.
It does work. It is a broad, extensive knowledge base, based on the principles
of linked data, a superset of the data, machine-extracted from wikipedia. It is
crowd and bot curated. Anybody can contribute via its RDF enhanced Mediawiki UI.
In the true spirit of linked data it has outbound links to other open knowledge
bases, Google knowledge graph, BBC Things, Geonames, and MusicBrainz to name a
few, and also links to the information and images served up by its older
sisters, Wikipedia and Wikimedia.

Provenance claims can be asserted on any wikipedia statement/fact - in the form
of references and sources.

It gets better - Wikidata has a public SPARQL 1.1 endpoint, and a rather nice UI
[https://query.wikidata.org] for querying it. Again this works, and seems to
deal with contention well. Over the course of several weeks and a lot of
querying on this endpoint, I never experienced any issues.

However...

The bad
It’s the model. I am struggling to work out what was the thought process that
went into this. For those of you familiar with RDF
[https://en.wikipedia.org/wiki/Resource_Description_Framework], you will
understand RDF is a W3C framework and standard for creating models that describe
some domain (ontologies), and describing things in that domain using statements
that conform to those models. The RDF schema [https://www.w3.org/TR/rdf-schema/] 
has patterns and rules for defining classes of Things, and the properties that
describe them. Statements such as
<A> rdf:type <C> declares that resource <A> is an instance of some class <C>.
<D> rdfs:subClassOf <C> declares <D> to a be a subclass of <C> .
Object Oriented software developers have a natural affinity to this form of
modelling

Wikidata is built upon the RDF framework. But instead of describing wikidata
resources (things) using the RDF schema, wikidata has built its own resource
description framework out of RDF (the resource description framework), thus
creating an entire abstraction layer between its own model of the world and RDF.

Wikidata resources are divided essentially into two distinct sets. A set of
property resources and a set of instance resources, known as items. The items
set is the set of all the things in the realm of human knowledge. The set of
properties is the collection of semantic predicates that items can be described
by. The property resources all have unique IDs prefixed with P and the item IDs
are prefixed with Q.

Now here is the kicker. Instead of using the RDF schema to define the properties
and classes, wikidata have defined their own schema that sort of mirrors the RDF
schema. There is a wikidata property P31
[https://www.wikidata.org/wiki/Property:P31] “instance of” that is semantically
equivalent to rdf:type. Property P279
[https://www.wikidata.org/wiki/Property:P279] “subclass of” is semantically
equivalent to rdfs:subClassOf. Classes themselves are declared as items, and
items are then described using these properties (not using the RDF schema).

If you are now confused, you probably should be, as anyone with any experience
of RDF being introduced to wikidata will be. Why does wikidata just not use RDF
schema you will ask yourself. I did and have no good answer (yet). This
abstraction layer, means wikidata loses all the benefits of RDF schema, and
gains a frankly painful amount of confusion. Any consumer of wikidata first
needs to understand the wikidata schema. Semantics in wikidata have been
redefined - badly in my opinion.

and the ugly
It’s the obfuscation. So what’s wrong with opaque identifiers I hear you say.
Indeed nothing, it makes perfect sense that all the Items (the things), should
be identified by globally unique opaque IDs - the dataset is large and
crowd-curated, conciseness and consistency works, names of things may change, 
hackability
[http://www.jenitennison.com/2009/07/25/opaque-uris-unreadable-uris.html] is not
a concern. In fact the one in the title of this article I have no problem with.
But I do have a problem with the wikidata properties being obfuscated too. The
entire set of wikidata properties including those that are the schema itself
(the one that is not the RDF schema) are opaque and not human understandable.
This makes building queries and applications on wikidata really painful and time
consuming.
Ontology models are like a programming language, informing the user how to build
things using its model. Obfuscating the model is akin to obfuscating a
programming language. Imagine if all the lemmas, grammars, and reserved words in
Java or Python were not human readable, but encoded as a set of opaque IDs. Yes,
it is the same with ontologies, they need to be human understandable too (at
least until human developers are entirely replaced by AIs). Wikidata is
essentially a giant obfuscated graph of nodes and edges. You have no idea of the
semantic meaning of an edge without dereferencing the edge via a query or the
wiki.
The reason we write code that is self-documenting is well understood. Thoughtful
naming of variables and functions is critical in delivering code that is deemed
to be self-documenting. The same principle should apply to ontologies.

For a few dollars more..
More bad - There is some fairly unusual ontology model when you dig below the
surface. For example Clint Eastwood (Q43203
[https://www.wikidata.org/entity/Q43203]), an instance of human (Q5
[https://www.wikidata.org/entity/Q5]) has occupation (P106
[https://www.wikidata.org/wiki/Property:P106]) actor (Q33999
[https://www.wikidata.org/entity/Q33999]). Actor is an instance of profession -
all good so far. But actor (Q33999) is also a subclass of artist, creator and
person. Thus semantically the range of occupation includes the class “person”.
Clint Eastwood is an instance of human, and has the job of being a person.

More good - back to the rather good SPARQL query UI. The UI provides very nice
hover and autocomplete features on wikidata properties and resources that
de-obfuscates the opaque IDs. This provides some mitigation. If it wasn’t for
this building queries would be difficult.

The SPARQL service (provided by the graph database, Blazegraph
[https://www.blazegraph.com/] that wiki data is served from) has some very handy
plugin services enabled, not least the label & description binding service. This
allows you to write a SPARQL query and bind instance labels very easily, in any
available language, without having to write all the joins and language filters
in your query. For example :


# get actors in the film "The Good, The Bad, and the Ugly"
SELECT ?actor ?actorLabel ?actorDescription WHERE {
  ?actor wdt:P106 wd:Q33999 .
  wd:Q41483 wdt:P161 ?actor .
  SERVICE wikibase:label { bd:serviceParam wikibase:language [AUTO_LANGUAGE],en,es,it" }
}



In the query above, (try it
[https://query.wikidata.org/#select%20%3Factor%20%3FactorLabel%20%3FactorDescription%20where%20%7B%0A%20%20%3Factor%20wdt%3AP106%20wd%3AQ33999%20.%0A%20%20wd%3AQ41483%20wdt%3AP161%20%3Factor%20.%0A%20%20SERVICE%20wikibase%3Alabel%20%7B%20bd%3AserviceParam%20wikibase%3Alanguage%20%22%5BAUTO_LANGUAGE%5D%2Cen%2Ces%2Cit%22%20%7D%0A%7D]
) the label service automatically binds label and description variables to any
named variable in the query. If you a querying for many variables, this is a big
time saver.

Roll credits..
I do have reservations about allowing the crowd to curate the underlying model
as well as the knowledge itself. The model is something that needs strong
governance, and solid grasp of information architecture to ensure quality and
semantic correctness.

All up, I have enjoyed working with wikidata, if you can invest the time in
getting to grips with the schema and model, it is a superb, extensive source of
machine readable information, the best of the current breed, with comprehensive
links to open linked data sets (the primary reason we used it as knowledge
source). If only the model was self-documenting, and it used RDF for the purpose
it was designed.

We have been working with linked data since our inception and have helped global brands with linked data strategy and implementation. It is a core discipline at Data Language. Linked data is a W3C backed pattern for publishing structured data using vocabularies like schema.org that can be connected together and interpreted by machines.

We have been working with linked data since our inception and have helped global brands with linked data strategy and implementation.

Linked data - proven expertise

Linked data

Wikidata - Q41483

We had some fantastic news this week. Innovate UK
[https://www.gov.uk/government/organisations/innovate-uk], part of the UK
Government Technology Strategy Board has awarded Data Language an innovation
grant to fund the development of a platform to deliver crowd and machine sourced
contextual commerce and structerd data on video moments.

Our aim is to structurally enhance videos at a moment level, which will then
improve the revenue publishers can make from video. The approach will also
provide a fair reward mechanism for crowd workers.

Watch this space!

Resolute Square Launches, powered by Data Graphs for speed to market

Related work

Why did Resolute Square choose Data Graphs?

The Knowledge Graph combines Single Customer View with Single Subject View

Another innovation in this project - Video Streaming

Our Products

Related work