Alexi Buckingham

Data Language provides AI products and scalable machine learning solutions for global brands.

AI Engineering By Knowledge Graph Experts

AI for High Value Knowledge Organisations
The Problem: Valuable knowledge locked away.

 1. Organisations have high value knowledge assets distributed across systems.
 2. Archives and services are not connected, and the value is locked away.

How we solve it: Liberate and connect your information.

 1. Our AI SaaS products are easily integrated and trained, using your own
    content.
 2. Simply plug in our services, and run your content through the APIs.
 3. Our SaaS learns on the fly, and scales up to whatever volumes you require.
 4. No need for cumbersome training cycles, or for Data Scientists to oversee
    it.

Want to find out more?
Book a meeting with us - we love helping with any type of project.

AI Engineering

Matt Shearer

Data Language is proud to be featured in the "AI for Business" supplement in The
Times (10th December 2020). 

This supplement covers a range of AI solutions across a spectrum of enterprise
use cases.

To implement AI successfully
 1. You need a consistent source of reliable data to feed the AI systems.
 2. You need leadership within the organisation who understand the need to make
    faster, data-driven decisions.

The Times : "AI for Business" - 10th December 2020.Read more
 1. Raconteur AI Section
    [https://www.raconteur.net/topic/technology/artificial-intelligence/]
 2. Data Language Explainable AI Platform
    [https://datalanguage.com/products/datalanguageai/explainable-ai-platform]
 3. Data Language Text AI Services
    [https://datalanguage.com/products/datalanguageai/text-ai-services]

The Times "AI for Business" Supplement

Our Text AI Services are easy-to-integrate SaaS tools that use AI and machine learning to make document classification as efficient as possible. 

Document Classification Text AI Service by Data Language

AI Document Classification
Data Language has been tackling the problem of document classification for many
years and has used their wealth of experience in NLP, data science, and text
analytics to create a product that is world leading. 

Our Text AI Services are easy-to-integrate SaaS tools that use AI and machine
learning to make content categorisation and tagging as easy and efficient as
possible. Our Text AI Services dramatically reduce the cost and effort
associated with tagging and organising written content, speeding up
behind-the-scenes processes and ensuring accuracy.

With our Document Classification Text AI Service, the difficult engineering and
scaling is done for you, dramatically reducing the total cost of ownership of
developing AI based classification in-house.

Document Classification Text AI Service main features
 * Fully managed Software as a Service, trivial to integrate into your systems
 * Language agnostic
 * Classifies content using your own vocabularies and dictionaries
 * Self optimizes and learns on-the-fly independently to your workflow
 * Dramatically reduces the time, cost and complexity of tagging content
 * Gives you more time to do what you do best

Benchmarking
We have benchmarked our Document Classification Text AI Service against Reuters 
RCV1 and Arvix Academic Paper Dataset AAPD and achieved impressive results.
 Read more about the benchmarks here
[https://datalanguage.com/blog/benchmarking-tagmatic].

Our Document Classification Text AI Service is perfect for almost any text
classification problem, and in any language, some of its primary use-cases are : 

Publishing
Our Document Classification Text AI Service was originally built to tackle the 
challenges that publishers [https://datalanguage.com/tagmatic-for-publishers] 
and media organisations have with automatic and human-assisted content tagging:
reacting to new terms (tags) fast as they break in news, automatic on-the-fly
training, consistency, disambiguation, editorial tagging style  - all this is
handled out of the box by our service. 

Law and Legal Tech
Our Document Classification Text AI Service is perfect for classifying legal
texts and documents, or part-of-documents such as contract clauses. Once
trained, it can be used to automate the tagging of incoming legal documents, as
well as archives of existing documents. We have demonstrated Document
Classification Text AI Service on Hansard of some 200,000 parliamentary
interventions for the 2005/6 period , classified with >7200 categories from the
parliamentary library classification vocabulary, and achieved extremely accurate
results.

Healthcare and Life Sciences
Through our work with Cochrane, we have extensive experience with bioinformatics
vocabularies such as SNOMED-CT, MESH, and MedDRA. Our Document Classification
Text AI Service is ideally suited for the automated classification and tagging
of content against these vocabularies, whether for classifying clinical trials,
 patient records, or general medical documents.

Free Trial
If you are a publisher or work with high volumes, or large content archives and
are working on or thinking of building your own automated tagging platform, then
our Document Classification Text AI Service is an ideal solution - you can be up
and running in hours, and reduce your total cost of ownership of building,
scaling, and hosting your own AI / NLP based classification solution
dramatically. 

Please get in touch for a free trial.

News and product updates about our Text AI Services. Read about how our Text AI products are being used in innovation projects, and by our clients.

Read the latest on our Text AI Services. Projects, Innovation initiatives, and new developments for AI features in specialized use cases.

News and Product Updates about our Text AI Services

Text AI Services

Data Language AI Products

Data Language AI for News offers a new market-leading automated news classification SaaS. It beats other SaaS because it uses your own tags.

Data Language AI for News - Automated News Content Classification

Automatically Tag your text with News Tags
We’ve trained our Data Language Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] using a News
training corpus, and are making it available for you to try out.

Simply add your text, and get News tags back. Get the latest News tags,
consistently, and improve your user journeys and SEO.

Industry-leading Auto-tagging AI
Our sophisticated Text AI automated classification service
[https://datalanguage.com/products/datalanguageai/text-ai-services] is designed
with years of experience in Data Science. It beats popular benchmarks, and
tackles all the normal data science issues faced with News auto-tagging.

Get access
Simply use the contact form to request access.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common specialist and scaling problems.

News, features and updates about our Data Language AI Products. Find out how our AI Products solve common digital transformation challenges.

Data Language AI - News and AI Product Updates

Data Language AI

Data Language AI for News - Automated News Classification

Paul Wilton

Data Language's Text AI Services for Publishers provides automatic classification, entity extraction, relationship extraction, and recommendations.

Text AI Services for Publishers - Our Industry-leading SaaS

Our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] were
originally designed to meet the challenges of automated content workflows in the
publishing industry, and are ideally suited to AI classification and automated
tagging of news, sport and entertainment content.

Why do publishers tag their content?
Most large publishers need to tag or classify their content for a number of
reasons :

 * to assist internal processes and work-flows
 * to improve search and discovery
 * to provide richer content and user analytics
 * to drive advertising
 * to deliver richer user experiences

Typically tagging is a manual task, integrated into the editorial workflow and
the CMS. Increasingly, publishers are working to automate this process using NLP
and machine learning techniques.

Automated content tagging
Automating content tagging unlocks a lot more value, as the automated-tagging
system can be used to

 * improve tagging efficiency, consistency and quality on the news desks
 * fully-automate tagging of external news feeds
 * unlock value in long-tail content
 * automate tagging of other content around the organisation

For this reasons, publishers are investing large sums and time in building their
own automated tagging systems. However what may seem like a simple problem in
the R&D labs can turn into a much more complex and expensive problem, when put
into production.

How Data Language Text AI Services can help
O [https://datalanguage.com/tagmatic]ur Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] are an AI
and Machine Learning fully managed SaaS product. We built them to make it as
simple as possible to integrate automated tagging and classification into your
publishing systems, addressing all the key challenges :

 1. It is incredibly easy to integrate. Send it content to learn. Send it
    content to tag.
 2. It uses your own dictionaries, taxonomies, and classification vocabularies
    out of the box.
 3. The engineering challenge of scaling, retraining, and efficiently
    redeploying the machine learning models is done for you.
 4. It learns on the fly. As new terms arise in news, our Text AI Services
    [https://datalanguage.com/products/datalanguageai/text-ai-services] learn
    them quickly.

Total cost of ownership
Building you own AI based tagging solution is no mean feat. It is not just a
data science problem, but involves software engineering, testers, DevOps,
infrastructure, scaling automation, model deployment automation, continuous
machine learning model training automation, as well as the project management
and business processes that accompany a project of this nature. Both the total
cost of ownership and time-to-deliver is substantial.

With our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] you can be
up and running in hours, not months, at a fraction of the total cost and you can
put your data scientists to work on your market differentiating data outcomes
instead.

Get in touch
Use the contact form to get in touch - we are here to help.

Text AI Services for Publishers

Tolga Ergin

A great AI or ML model is worthless if it can't be delivered in a scalable, cost effective and technically sustainable way. Find out how.

Data Science at Scale - 10% AI, 90% Engineering

A great AI or ML model is worthless if it can't be delivered in a scalable, cost
effective and technically sustainable way. Our data science team excels in the
design and implementation of machine and deep learning models for the analysis
of data and media at scale.

Data Science seems to have something mystical about it, to the extent that
enterprises have almost come to consider it as a magical solution to impossible
problems. The reality is that behind DS there are tons of unfashionable data
engineering and analysis tasks, before statistical models can be applied to the
problem. 

Our team of talented experts here at Data Language can leverage years' worth of
technical excellence to ensure your AI solution can actually be scaled and
delivered at the highest standards of software engineering.

Understand your data 
Unlock the full potential of your data through early discovery, analysis, and
visualisation.

Learn fast
With so many variables to tweak, rapid feedback and short iteration cycles are
imperative to an effective delivery.

Models are software
As with any piece of code, statistical models also benefit from engineering best
practices for delivering at scale, with controlled total cost of ownership.

--

Read our white paper on rethinking how you do data science in your organisation:

 
[https://datalanguage.com/static/whitepapers/rethinking-data-science-in-the-enterprise.pdf]

Data Science at Scale

Fabio Colasanti

We have recently completed some new "Inline Entity Extraction" benchmarks. This
is a sophisticated "Named Entity Recognition" (NER) product that we launched as
part of our Text AI Services suite in 2020
[https://datalanguage.com/news/inline-entity-extraction-text-ai-ner]. It is
optimized for Knowledge Graph platforms in high-value knowledge organizations.
It identifies entities inline within text media and attributes rich metadata.

The Problem: Automated Tagging in Custom Domains
Our customers are high-value media publishers, knowledge organizations, and DaaS
providers. 

 * A huge portion of their business asset value is locked in their unstructured
   mixed media assets.
 * To liberate this value, accurate inline text classification of 100s of 1,000s
   of entities is required to enable structured connection with bespoke
   information architecture models and/or knowledge graphs.
 * Generic off-the-shelf NLP SaaS solutions do not recognize these custom
   entities, so a mapping exercise is required, and is often a too low
   resolution to provide value.
 * Many NLP solutions do not provide the rich, domain model-aware metadata,
   because they are not trained for the custom domain.
 * Other custom NLP solutions require larger training sets and golden corpora
   that cover the entire entity set.
 * "Build it yourself" becomes very expensive, with a large data scientist and
   engineering overheads, while organizations find themselves re-solving the
   same challenges that we have already solved.
 * "Manage it yourself" has a surprisingly large data science and engineering
   overhead, to overcome production challenges that we have already solved.

Testing methodology
We trained our Inline Entity Extraction SaaS to understand around 180,000
entities from a custom domain.  We then benchmarked it in a controlled
environment where two teams of annotators were asked to annotate 500 documents:-

 * The first team used the old-fashioned method of manually annotating a
   document.
 * The second team moderated the output of the model that had already scanned
   the documents and picked up entities, only needing to confirm or reject
   pre-determined results.

Results: High F1 Score
Here’s how our Data Language Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] Inline
Entity Extraction SaaS performed:

 * Precision: 0.94
 * Recall: 0.90
 * F1: 0.92

We have proven we can train a highly accurate model, specialized for a custom
domain, that can cope with hundreds of thousands of entities with very
high-performing metrics, using a golden corpus that covers only a small subset
of the entities. 

Achieving such high results on the classification of such a high number of
classes is very rare for a model like this in the wild.

Results: Huge Workflow Efficiency Gains
The findings of this exercise were that the error rate of the team that used the
Data Language AI model to pre-annotate the documents was substantially lower
than the team that worked without the model. 

 * The team using "Data Language AI" demonstrated a 57% drop in error rate.
 * The team using "Data Language AI" demonstrated a 53% increase in annotation
   coverage.

The measurable benefits were clearly efficiency and accuracy and added to the
coverage of entities being substantially larger, it’s clear that using our model
was a great success.

Should we keep Humans in the loop, or not?
We were also analyzing whether there is benefit in keeping humans in the loop
and this result proved surprising. While it was clear that helping humans out in
entity extractions tasks by pre-running the content through a model could
significantly assist their operation, we actually found that the error rate of
the model was actually slightly higher when humans were involved than when the
results were totally automated. In this case, humans reviewing results actually
makes things slightly worse!

There are some cases where perfect results are not mission-critical - e.g.
upstream automated tagging to simplify downstream workflow, where "the machines
do the heavy lifting". In these cases a certain amount of error is acceptable
and a totally automated process will work. 

In other cases, where errors are less acceptable, the confidence and evidence
metadata in our SaaS can be used to allocate "less certain" predictions to SMEs
and expert analysts for decisions and feedback loop optimizations.

Comparison to other Text Analytics NER Approaches
So what does our competition offer to compete with our results in the Named
Entity Recognition (NER) space? 

Well, it’s difficult to compare apples to apples in this scenario, as our
competitors suffer from the same problem outlined above: Complex custom domains
with small training corpora.

This is the exact advantage of our SaaS that these benchmarks emphasize. 

To put these results into the context of standardized testing of NER performance
elsewhere in the wider world and industry, one usually finds that most tests of
F1 approach the value of 1, only where these models work by identifying a small
handful of classes. Have a look - you will likely see "People, Places,
Organizations, Events"... the usual suspects, and this problem itself has been
solved daily for the last 12 years by almost every computer science student.

The Benefits of Data Language Text AI Services
The fact that our model can cope with a far greater number of classes than most
other providers, and still see such impressive results, is a strong
differentiator.

Another advantage of our Data Language AI Services
[https://datalanguage.com/products/datalanguageai] that we offer is that they
work out of the box at scale for your production platforms.

Our Inline Entity Extraction SaaS comes with all these benefits immediately: 

 * Ability to classify text content to 100s of 1,000s of entities in a custom
   domain.
 * A rich API for interacting with the machine learning service metadata.
 * Automatic management of machine learning model ‘fallout’.
 * Self-optimisation of the machine learning models, on the fly.
 * Capability to cope with an extremely high number of entities.
 * The ability to scale up indefinitely.
 * Production-ready engineering, out of the box.

All the engineering is done, and the security & maintenance are on us!

AI Relationship Extraction for Knowledge Graph Platforms
Later in 2022, we will launch a new commercial Knowledge Graph Relationship
Extraction service as part of Text AI Services - early testing has impressive
benchmarks for high-value knowledge organizations. More to come soon.

For more information on Data Language AI Services, check out the links below.

Data Langauge AI [https://datalanguage.com/products/datalanguageai] | Text AI
Services [https://datalanguage.com/lp/text-ai-services]

New Benchmarks from our Named Entity Recognition (NER) Text AI SaaS Product

Data Language announces promising results from this Innovate UK -funded AI
project, building on their long-running technology partnership with Cochrane. 

Innovate UK Project
Title: “AI Classification of COVID-19 studies to ensure clinical guidance and
healthcare policy is informed by current meta-analysis of all available evidence
”

This Innovate UK grant-funded project ran from August 2020 to April 2021.

Data Language has worked with Cochrane since 2014, designing and delivering a
microservice-based linked data platform that encodes an authoritative knowledge
graph of healthcare clinical evidence. This has included work to refine, iterate
and scale the platform whilst creating a suite of applications to publish
evidence in healthcare as structured data. The platform enabled Cochrane to
rapidly deploy their Covid 19 Study Register in response to the coronavirus
pandemic in 3 weeks last March.

This Innovate UK -funded AI Project has delivered a new capability in Data
Language’s Text AI services to enable automated creation (“clinical coding”) of
PICO Graphs, using values from Cochrane vocabularies and standard medical
vocabularies assigned to Population, Intervention, Comparison and Outcome
structures for clinical analyses.

The specific outcomes from the initial phase of project work can be summarised
as:

 1. We created a working Machine Learning approach.
 2. We have a clear set of next steps.

Following the successful conclusion of the project work to develop the AI
components, the next steps are to focus on key optimizations to move it into
productionization and commercialization. Timeframes are yet TBC.

Results from our Text AI Services Covid-19 Project with Cochrane

We are proud to announce the new member of our Text AI Services
[https://datalanguage.ghost.io/products/datalanguageai/text-ai-services] family:
Inline Entity Extraction. This new AI NLP service was created for high-value
knowledge organizations who want to accurately classify their text media at a
granular, inline-level.

The Problem: Automated Tagging in Custom Domains
We created this service to solve the problems that other Named Entity
Recognition (NER) NLP SaaS - and "build it yourself" approaches - could not
solve.

Our customers are high-value media and knowledge organizations. A huge portion
of their business asset value is locked in their unstructured media assets,
which they must leverage to add value to their subscription, DaaS, targeted
advertising, and other business models. This unstructured content must be
connected to their custom business domain via knowledge graphs and other
structured data systems:-

 * Complexity: To liberate this value, accurate inline text classification of 
   100s of 1,000s of entities is required to enable structured connection with
   bespoke information architecture models and/or knowledge graphs.
 * Custom Domain: Generic off-the-shelf NLP SaaS solutions do not recognize
   these custom entities, so a mapping exercise is required, and is often a too
   low resolution to provide value.
 * Rich Metadata: Other NLP solutions do not provide the rich, domain
   model-aware metadata, because they are not trained for the custom domain.
 * Small Training Sets: Other custom NLP solutions require larger training sets
   and golden corpora that cover the entire target entity set.
 * Cost of Build: "Build it yourself" becomes very expensive, with a large data
   scientist and engineering overheads, while organizations find themselves
   re-solving the same challenges that we have already solved.
 * Cost of Maintenance: "Manage it yourself" has a surprisingly large data
   science and engineering overhead, to overcome production challenges that we
   have already solved.

Our Solution: Text AI Services Inline Entity Extraction
Our Inline Entity Extraction SaaS handles a far greater number of classes than
other NER NLP solutions, while delivering impressive performance results. It
also works out of the box at scale for your production platforms.

Our Inline Entity Extraction SaaS comes with all these benefits immediately: 

 * Custom Domains at Scale: It classifies text content to 100s of 1,000s of
   entities in a custom domain.
 * Powerful API: It provides a rich API for interacting with the machine
   learning service metadata.
 * Does its own ML housekeeping: It automatically manages machine learning model
   ‘fallout’.
 * Self-Optimizing: It self-optimizes its machine learning models, on the fly.
 * Scalable: It is able to scale up indefinitely.
 * Production-Ready: It is Production-ready, out of the box.

All the engineering is done, and the security & maintenance are on us!

For more information on Data Language AI Services, check out the links below.

Data Langauge AI [https://datalanguage.com/products/datalanguageai] | Text AI
Services [https://datalanguage.com/lp/text-ai-services]

Product Launch: Inline Entity Extraction SaaS - AI Custom Domain Auto-Tagging

As part of our Text AI Services SaaS product range
[https://datalanguage.ghost.io/products/datalanguageai/text-ai-services], we are
proud to announce the launch of our cutting-edge Text Classification service.

The Problem: Automated Tagging in Custom Domains
We created this service to solve the problems that other NLP Text Classification
products could not solve.

Our customers are high-value media organizations. A huge portion of their
business asset value is locked in their unstructured text content:-

 * Aboutness: To liberate this value, an accurate classification of content
   "Aboutness" is required to classify content to the organization's bespoke
   information architecture models and/or knowledge graphs.
 * Custom Domain: Generic off-the-shelf NLP SaaS solutions do not recognize
   these custom aboutness classifications.
 * Rich Metadata: Other NLP solutions do not provide the rich metadata, because
   they are not trained for the custom domain.
 * Rapid Learning: Other NLP solutions can't learn on the fly, and require
   expensive and time-consuming training cycles to learn new concepts.
 * Cost of Build: "Build it yourself" becomes surprisingly expensive, with a
   large data scientist and engineering overheads, and organizations find
   themselves re-solving the same unforeseen challenges that we have already
   solved.
 * Cost of Maintenance: "Manage it yourself" has a surprisingly large data
   science and engineering overhead, to overcome production challenges that we
   have already solved.

Our Solution: Text Classification AI
Our Text Classification SaaS trains itself and learns your team's in-house style
while delivering impressive performance results. It also works out of the box at
scale for your production platforms.

Our Text Classification SaaS comes with all these benefits immediately: 

 * Customized to your Domain: It classifies text content to 100s of 1,000s of
   entities in a custom domain.
 * Powerful API: It provides a rich API for interacting with the machine
   learning service metadata.
 * Does its own ML housekeeping: It automatically manages the machine learning
   model ‘fallout’.
 * Self-Optimizing: It self-optimizes its machine learning models, on the fly.
 * Scalable: It is able to scale up indefinitely.
 * Follows your in-house style: It learns the tagging style of your SME team.
 * Production-Ready: It is Production-ready, out of the box.

All the engineering is done, and the security & maintenance are on us!

Data Language are specialists in AI Text Classification. This includes automated tagging, machine learning, language agnostic systems, self-optimisation, and specialist text classification areas.

Read about products and projects in AI Text Classification at Data Language. This includes automated tagging, machine learning, language agnostic and specialist text classification areas.

AI Text Classification - News, Technology and Product Updates

AI Text Classification

Data Language AI: Text Classification SaaS

Tagmatic is a service for automatically tagging your content.

It uses sophisticated machine learning techniques to learn how you tag your
content on-the-fly and then will tag new content automatically.

Tagmatic uses your own tag dictionaries. Whether you tag with simple text terms,
identifiers, taxonomies, or linked data, Tagmatic will tag with those. 

There is no data mapping exercise needed, just send your content & tags to
Tagmatic and it gets to work. Tagmatic quickly learns new tags too.

It is incredibly easy to integrate into your CMS or editorial workflows via two
simple web service interfaces.

If you are looking for an automated tagging solution, then look no further.

If you are developing your own automated tagging solution, then pause, give
Tagmatic a try - and be up and running quickly.

Update @ 2020: 
Tagmatic has been renamed as "Text AI Services: Text Classification
[https://datalanguage.com/products/datalanguageai/text-ai-services]" as part of
our "Data Language AI" product suite
[https://datalanguage.com/products/datalanguageai].

News, Blog Posts and Innovation updates about Tagmatic - our sophisticated AI automated content classification SaaS product. We have taken care of all the engineering and data science for you: Tagmatic learns on the fly, is self-optimising, and language-agnostic.

Tagmatic is an AI automated text classification SaaS product by Data Language. It learns on the fly, is self-optimising, and language-agnostic.

Tagmatic - News and Product Updates about our Text Classification AI

Tagmatic

Introducing Tagmatic

Text AI Services combine machine learning and AI to provide world-class text classiciation. Self-optimising, language agnostic and full managed.

Launching Text AI Services | Efficient Text Classification

We have just launched our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services], a fully
managed, SaaS-based set of text services using AI and Machine Learning to
automate key processes. As essential tools in any digital transformation
journey, our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] are the next
generation in NLP and text analytics.

With decades of experience in understanding and solving pain points in
structured data, we have now developed a world-class service for ai text
classification. Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] are perfect
for out of the box machine learning to make your text tlassification and
recommendations more logical, accurate and efficient.

Find out more about Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] | Book an
intro call with the Text AI Team
[https://calendly.com/datalanguage/data-language-ai-intro-call]

Launching Text AI Services

(Update @ 2020: "Tagmatic" has been rebranded as the Text Classification service
in our "Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services]" product
suite).

It may be argued that customer satisfaction is the only and ultimate KPI that
really matters when judging the success of a product. In which case Tagmatic
would pass with flying colors, given the fantastic feedback we are receiving
from clients. 

Remarkably enough, we found that sheer inferencing performances are often at the
edges of our clients' considerations: this is partly due to the intrinsically
tailored nature of Tagmatic's models, but mostly attributable to a plethora of
invaluable qualities that go beyond the naked F1 scores: being able to quickly
spin up a model on your own content and vocabulary, and then interact with it
via a minimalistic HTTP interface, zero maintenance, on-the-fly learning of new
classes, low latency, security, scalability and high availability out of the
box, are crucial factors to enterprise-scale deliveries. 

Your Intellectual Property lies in your content, your data, your data models,
and how you use the classifications. It is not in commoditized compute
solutions.
Your resources are thus more effectively spent developing and exploiting your
market differentiating IP, rather than spending way too much time and money on
trying to replicate the engineering complexities of off-the-shelf MLaaS
solutions which may not be your technical team's core competence.

This is where Tagmatic comes in. The hard work and complex engineering has been
done, just bring your content!

The datasets
RCV1 is arguably the most popular dataset for multi-label classification,
comprising 800,000 newswires by Reuters tagged with 103 classes. Oddly enough
for such a totemic dataset, the breadth of the vocabulary is anachronistic when
compared to some of our clients, modern global publishers who routinely maintain
taxonomies with thousands of topics.
The dataset is available here
[http://www.ai.mit.edu/projects/jmlr/papers/volume5/lewis04a/lyrl2004_rcv1v2_README.htm] 
although unfortunately not in its original XML format with full HTML markup,
which we would normally leverage as part of the classification task. Also the
corpus comes already tokenised and train/test datasets pre-canned. Besides
taking away part of the fun, this has also negative implications on the
vectorisation logic Tagmatic implements, making it largely ineffective
unfortunately. For the purpose of this benchmark, we have used the full train
set (23,000) and a reduced test set (100,000).

AAPD (Arxiv Academic Paper Dataset) is a collection of research papers covering
a number of subjects organised in several classification systems. The dataset 
here [https://www.kaggle.com/neelshah18/arxivdataset] is annotated with a number
of classification schemes, of varying level of quality. We have carved out a
subset corpus in the Computer Science domain, comprising of 32,000 papers and 39
subjects in total.

The benchmark
We have considered a number of models amongst the most widely employed for
multi-label classification tasks. The reference evaluation metric is
weighted-F1.

RCV1 AAPD One-vs-Rest - LinearSVC 0.78 0.71 One-vs-Rest - SGD 0.74 0.62 
One-vs-Rest - XGBoost 0.75 0.64 Classifier Chain - LinearSVC 0.79 0.71 Label
Powerset - MultinomialNB 0.55 0.55 Label Powerset - RandomForest 0.60 0.55 
Tagmatic 0.80 0.74 
Tagmatic offers out-of-the-box improvement over a number of basic
implementations. Granted, a team of Data Scientists might possibly come up with
better optimised flavours of such models, pushing performances up. But at what
cost? And by how much? And what then? There is still the much bigger task of
bringing that model to fruition in a real world environment, requiring
non-trivial development effort. Is that really worth it? 
Unless text classification is either your USP or a strong market differentiator,
then the answer ought to be a resounding no. 

Tagmatic is production-ready, enterprise-grade, and available to you right now -
see our Text AI Services
[https://datalanguage.com/products/datalanguageai/text-ai-services] product.

Lastly - Using Tagmatic (Text AI Services) on Hansard
We have also benchmarked Tagmatic against Hansard data: a corpus of ~200,000
parliamentary interventions for the 2005/6 period , classified with ~7200
categories, scoring a whopping 0.83. More on this on the next blog post. 
Make sure you sign up for our newsletter [ https://datalanguage.com/contact-us] 
and follow @datalanguage [https://twitter.com/datalanguage] on Twitter for all
the latest.

Text AI Services

Our Text AI Services work

Text AI Services Blog Posts

News about Text AI Services