Tolga Ergin

A great AI or ML model is worthless if it can't be delivered in a scalable, cost
effective and technically sustainable way. Our data science consulting practise
excels in the design and implementation of machine and deep learning models for
the analysis of data and media at scale.

Data Science seems to have something mystical about it, to the extent that
enterprises have almost come to consider it as a magical solution to impossible
problems. The reality is that behind DS there are tons of unfashionable data
engineering and analysis tasks, before statistical models can be applied to the
problem. 

Our team of talented experts here at Data Language can leverage years' worth of
technical excellence to ensure your AI solution can actually be scaled and
delivered at the highest standards of software engineering.

Understand your data 
Unlock the full potential of your data through early discovery, analysis, and
visualisation.

Learn fast
With so many variables to tweak, rapid feedback and short iteration cycles are
imperative to an effective delivery.

Models are software
As with any piece of code, statistical models also benefit from engineering best
practices for delivering at scale, with controlled total cost of ownership.

--

Read our white paper on rethinking how you do data science in your organisation:

 
[https://datalanguage.com/static/whitepapers/rethinking-data-science-in-the-enterprise.pdf]

Data Science at Scale

Paul Wilton

Scalable enterprise data science is 10% AI and 90% engineering.

Our Tagmatic text classification AI platform scales to >100k parallel predictive
micro-models, and returns an ensemble prediction result in under 1 second.

100k

We benchmarked our Tagmatic service against Hansard data: a corpus of ~200,000
parliamentary interventions for the 2005/6 period , classified with ~7200
categories, F1 scoring a whopping 0.83

0.83 Fabio Colasanti

Earlier this week, this tweet
[https://mobile.twitter.com/random_walker/status/1196870349574623232] appeared
on my feed.

> Much of what’s being sold as "AI" today is snake oil. It does not and cannot
work. In a talk at MIT yesterday, I described why this happening, how we can
recognize flawed AI claims, and push back. Here are my annotated slides: 
https://t.co/iCpyFw5urN pic.twitter.com/pmOTI3vq8p [https://t.co/pmOTI3vq8p]

— Arvind Narayanan (@random_walker) November 19, 2019
[https://twitter.com/random_walker/status/1196870349574623232?ref_src=twsrc%5Etfw]

Prof Narayanan's deck
[https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf] has
reached a relatively sizeable audience and received widespread positive
commentary. After all, anyone who has had any involvement with AI in the last
few years can 100% relate with the catchy title of his deck.
Hey - we're just about starting to see a feeble light at the end of the
blockchain tunnel, right?

But, but...
Something doesn't quite sit well with me. So I'm going to stick my neck out and
respectfully reject, if not the conclusions, the way Prof Narayanan has chosen
to make his point. Or rather, the way he has chosen to make his real point.

I find this deck deceptively coy, as the actual message is not "How to recognize
AI snake oil" but rather "Why I don't want AI anywhere near socially sensitive
topics", which would have been a perfectly valid argument, and made for a
fantastic set of slides, but required a slightly different set of arguments and,
most importantly, to be explicit about it.

At the end of the day ML/AI is about identifying patterns in data and draw
inferences, so of course it can "predict the future". Google Maps does it all
the times when it gives you an ETA for your journeys. In Prof Narayanan's own
examples, bail decisions and border control are not performed by machines, but
based on the prediction of machines. It's not about predicting the future: it's
called decision support, performed at speed that is beyond human capabilities.
Is the way banks and insurance companies make decision on mortgages and premiums
snake oil too? 
Prof Narayanan seems exceedingly preoccupied about "massive transfer of power
from [...] workers to unaccountable tech companies" and points at "enormous
commercial interests" that "we must resist" and "push back". He is of course
completely entitled to give the whole argument a political spin, but I'm not
sure this helps his arguments.

The real issue here is that some problems, and especially socially impactful
ones, carry high personal risks if an inference is wrong (problem 1). Therefore,
in order to minimise that, large amounts of potentially sensitive data may be
required (problem 2) and in most, if not all, cases there is a strong argument
for transparency, accountability and human-in-the-loop that makes such
propositions much more expensive and challenging from both a technical (problem
3) and governance (problem 4) perspective. 
Bias poses no doubt massive challenges (problem 5), not just in its
identification and removal, but also in its very definition, as what turns
common sense into bias is a function of our set of social and cultural values,
which greatly varies in time and space. Should history be changed because it
doesn't fit our Western 21st century view of the world? I don't have an answer,
but at least I'm asking the question.
Also, there is the topic of regulation which can't be ignored in these cases:
who establishes the boundaries? Who is it meant to enforce them? Do we need a
universal legislation, or would locally applicable frameworks suffice? Who makes
a call on what is an acceptable risk vs potentially beneficial outcomes -
supranational bodies, governments, individuals, etc.. ? (problem 6)

So the better discussion to have is around the risks involved in deploying AI in
socially sensitive fields, what is the minimum level of benefits it must bring
about to make it worth us facing the challenges above, and how do we go about
having that conversation.

Just branding the whole thing as snake oil is sidestepping the issue and largely
unfair, because it's bypassing the actual conversations to be had and going
straight for the outcome that Prof Narayanan - in arguably a biased opinion -
has decided is what's best for us all.

AI is a core discipline at Data Language. We have an expert team skilled in machine learning and deep learning, and importantly we also have the engineering and deployment skills to deliver AI at scale for your production systems.

AI is a core discipline at Data Language: Our expert team is skilled in machine learning and the engineering skills needed to deliver AI at scale.

Artificial Intelligence delivered at scale - our core discipline.

Artificial Intelligence

We need to talk about AI

Julian Everett

In this post we are going to look at some of the ways in which AI and data
science is impacting the practical delivery of technology solutions today. To be
clear from the outset, this will not include a discussion of future-looking
topics that have been getting coverage recently such as fully automated code
generation or algorithmic product development, etc. Rather, we will be looking
the impacts which are already happening right now, the consequences of which are
often just as significant albeit in different ways.

Strategic Alignment Model
To get a clearer idea of what is going on, it will be helpful to start by
reviewing the dramatic shift that has occurred recently within the discipline of
data engineering. In order to do that, we are going to use a version of Niel
Nickolaisen’s Strategic Alignment Model
[http://www.accelinnova.com/strategy.html], which is one of our most frequently
utilised tools in the inception phases of data programs.

It splits an organisation’s capabilities in two core dimensions:

1.     Whether or not they are market differentiating

2.     Whether or not they are mission critical core competencies

This then informs technology delivery strategy as follows:

·      Systems which deliver business capabilities which are mission-critical
market-differentiators you are going to need to build - as by definition there
will be no commoditised solution available yet otherwise it wouldn’t be a
differentiator - and you are going to need to build them very well whilst
effectively managing risk and uncertainty, as this is where your future revenue
streams are coming from (as a side note, this quadrant represents almost all of
our consulting work).

·      Systems which deliver business capabilities which are mission-critical
but not market-differentiators should be bought in. That’s because they are
generally commoditised solutions to common problems (ERP, payroll, invoicing,
etc), hence re-inventing those solutions is a pointless waste of money. In some
cases, organisations might have non-standard workflows which don’t match
commodity solutions, but in the long run it is almost always better to change
those workflows rather than build bespoke software.

·      Systems which deliver business capabilities which are not
mission-critical but are market-differentiating are generally good candidates
for partnerships, i.e. with businesses for whom the newly developed capabilities
will become core competencies.

·      Systems which deliver business capabilities which are neither
mission-critical nor market-differentiating should generally be avoided, as
their value is as best highly questionable.

Such analysis frequently provides very useful insight to organisations. However,
a critical point to remember is that markets are dynamic, so such
categorisations are never fixed. Indeed, it is interesting to see how
technologies traverse this diagram over time. The most common transition is from
market-differentiator to non-market differentiator as technologies mature: new
features and capabilities that were once enough to attract new customers and
generate additional revenue become widely adopted, turning into commodity
features that are required by any product attempting to enter the market or
protect its existing revenue streams. There are also examples, such as online
video within media organisations, which start out as non-mission critical
differentiators before becoming mission-critical differentiators and then
non-differentiating core competencies (as a side note, the other important shift
as functionality matures from differentiating to non-differentiating is the
associated switch in customer priorities from functional to non-functional
requirements – what matters for non-differentiating systems is
standards-compliance, portability and the avoidance of vendor lock-in, etc).

Data Engineering As Market Differentiator
So what does this have to do with data engineering? Well, the really interesting
thing about the recent history of data engineering is that is represents the
much less common example of a technology moving in the opposite direction –
upwards, from non-differentiating to differentiating:

Data engineering used to be the domain of the back office: of ETLs, PL/SQL
programming and cron jobs. It was one of the areas of software development most
insulated from the market pressures which shaped agile software delivery
techniques and lean product development. It was also one of the least impacted
as a result: rare indeed was it to find an ETL developer writing their code
test-first.

However, the revolutions in firstly data analytics and statistical software
development, and then in machine learning techniques, have radically transformed
the discipline and moved it front and centre in the new era of data-driven
product innovation. This has created a sudden critical demand firstly for access
to these new technologies, a need which has driven the recent adoption and
growth of data science as a commercial capability; and secondly for the software
development skills that allow innovative product ideas to be tested and iterated
rapidly under conditions of uncertainty without incurring crippling amounts of
technical debt.

One of the most prevalent organisational responses to these demands is for
data-driven product delivery programs to employ three discrete technology
functions:

1.     Data science teams: typically, recent PhDs with good theoretical
knowledge but little experience of commercial software development. They often
work in what is effectively an R&D function developing prototype implementations
in their preferred language.

2.     Software delivery teams: where these function best, they are most
typically staffed by experienced/senior developers well versed in extreme
programming and continuous delivery techniques and ideally who have worked
together previously (the importance of this last point is so often overlooked –
no matter how good the individual members are, if the team itself is new then
you are going to burn significant time and cash whilst relationships, trust and
delivery flow are negotiated and established
[https://en.wikipedia.org/wiki/Tuckman%27s_stages_of_group_development]).

3.     Machine learning engineering teams: who are responsible for taking the
outputs of data science and adapting them into something which can be integrated
by software delivery. I use the term “adapting” rather than “refactoring” here
intentionally, as due to the “specialised” nature of the data science algorithms
the aim is normally to add runtime tooling into a software execution wrapper
that allows the prototype implementation to be invoked concurrently or to handle
larger datasets. Most typically this will be suites of map-reduce and derivative
tooling (not least because these teams are also the most natural home for data
engineers who invested in Hadoop stack upskilling during the first data
analytics revolution).

Implications for Software Architecture
This differentiation is having profound, and most highly undesirable, results.
My colleague Fabio Colasanti has previously written persuasively
[https://datalanguage.com/features/data-science-is-about-building-software] 
about the consequences at the code level. In the rest of this article, we will
explore the implications for software architecture, and the resulting impacts on
the business value actually being realised by these programs of work.

Arbitrary Service Boundaries
Conway’s Law [http://wiki.c2.com/?ConwaysLaw] states that the architecture of
systems tends to reflect the architecture of the organisations that build them.
The most reliable way to mitigate this tendency is to create a single
cross-functional delivery team that includes domain experts and works as close
to customers as possible. That’s because the domain model and business
capabilities encoded into software create the channels along which information
flows and business value is then generated and delivered.

In the absence of such an integrated, cross-functional approach, the likely
outcome will instead be an architecture where service boundaries reflect the
arbitrary divisions of labour in the construction of the codebase. Is that
really so bad? Well, yes actually - it is. Once the code sequences required to
implement and integrate the business capabilities are defined, it becomes
immediately apparent that the natural execution flow is being critically
disrupted by meaningless service interfaces that require mapping layers,
additional infrastructure complexity, kill performance, undermine cacheability,
expose security vulnerabilities, and introduce a host of other avoidable
problems.

In doing so, they also undermine the chances of a successful delivery.
Successful technology delivery is ultimately a question of risk management: of
identifying risks and then managing or mitigating them so that their cumulative
impact is as low as possible. That means ruthlessly removing all avoidable
sources of risk and noise - so that the inherent risks of the market (changing
customer demand, competitor behaviour, etc) can then be observed, responded to,
and their associated opportunities exploited.

Self-generating Complexity
In response to the implementation barriers created by arbitrary service
boundaries, delivery teams then typically attempt to work around them via the
lowest friction route available. Rather than conducting a politically
unpalatable root cause analysis and proposing a simplified approach that
conflicts with the structure of the overall delivery program, it is easier to
try using additional software and infrastructure complexity to address the
problem. This results in an escalating feedback loop of bad design: not only is
the architecture unnecessarily complex already, but additional complexity is
then introduced to manage that complexity.

Unfortunately, the only way to break out of this loop is for some brave soul to
put their hand up and conduct a “survival of the fittest” design cull, where
every component of the architecture is ripped out and tested for critical system
viability until what remains is the minimal skeleton actually required to
deliver the baseline functionality.

Machine Learning as Universal Panacea
A more subtle variant of self-generating complexity relates to the application
of machine learning itself. Given the relative immaturity and vast potential of
the field of data science, there is now the additional lurking temptation to try
using ML to work around any problem rather than analysing and solving its root
cause. However, as Robert Frost said “the best way out is always through” – a
phrase which captures the essence of how to avoid incurring crippling technical
debt. Applied to data-driven product innovation, that means conducting patient
and precise analysis to identify underlying causes of problems and solving them
at source rather than building an ever-expanding shanty town of workarounds that
is liable to catastrophic and terminal failure.

Effective Mitigation Strategies
What techniques can be employed to mitigate against these problems and create a
reliable foundation on which to base your delivery? In our experience the two
most valuable tools are a domain model and sets of BDD scenarios (or terse use
cases) which describe the state and behaviour on which the anticipated business
value depends:

Domain Model: this is a collaboratively defined conceptual model of the problem
space shared between the subject matter experts and technical staff. Its key
importance is that it explicitly names the entities of the business domain and
their relationships to each other. In doing so, it creates a worldview and
language that is shared by all stakeholders/members of the delivery team (a
purposefully limited and relativistic scope, which is enough to ensure use cases
can be fulfilled whilst avoiding the many pitfalls and dead-ends of enterprise
data modelling). It is not a knowledge graph, ontology or database schema.
Rather it is something which SMEs might express as a frequently iterated
whiteboard diagram of labelled entities and connectors, and developers might
describe using an ERD or high-level class diagram. This foundation is what
allows code to be written in a way that precisely and unambiguously matches the
collective intent of the team. Without it, you are building on quicksand.

BDD Scenarios: once a shared language has been established which describe the
entities in the domain and how they relate to each other, the state transitions
involved in the information flows and generated business value can then be
captured (in practice this tends to be highly iterative, with new scenarios
being sought that break the model and thereby drive its improvement). The
emphasis here is not on automated testing capability – although that can be a
very helpful by-product - but the communication of state transitions and
workflows [https://lizkeogh.com/2013/10/24/bdd-before-the-tools/]. 

Once the workflows and information flows have been defined via which business
value will be delivered, questions concerning the optimal deployment of human
and machine agents can then be correctly and meaningfully addressed. At this
point, it is then really important to return to the Strategic Alignment Model
for one more pass. Newly equipped with a powerful model that clearly describes
the state/data and behaviour of your domain, you can review each entity and
function in turn to create an unambiguous and precise definition of what your
market differentiators really are. Is it really the bespoke algorithm that your
data scientists are working on, or is it in fact the data that the algorithm is
outputting? If the latter case, is it really justifiable to keep investing in
developing something in-house which ought to be (and will almost certainly end
up eventually) a commodity purchase? These considerations will then be primary
factors informing product roadmap planning – a subject beyond the scope of this
post.

The result should be a system architecture that is the simplest thing possible
[https://www.jamesshore.com/Agile-Book/simple_design.html] to deliver the
identified business value, and as a result is the cheapest to build, contains
the minimum number of things that can go wrong, and is fastest to adapt to
future market change.

Current impacts of AI on Technology Delivery

For Data Science to truly live up to expectations, it must embrace software craftsmanship.

Data Science is about building software

Earlier this month I was chatting with my old friend Pierluigi Casale
[https://www.linkedin.com/in/pierluigi-casale-41271430/] (Data Scientist at
TomTom, AI lecturer and researcher) about what seems to be the question of the
century: "What is Data Science"? Tongue in cheek, I offered my very own answer:
"Data Science is that thing you get when someone with a PhD plays with a Jupyter
Notebook, and a technical architect refuses to let it anywhere near production
because it doesn't have a single unit test". 
Although sporting a PhD himself, Pierluigi agreed that my slightly caustic
definition was indeed something he observed himself, and that he too was wary
of. He mentioned how he often concludes the many talks he gives at conferences
around the world with a diagram from the notorious paper "Hidden Technical Debt
in Machine Learning Systems
[https://papers.nips.cc/paper/5656-hidden-technical-debt-in-machine-learning-systems.pdf]
" by D. Scully et al.


There is indeed a widespread assumption that Data Science and Machine Learning
belong into a hazy limbo somewhere between data analysis, analytics, and R&D.
Within this mystical space, a set of practices otherwise regarded as inefficient
seem to be accepted as the norm: manual mining and manipulation of data; one-off
transformations virtually impossible to replicate; casual scripting of code with
total disregard of any development standard; artefacts laying around and code
running here or there...
When Scully talks about "glue code" and "plumbing", he is pointing at the right
problems but possibly underplaying the issue by being too humble about it. So
forgive my bluntness and let me put that straight: Data Science is about
building software.

Whether the context is academia or enterprise, research or commercial alike, the
adoption of Software Development best practices, tools and methodologies is the
key for efficient delivery of outcomes. At Data Language we have worked with a
number of clients at different levels of "enlightenment", and witnessed first
hand how injecting software craftsmanship into Data Science can lead to
profoundly transformational effects. We can also testify how the development of
our Tagmatic [https://datalanguage.com/tagmatic] service has been indeed 10%
Machine Learning and 90% software engineering: not much we could have done with
our shiny model without things like containerisation, high availability, high
throughput, load balancing, scaling, automation, security, governance,
monitoring, and many others. 

The adoption of established best practices in Software Development will boost
productivity, shorten the feedback cycles, and produce more robust code that is
easier to change, maintain, and ship.

Six things we at Data Language do and you should too
1. Ditch the darn notebooks
I'm fully aware this is highly controversial bordering sacrilegious for many,
but the truth is these type of editors foster bad habits and are holding you
back. They are like comfy, safe, cute trikes you got used to riding when you
were three; but if you actually want to go somewhere other than your garden,
you’ve got to move on and get yourself a bike. 
At this year's JupyterCon in NY, the talk widely regarded as best of the lot was
"I don't like notebooks.
[https://conferences.oreilly.com/jupyter/jup-ny/public/schedule/detail/68282]"
by Joel Grus. I wholeheartedly invite you to check it out.

2. Version control
A common refrain in favour of notebooks, is that they enable collaboration.
Oddly enough, when further engaged on this topic and asked about which version
control tool they use, what's their branching strategy, how often they merge,
etc... it all unravels. 
Collaboration is mighty important, so make sure you get on top of your version
control strategy. (And ditch the darn notebooks)

3. Test automation and CI
Arguments for Test Driven Development and Continuous Integration are widely
available and universally recognised as the single crucial factor that enables
faster, safer, better code. The very same arguments apply to Data Science. The
ability to move faster with your code enables quicker feedback and shorter
iterations: you will be able to experiment and prove ideas quicker, boosting
productivity. You will also end up with code that is designed better, and it can
be transitioned into a production environment more easily.

4. Infrastructure as code
In order to take full advantage of the ongoing democratisation of computing
power, it is crucial to write code that is portable and can be deployed wherever
is faster to run it. In a typical development setting, we want be able to
provision computing resources in the cloud, deploy our code there, execute it,
collect outputs and dispose of the remote resources. And we want to be able to
do this in a fully automated fashion, at the push of a button.
Many ways to skin this cat, but if I were to single out one thing you should do,
and learn to do well, is containerisation. Plenty of great Docker learning
resources around: get stuck in!

5. Data management
As in any other scientific setting, it is mighty important for experiments to be
repeatable under the same conditions. We want to make sure immutability of
inputs is preserved through transformations. Versioning and artefacts management
tools play an invaluable role in the development of Data Science solutions. This
goes hand in hand with practices discussed in points 2) and 3), and falls
naturally from there.

6. Metadata management
Often the journey is just as valuable as the destination. As much as possible
(and to the extent that is useful, of course!) we want to be able to retain key
information about the process at hand, so that we can better interpret, analyse
and enhance its outputs. The spectrum here is broad, from basic logging to
systematic capturing and analysis of metadata. The point is: we want to be able
to fulfil this function in the way that maximises the insights offered by the
data being collected. 
Metadata management is at the core of many activities that are either routine
(eg compare performance of models), or required for production (eg backout
strategy for a failing data pipeline), or further reaching into the realms of
ethics and accountability (eg quantitative justification of a Machine Learning
prediction). It is important we do it well.

Conclusions
I fully appreciate some of the recommendations above may feel daunting to the
more traditional Data Scientists, due to such activities not (yet!) being
typically associated with their profession. To some extent, we witnessed
something similar in the past with network and infrastructure specialists, now
evolved in DevOps that design and build infrastructure solutions through code,
fully embedded in the fabric of development teams from the onset, sharing tools
and methodologies. 
Similarly, the key for enabling the evolution of Data Science from R&D practice
to fully fledged enterprise function, lays in bringing data scientists and
developers closer: before long the distinction will blur, and specialist
knowledge will thrive on a common substrate of working practices. 

At the end of the day, remember, Data Science is about building software.

We are specialists in the design, implementation and deployment of Scalable Data Science solutions. We deliver practical data science implementations for both our own SaaS products, and for our clients' bespoke platforms and products, for the analysis of data and media at scale.

We are specialists in the design, implementation and deployment of Scalable Data Science solutions. Find updates, News and insights on Data Science here.

Data Science - our Work, Features, News and Updates

Data Science

Data Language helped Euromoney distill a broad and complex enterprise
engineering problem into a deliverable technical and data strategy. I would be
happy to recommend them to anyone.

Data Science at Scale

Understand your data

Learn fast

Models are software

Related Blog Posts