Kickstarting your ModelOps journey
Artificial Intelligence (AI), machine and deep learning continue to make great strides in the enterprise environment. Indeed, 2019 will be the year that we really start to see AI make an impact on business process.
But when we talk about AI in the context of business processes, we don’t just mean the cutting-edge innovative sort, we’re talking about the ‘boring’ stuff too.
The importance of ‘boring’ AI in your business should not be underestimated because it is often the driving force behind the day-to-day decisions your business makes.
Your ability to understand how your business is evolving is directly proportional to how your AI model is performing. With this mind, it’s vital to ensure that this ‘boring’ AI is kept fit and effective. There are several reasons for this.
AI is still nascent within the enterprise environment, so we need to start maturing the way that we manage it – particularly for those industries which directly impact the consumer and are heavily regulated, such as financial and professional services, and the life sciences and pharma sectors.
Although you may have spent a lot of time creating a great machine learning model, having chosen different model architectures and features to create something highly performant, this is just the start of the journey.
AI models that haven’t been curated become dilapidated and don’t perform. This can result in below par business outcomes, as well as a bad reputation for your business function within your wider organisation
To ensure peak performance, you need to constantly feed and curate your data, capturing the features that are being fed into it – and this is where ModelOps comes in.
Introducing the ModelOps team
In the machine learning world, we’re only as good as our last prediction. So, the concept of retraining, swapping architectures, looking at new algorithms, approaches and data sets is important.
If we learn from what has gone before around the evolution of the DevOps team, then our view is that you need a specialist ModelOps team now, more than ever.
Typically made up of data scientists, software engineers, data management, governance and data quality experts, this team is expert in their various fields and truly understands what is happening under the hood of your organisation – from data inputs and data bias to ensuring consistent model ‘explainability’.
It is the ModelOps team which not only keeps an eye on your model performance, looking at outputs and understanding areas of drift, but also retraining your AI model. They are responsible for implementing remediation plans to return the model(s) to peak performance, whilst constantly engineering new features.
The feature engineering function is starting to take a front seat in the ModelOps as teams begin to implement and roll out feature stores which enable the critical operations of feature generation, feature governance (lineage and monitoring), as well as feature injection as part of model serving.
Where does ModelOps sit within your organisation?
Is tech function within your IT department or a business function in your average business department?
This is something you can trial over time – see what works for your individual circumstances.
However, your ModelOps team should work closely with DevOps as they leverage the core components that DevOps has created.
ModelOps does the day job around decision governance and fidelity of both the model results as well as the features that are generated and served, whilst DevOps handles the operational aspects of setting up APIs, cloud infrastructure and scaling the whole environment.
Your ModelOps team will rely on the ability of DevOps to empower them. In this way, it’s a true partnership which will ensure effective AI model deployment, maintenance and operation.
Getting internal buy-in
Whilst data scientists understand the need for constant AI model maintenance and retraining, the average stakeholder within the business probably doesn’t.
Your stakeholders have needs from an application and they understand that there is an operational cost, but the cost of keeping a team, retraining, feature engineering, and understanding the performance of a model, is probably very new to them.
You need to take them on this journey too if you are to get the financial backing you need to fund your ModelOps approach.
Investment in the feature store is critical if your business is to reap the benefits of AI. It’s the absolute cornerstone of your approach to ModelOps and will be the deciding factor between whether your AI is able to scale from a handful of models, to many thousands of them, all executing decisions in real time.
Increasing demands around model explainability provided by ModelOps teams will help in this regard,
Ensuring decision fidelity
We often hear talk about data quality and how it needs to be governed and we expect to see more of this in relation to AI and model explainability over coming years.
With increasing regulatory pressure, particularly on the back of the recent implementation of GDPR legislation and the rise in AI bias,[1] there are calls for more model transparency to ensure the rationale behind business decisions can be justified.
With this in mind, we expect to see ModelOps teams start to talk more about how decisions are governed and how they are able to verify decisions using more than one model.
This is about decision quality. AI models shouldn’t be executed in isolation but in parallel, whilst instituting a governance process over the top, and it should involve all stakeholders within your organisation.
If your stakeholders can’t interpret your model’s results or aren’t able to explain to a customer how your model arrives at a decision, then the ModelOps team needs to come up with a better way of visualising this.
To do this, it’s important to create a decision vault which will capture inputs, outputs and business decisions.
Creating a decision vault
Bank vaults allow us to store money in a safe, secure environment, allowing us to revert back to it to count it at a later date, or take it out when we want to.
The same can be said of the AI decision vault – an immutable store of all your business’ AI-based decisions.
It brings together the whole decision-making process in a secure, tamper proof vault – from the model inputs and the model itself, to the final decision, and will allow your business to justify historic decisions to regulators.
The fact that you have one and that it is continually evolved by your ModelOps teams, will demonstrate to the regulator that you are looking at model and feature performance, and that you are committed to engineering biases out of your decision process as far as possible, to make decisions in a considered way.
Kickstarting your ModelOps journey
AI is evolving quickly and businesses need to ensure that their teams evolve to deliver against business and regulatory needs, whilst empowering their people too.
It’s vital to ensure that your AI is kept fit and effective. The first step in doing so is understanding the need to consistently curate your AI models. To do this, you need an effective ModelOps team.
At 6point6 we have in-depth experience of helping clients to achieve this by providing expert insight to kick-start your ModelOps journey as well as integrating your feature store into your AI strategy.
Our proven methods for understanding your AI requirements, coupled with feature store implementation, machine and deep learning expertise, will help you to develop out your AI and deep learning capabilities in line with the needs of your business, as well as the demands of your regulators.
Get in touch if you’d like to chat to us.
Written by Gary Richardson, Chief Digital Officer
