With any digital transformation project, we develop reusable capabilities to create economies of scale. By prioritising certain elements of the data platform, we’ll demonstrate to your business the progress we’re making. We call this system a data platform.

What is a data platform?

A data platform is a core set of technologies that will collect data from across the business and carry out analytics to provide insight and predictions never realised before. 

It brings together data from both internal and external systems, such as clickstreams, usage logs, customer service journeys, transactions, payments and user behaviour. Because of this, the data platform means data scientists are able to train predictive algorithms more efficiently. This transforms analysis and reporting to being data driven. 

It also provides the infrastructure to run data analytics in the cloud and is able to spin-up and spin-down as demand changes.

Filling in the gaps

A data platform allows us to plan and build the capabilities at the heart of the transformation and means we can identify:

• the skills, processes, decisions, governance and output data pipelines that are used in common across many of the desired use-cases. 
• functionality and processing techniques that will lower costs across the whole portfolio and speed up the development of future use-cases
• the people involved in the project 
• technology choices that will make reusing capabilities easier when we develop data pipelines.

At the highest level the platform can be divided into three sections: data sourcing, data engineering and platforms, and data products.     

Within each of the three main sections there are subsections that focus on particular tasks. We can then look into each of these subsections in much greater detail to see a breakdown of the individual activities and people needed to achieve each task.

For example, one of the data engineering and platforms subsections is data acquisition. The activities that need to be carried out to complete this task are:

• planning the data feeds 
• scheduling the data feeds 
• managing the data suppliers 
• registering the dataset 
• configuring the upstream SLA 
• managing the data resupplies.

The people needed to carry out these tasks are:

• a data engineer
• a data analyst 
• a software engineer
• a data architect / modeller.

Stages of development 

Visualise everything

When we’re looking at which early capabilities we should develop, we want features that your business can use but that also contribute to the development of the full data platform. These features will showcase the work to the rest of the business so that users can begin to see the benefits of the project. We do that by developing the capabilities that provide visualisation first. 

Showcasing early use-cases demonstrates the new capabilities as they’re developed, which increases understanding, acceptance and adoption of the new capabilities and the project as a whole. 

On their own, visualisation capabilities don’t harness the economies of scale that using a data platform provides. This is because they don’t develop core DataOps capabilities, but they do surface the data to users quickly. Developing these capabilities is still fairly costly, but because they feed into the overall platform, they will reduce the cost of future capabilities as the project continues. 

Observe everything

The next set of capabilities we develop are those that allow us to observe everything. This starts to make use of strategic components that ingest all your data to the platform. 

Observation capabilities uses DataOps principals and tools to automate data ingestion and data quality. At this stage we start to create an economy of scale to feasibly ingest all your data assets. 

Developing these capabilities begins to drive down the costs of delivering new use-cases, and this is where we start to see previously marginal use-cases looking viable.

Automate everything

Finally, we can develop the remaining capabilities that will automate everything. Now we really start to make use of data by making predictions and moving towards automated end-to-end processes. 

By enabling both the components and the data in the data platform, more and more of the use-cases become cheaper to develop and you’ll start to really feel that your business has been digitally transformed.

Want to find out more?

If you’d like to find out more about how we can help your business in their digital transformation by building a data platform, please get in touch with us today.