Supporting the drive to data-driven innovation

In their drive to become a data-driven organisation, they wanted a centralised data platform that would enable anyone in the business to access relevant data to inform decisions. Our client has a portfolio that includes over 8 million journal-specific websites, and each journal has thousands and thousands of articles.

This global business has a progressive outlook to succeeding in a competitive, changing industry. Their process generates vast quantities of data that are used across the organisation to determine the direction of the business.

Our goal was to enable anyone in the organisation to access and integrate relevant datasets without having to perform those same tasks every time.

Six months to the first use case

6point6 took on the complex task of designing and building a data platform to stream datasets from across the business, so they can be accessed in the cloud for multiple use cases across the business.

We were part of a wider team responsible for delivering the platform and helping our client determine what they wanted to achieve from the engagement. We also assisted in identifying initial use cases to prove immediate business value as the platform was being built.

Working with their team, we designed and built a bespoke DataOps framework that automates the deployment and run of complex production-grade data pipelines in a way that fits into their existing best practices. They now have world-class operational data management facilities that extend their best practices in engineering to also deliver data management. Use cases were executed as the service was being built, to showcase the capability to the wider enterprise.

The work involved synchronising key submission system data from over 3,500 journal-specific websites to the cloud, and integrating them into the data platform. To reduce errors that arise from manual deployment in configuring different components, we took an automation-first approach to creating, testing, deploying and monitoring those data pipelines on the platform.

The design and build of our DataOps components were guided by a clearly articulated target Data Operating Model. We proposed and delivered this operating model along with the platform architecture, to emphasise that true data-driven transformation needed to come from a blend of expert people and processes, as well as technology components. 

The capabilities we delivered included:

• A data quality toolkit that can work across the business that works independently of the schema of the dataset.
• A data operations solution that integrates with on-premise and SaaS services. 
• Improved data maturity across the organisation through coaching and mentoring employees.
• A data registry that manages data pipelines, cloud resources and data lineage.

Capabilities to support additional use cases at a lower cost

Understanding the capabilities required from the platform enabled us to bring together the complex systems and datasets across their portfolio and build industrialised components that could be used to deliver those capabilities.

Their engineering approach is highly agile, and together we implemented the concept of minimum viable product (MVP) – building ‘just enough’, learning quickly, and continuously iterating – collaborating with multifunctional teams to focus on use cases that would add value to the business as the platform was built.

This approach influenced our view of how to build the system: nothing superfluous was added, and every feature was implemented in response to a business need or technical requirement.

We picked their top four use cases to showcase results as we built the platform. We worked with teams across the organisation to understand their requirements, gather relevant data and ensure the quality of that data, and show how the platform could be valuable to them.

This paved the way for additional use cases that had been identified, and supported their transformation to a data-driven business.

A factory for innovation

Our new data infrastructure will support a ‘factory’ of use cases well into the future for our client – and at a much lower cost, thanks to economies of scale.

In effect, their investment will enable ongoing innovation as the platform will support new data pipelines and more data in the backend. Teams within the organisation can easily access the platform to identify and implement new use cases themselves.

They’ll also be able to generate new insights, for example, joining data relating to submissions with marketing information to measure the impact of marketing activity in a way they weren’t able to before.

With a cloud data platform and data architecture enabling a consistent way to access and manage information, our client is well positioned to become a data-driven organisation.

Results that matter

Our new data infrastructure will support a ‘factory’ of use cases well into the future for our client:

• Standardised components create economies of scale and lower the cost of creating new use cases. 
• The platform will support new data pipelines and more data, enabling ongoing innovation without repeatedly investing in duplicated work.
• Integration of data from over 3,500 journal-specific websites generates new insights into customer preferences and the success of marketing efforts.