NHS Digital

NHS Digital, Richard Sutcliff, Senior Project Manager, richard.sutcliffe2@nhs.net

Start: 04/01/2018, End: 31/03/2018

Challenge:

NHS Digital process hundreds of separate data collections (with over 2000 permutation based on local data flows) that are submitted and stored in a variety of formats. Without a central source of truth data is processed differently in different teams leading to data quality problems, inconsistencies and a large overhead maintaining different data pipelines and reports. The overall objective of the POC was to demonstrate how a centralised knowledgebase for master data, reference data, data specifications and rules could support efficiencies in terms of data quality, derivations and analysis. 

How we helped:
To address these problems, we deployed the enterprise version of our open source metadata registry, the Metadata Exchange, which acts as a master data repository for standards and specifications. It provides a central source of truth where data definitions, rules and metrics could be created and queried by participating sites and analysts within the organisation. 

We imported the complete NHS data dictionary including business definitions, data types, attributes and machine processible business rules. We provided training for key team members and worked with the existing team to migrate from the existing excel spreadsheet repository, as well as an import of ~2,000 DLP provided local flow schemas, with development of a JSON schema ingestion facility. In addition to the key capabilities of the metadata exchange, we worked with NHS digital to integrate and prove that master data could be leveraged through our REST API as part of a reusable data pipeline to validate, store and report on data using Regex, Drools (DRL) and DMN.

Transferable Lessons:

NHS Digital have a repository of master data, however, it is not shared and processed in a consistent human and machine-readable format. By storing the data in a central repository, it could be leveraged to improve data quality and increase the efficiency of data validation and reporting. Since working on the NHS Digital POC we have partnered with UK Health Dimensions and loaded their quality assured master data management repository into the Metadata Exchange. As NHSE and NHSI both have licenses for this quality assured reference data and our tooling would allow more users to make the most of this resource.