Impact case study database

Newcastle University’s research has contributed to the unique extensible design and widespread adoption of the PROV standard for data provenance. The model was endorsed by the World Wide Web Consortium (W3C) in 2013, and since then it has become the de facto standard for capturing and exchanging provenance. Data-intensive organisations have also adopted PROV for internal use to add value to their datasets.

PROV has gained extensive reach geographically (UK, EU, USA, Australia), across disciplines (Geoscience, Climate studies, Medicine, public information services) and sectors (Government, Business, Science). Links to web-published information by beneficiaries of PROV are collated in [E1, E5]. The investment required to incorporate PROV into existing data stores, creating extensions, and changing working practices is indicative of its value to beneficiaries.

The use cases below illustrate the level of impact of PROV on three high profile organisations: NASA/ USGCRP (US Global Change Research Program), global Pharma company Astra Zeneca, and the National Archives in the UK. An introduction to how they use PROV and why, is provided in [E1]. Below we highlight key points. In the words of their programme managers, the benefit has been in making their information more authoritative and trustworthy in the eyes of their users, and to engender positive changes of internal working practices concerning data governance.

1. NASA / USGCRP

NASA JPL manage the US Global Change Information System (GCIS) https://data.globalchange.gov/, on which the National Climate Assessment (NCA) reports in the USA are based. These publicly available reports, commissioned by the USGCRP (Global Change Research Program), inform and influence policy debate on climate change and the environment, within the USA and internationally. Impacts include:

Change in working practice & policy. The use of provenance in the GCIS was recommended by the Federal Advisory Committee on climate assessment and mandated by the US Administration (President Obama at the time) [E3 Appendix 3]. PROV, along with the GCIS ontology and other metadata vocabularies, is used systematically in the GCIS to enforce the traceability of all of the about 50,000 individual resources held in the database [E3 section 5]. According to Dr. Sherman of USGCRP [E6], all contributors to the GCIS are now required to curate their contributed content, including any climate science data. PROV metadata must now be supplied alongside any resource contributed.

Since 2012 NASA first, and now USGCRP, have demonstrated long-term commitment to PROV and to data curation more broadly, by providing sustained funding for 3 FTE staff [E6].

Effect on policy debate provided by transparency and assurance of the data held by the GCIS. PROV and the GCIS ontology have effectively promoted a culture of data curation and data trust within the climate science community. This percolates down to the reports themselves, which are public, and where PROV elements are exposed both for human reading and in machine-processable formats (XML, RDF), providing trust into climate science as part of public discourse. According to Curt Tilmes, Data Scientist at NASA detailed to GCRP at the time, this was a significant benefit of PROV and enabled them to have “ incontrovertible consensus” to support climate debate [E1 page 13].

NASA has extended the PROV model [E2] for use in the Planetary Data System (PDS4) which contains scientific data from the solar system planetary missions. The data, which includes historical datasets from the Voyager and Cassini missions is now continually updated and verified through the PROV extension. This enables valuable analysis: for example, the provenance schema for the Voyager ISS geometric calibrated images allows tracing information to be used by exoplanet scientists for analysis [E1 pg. 13].

PROV has benefitted satellite construction. Satellites are made up of a large number of constituent parts procured via a long chain of intermediary suppliers. Counterfeit parts pose a particular problem for NASA. Dr Tilmes states that PROV data provides the audit trail that allows identification of the source of counterfeit or faulty parts [E1 pg. 7].

1. UK National Archives and the Gazette

The National Archives in the UK (NA) maintains 11 million historical government and public records in addition to current documents. Their data underpin the UK Government Gazette and legislation database. To assure data after digitisation, the NA have mandated the systematic inclusion of provenance [E7] as part of all the documents published by the Gazette. Recorded web traffic to the UK Government Web Archive suggest over 1.7 billion redirected hits to the website (NA website).

Change of working practice as a result of the requirement by the National Archives (NA) to include provenance. All of Gazette data must now be supported by provenance statements. PROV has made this possible and cost-effective, as all the hard design work has already been done. The Gazette has committed dedicated staff to maintain and support provenance curation (Dr. Cresswell interview [E7]).

Maintenance of authority & correctness of data after digitisation. The Gazette is the authoritative source for public documents in the UK, as notices published by them are afforded legal standing - traceability and trust in the information are therefore paramount. Originally only available as physical verified documents, the data has maintained integrity through the use of PROV during the digitisation effort. Data retrieved from the Gazette must be traceable: It is “ the official public record, it has credibility, it has that kind of grand strength […] now there is a provenance trail for every single notice […] it will tell you what happened to the notice since it came to us for publishing, and every step that happens within that notice journey.” Janine Eves, Business and Operations Director, The Gazette [E4].

Traceability of legislation data. Legislation.co.uk, underpinned by the National Archives, is the official web-accessible database of the statute law of the United Kingdom. PROV provides traceability to legislation data. This has been especially useful to maintain the trace of legislation originating from EU law to support exit arrangements (Dr. Cresswell interview [E7]).

1. AstraZeneca

AstraZeneca is a global pharmaceutical company with a portfolio of speciality care and primary care medicines. According to Dr Tom Plasterer, Director of Bioinformatics, Data Science & AI [E8], a provenance model was needed to support internal processes and PROV provided a community-accepted solution for that. The process of adopting PROV along with other ontologies started in 2013 as part of a million-dollar project, where PROV is estimated to account for about 5-10%, with continued maintenance to date. This effort resulted in a change of working practices, where the use of shared vocabularies now informs data governance and promotes transparency: “ its vital importance covers … processes from drug discovery, target identification, target validation, through to trial design, evaluation, clinical trials, and how data is managed” [E1]. PROV has also enabled a new internal business intelligence system called CI360 to be developed, which bring competitive advantage to the company. It alerts scientists of possibly actionable news about competitors’ products. The technology is based on the popular concept of “nanopublications” (http://nanopub.org/\), where scientific statements are systematically annotated with provenance assertions. PROV ensures that the statements are properly corroborated and thus safely actionable.

Selected further cases of PROV in use.

Other notable cases of PROV adoption, are documented in [E5]. Amongst these:

• The Allotrope Foundation, an international consortium of pharmaceutical, biopharmaceutical, and other scientific research-intensive industries, develops and adopts specifications to standardize the acquisition, exchange, storage and access of analytical data captured in laboratory workflows. PROV is part of this suite.

• PROV has been adopted by Health Level Seven International (HL7) Fast Healthcare Interoperability Resources (FHIR), part of NHS Digital UK. Beneficiaries are clinicians, researchers, and regulators who are better able to trace, reproduce, and analyse scientific data.

• CSIRO, Australia’s largest government research organisation, has extended PROV for earth sciences data.

• The Netherlands Government‘s database of national registration of land and buildings incorporate PROV extension, “BAG”. Ministries, water boards, police forces and security regions, are obliged to use the data from the registrations.

The prevalence and pervasiveness of consumer electronics from smart phones to the Internet of Things has created a need for more efficient power management on devices where battery life and fast charging are important considerations. Before the impact period, the vast majority of Power Management Integrated Circuits (PMICs) used in consumer electronics employed synchronous (clocked) designs which are limited by their power conversion abilities – a bottleneck for the new generation of devices. Asynchronous designs, though more efficient, were not typically used in industry due to the difficulties in design and verification, and a lack of practically applicable tools [E1].

Newcastle University research implemented in the Workcraft framework and industrial collaboration by researchers have been seen by industry as the [redacted], in particular PMIC. Specifically, novel techniques have been applied to the industrial production process resulting in PMICs capable of higher efficiency of power conversion. Electronic devices using new PMICs benefit from longer battery life, faster charging and smaller power management components which allows for greater flexibility in physical design. For manufacturers of electronic chips and devices, this translates to cost reductions and market advantage.

Workcraft has also provided industry with tools that allow more rigorous verification. PMICs make millions of control decisions per second and so the responsiveness and robustness of PMICs are crucial, as a single misstep can short-circuit the battery and permanently damage the device.

Dialog Semiconductor PLC

Researchers have achieved impact through sustained industrial collaborations with multinational world leading electronic chip manufacturers, Dialog Semiconductor PLC and Analog Devices Inc. The impact of Workcraft is evidenced by the growth of these manufacturers, and by the industry advantage gained by their customer organisations. The Workcraft team have been recognised by industry as [redacted]. The benefits of enhanced power conversion also extend to manufacturers of mobile phones, smartwatches and other electronic devices containing asynchronous PMICs (in the form of market and business advantage), and to users of such devices (longer battery life and faster charging).

Dialog is the exclusive supplier of PMICs for Apple Corporation’s iPhone, iPad and Watch, which represented 75-80% of Dialog’s sales [E4]. Collaborations with Dialog began in 2014 through joint R&D programmes and training of over 80 engineers based in the UK, USA and Europe in the use of Workcraft. The software has been [redacted]. Dialog holds two patents based on these novel technologies [E5]. In 2016 Dialog reported that power conversion, one of their three main business segments, delivered a year-on-year growth of 54% [E6]. By then, Dialog’s power conversion controllers held close to 70% of the fast-charging smartphone and computing adapter market. The importance of this technology to the industry is evident in the 2018 purchase of Dialog’s power management business and 3-years supply of PMICs by Apple Corporation in a $600,000,000 deal [E3] – the largest deal of its kind by Apple.

Analog Devices Inc.

Another key pathway to impact on the industry is through USA-based world leading chip manufacturer Analog Devices Inc. who see asynchronous design important for the success their portfolio of over 45,000 products [E1]. In 2018, the company invested into training their engineers from the USA and UK in using Workcraft. Due to this, they are able to conduct formal verification of correctness of analogue and mixed-signals electronics to identify all potentially problematic configurations and states – a task was “virtually impossible to achieve” previously [E1]. An important element of the impact is in engineering education which will continue to have impacts for the future electronics industry. Analog Devices acknowledge that formalised training for asynchronous logic design is “extremely rare worldwide” [E1], however it is essential for new technologies to be accessible to industry.

GitHub, Inc.

Research on algebraic graphs [P4] incorporated in Workcraft was also made available as an independent open-source software library algebraic-graphs [P5] implemented by the researchers. GitHub deploys it in their semantic library ( https://github.com/github/semantic) for parsing, analysing, and comparing source code across multiple programming languages. GitHub have stated [E7] that they use algebraic-graphs to [redacted]. GitHub Inc., since 2018 a subsidiary of Microsoft, is the world’s largest source-code hosting platform. It reports over 67,000,000 users and over 213,000,000 repositories and is the 74^th most visited website according to Alexa ranking.

Widening accessibility

Workcraft has over the impact period been adopted into teaching at a number of universities: MSc System on Chip at Southampton University; 02204 Design of Asynchronous Circuits at the Technical University of Denmark; and CSC3324 Understanding Concurrency, EEE8043 Design of Asynchronous Low Power Systems, EEE8087 Real-Time Embedded Systems, and EEE8124 Low Power VLSI Design at Newcastle University.

Workcraft is an open-source project that is available publicly at workcraft.org. Since 2014 there have been 32 public releases of Workcraft, which were downloaded approximately 20,000 times from approximately 4,700 unique IPs. The open-source library algebraic-graphs is also publicly available [P5] and was downloaded approximately 9,600 times since 2016.

The Arjuna transaction system software, ATSS (also known by its new name Narayana [E1]) plays a central role in enhancing Red Hat middleware products with transactional services [E1, E2].

The Activity Service [P1], itself an Object Management Group (OMG) industry standard (known as the Additional Structuring Mechanisms for the Object Transaction Service) formed the basis of a number of ACID and non-ACID extended transaction standards for Web services produced by OASIS, a global standards consortium [P2]. The structure of the AtomicAction module of ATSS/Narayana made it relatively straightforward to incorporate the features required by the Activity Service so that both ACID as well as non-ACID extended transactions (LRAs) can be supported with equal ease [P2]. Thus, the ATSS/Narayana software was instrumental in the design of standards conformant Web-service transaction protocols that are within Red Hat’s Java middleware platform product JBoss EAP (Enterprise Application Platform).

Since its inception, ATSS/Narayana has continued to be a central part of Red Hat's Middleware offering, providing the transactions engine for the hugely successful Java middleware platform product EAP. ATSS/Narayana has increased the scale of its impact, as EAP sales have continued to grow at around [ redacted] and increased the breadth of its impact by integration into a number of additional Red Hat middleware products, including Quarkus, Vert.x, Fuse, AMQ, BPM Suite, Thorntail and DataGrid [E1, E2].

In July 2019, IBM closed its landmark acquisition of Red Hat for $34bn and Red Hat product sales are now growing faster, by leveraging the deep IBM customer base. Red Hat revenue increased 18% in the first quarter of 2020, sales increased by 50% over the previous year and Red Hat signed the largest deal in its history [E3]. Whilst Red Hat has never reported on individual products, it is internally recognised that the Red Hat Enterprise Linux product attracts around 50% of global revenue with Middleware sales accounting [redacted] Red Hat reported full fiscal year total revenue of $3.4bn in 2019 [E4] and as every EAP sale is a Narayana/ATSS sale, the additional scale of impact is visible. Integrated in EAP, ATSS/Narayana now supports global mission critical business applications attracting sales of [ redacted].

The additional breadth of ATSS/Narayana impact can be seen through greater market share and a larger developer community for the Red Hat products in which it is integrated. Market share for EAP continues to soar. In 2017, Tomcat was the clear leader with 63.8% of the Application Server market with EAP/WildFly in second place at 13.8% [E5]. Red Hat leads the Tomcat project and ATSS/Narayana is also added to the Tomcat distribution. This has created a distribution channel for ATSS [redacted] The channel has widened further for ATSS/Narayana recently with Microsoft partnering with Red Hat to enable users to run JBoss EAP on the Azure App Service [E6].

The impact period 2013-2020 has seen enterprises moving their IT services to the cloud. Cloud platforms are increasingly building applications from loosely coupled modular components, termed microservices. This move to building applications from loosely coupled microservices has accelerated the interest in non-ACID extended transactions (LRAs) as the composition mechanism. The Microprofile project of the Eclipse Foundation (that hosts a global community of active open source projects) is one of the most prominent efforts aimed at optimizing Enterprise Java for the microservices architecture. The Eclipse MicroProfile LRA is based on WS-LRA extended transaction model developed by the OASIS Web Services Composite Application Framework Technical Committee [E7]. WS-LRA is part of a number of ACID and non-ACID extended transaction standards for Web services produced by OASIS that - as stated earlier- based on the Activity Service (see also [E8]).

Red Hat’s MicroProfile LRA implementation is based on ATSS/Narayana. Red Hat's new Java platform Quarkus now incorporates MicroProfile. In less than a year since it was released, adoption of Red Hat's Quarkus has skyrocketed with 16% of enterprise java developers now using the framework [E1]. In September 2019, Red Hat recorded 55,000 new users and 250,000 page views for Quarkus in one month [E1]. In addition, customers of IBM's Cloud Pak for Applications product now have access to the full portfolio of Red Hat's Runtimes and associated components. EAP and Quarkus are part of Red Hat Runtimes, which since 2019 is shipped in IBMs Cloud Pak for Applications and pushed to new and existing customers as the suite of software they should use when either migrating from WebSphere or thinking about building new Cloud-based applications. In this way it is clear that EAP, Quarkus and therefore ATTS/Narayana are becoming the default for IBM customers building new applications [E9].

The impact presented above is on the growing global market for Application Infrastructure and Middleware (AIM) Software, estimated at US$30.6bn in 2020 and projected to reach US$45.7bn by 2027 [E10]. The research also continues to have an impact through economic benefit to the UK. Red Hat has continued to invest in its European Middleware HQ based in Newcastle, moving to Newcastle University's Catalyst building in 2019. [ redacted]