Impact case study database
Search and filter
Filter by
- King's College London
- 11 - Computer Science and Informatics
- Submitting institution
- King's College London
- Unit of assessment
- 11 - Computer Science and Informatics
- Summary impact type
- Technological
- Is this case study continued from a case study submitted in 2014?
- No
1. Summary of the impact
Automated plan construction and execution, developed by King’s researchers, has enabled Schlumberger, a Fortune 500 Oil & Gas (O&G) Services Company, to deploy automated drilling capabilities, a technology that no other company is yet capable of providing. This has brought the company improvements in efficiency, consistency and safety, along with a competitive edge. Deployment started in 2016 and was by early 2020 in use on more than 28 rigs in the Permian Basin and elsewhere, including offshore, representing a value estimated in excess of USD150,000,000 per annum (through improved efficiency and consistency).
King’s research in automated planning and plan-execution has enabled Schlumberger to fulfil a longstanding ambition in the O&G industry to increase drilling automation, applicable to geothermal and exploration wells. King’s work on the AI planner has become the core of Schlumberger’s automation products and services - in the short term the company has plans to rollout the approach across the organisation, and ultimately aims to make this the industry standard for drilling automation.
2. Underpinning research
A planner is a system that takes a declarative domain model description, describing what actions are available, and a problem description, capturing an initial state and a goal condition. From these it generates a proposed course of action to transform the initial state into a state satisfying the goal, using the actions supplied. The planner itself is domain agnostic – it exploits the underlying nature of causal relations between actions to build plans with whatever domain it is supplied. The advantage this offers is that it is comparatively easy to deploy the planner to produce plans for widely ranging and very different domains.
Fox and Long have, for many years, specialised in temporal planning – the branch of planning that focuses on plans with temporal structure, including continuous change, concurrency and multiple coordinated agents [R3, R5, R6]. In their work since joining King’s in 2011, they devoted considerable effort to the problems of harnessing the generation of plans for the automation of controlled systems, interpreting plans as control instructions and managing the mediation between sensed signals and expected state [R1, R2, R4]. This work was conducted in the EU-funded PANDORA project [F4] on underwater vehicles (for inspection and maintenance of sea-bed structures), in the EU-funded SQUIRREL project [F5] on small indoor robot platforms (for tidying rooms and interacting with children in various games) and in the EPSRC-funded project on Future Power Networks [F3] (controlling distribution-level networks to ensure voltage levels and consistent supply). Research funded in these projects, the EPSRC AIS programme [F1, F2] and also the ASUR programme [F6] have all been vital to the success of this work, providing multiple contexts in which to explore the problems of linking planning and execution and the process of responding to plan failure with replanning.
Further research conducted at King’s that has proven valuable is linking a planner to an external solver. The benefits of a generic planner rest in the idea that a user need only focus on representing the actions of their domain in order to make use of the planner. A price for this is that the domain representation language is not tailored to any particular domain and some specialist structures in a domain might be difficult to model. Fox and Long have found an effective way to connect an external solver to a planner, in which the complex dynamics are captured in the external (specialised) solver and accessed by the planner through a simplified query interface [R3]. This makes it possible for the planner to build plans that manipulate complex structures, while relying on the external solver to manage the details of the state changes that the plan anticipates in those structures.
Fox and Long proposed the use of temporal planning to provide strategic control for automated drilling systems, relying on both automated planning and also the algorithms for interpretation of plans as instructions for execution by physical hardware [e.g. R1, R2, R4]. Plans are produced using a planner based on the COLIN/POPF [R6] planners built by Fox, Long, Coles and Coles. The planner relies on models written in PDDL2.1, a temporal modelling language developed by Fox and Long in 2003. In projects at King’s, Fox and Long developed the means to link plans to execution, including an architecture for execution of plans using the Robotic Operating System (ROS), ROSPlan [R4]. This work was originally developed for use in underwater vehicles [R1, R2], but subsequently more broadly. The work has been proved influential in the robotics community, but also informed the development of a plan execution language and dispatcher for Schlumberger in 2013, leading to the patent applications [PAT1, PAT2]. This work rests on extracting the temporal plan structure developed in the planner, to express in an executable and flexible form. In particular, it facilitates the identification of errors during execution, to trigger an appropriate response – replanning or shutdown, depending on the situation. The important technical elements of this work were in identifying the semantics of the dispatch process for the plans expressed in this language, providing a precise formal foundation and specification for the implementation of plan dispatchers in multiple applications within Schlumberger.
This case study is closely related to that submitted by University of Strathclyde: the fundamental planning research was conducted by Fox and Long at Strathclyde, before joining King’s, where the key contributions on plan execution, planning with continuous processes and on linking external solvers were carried out.
3. References to the research
[R1] Cashmore, M, Fox, M, Long, D, Magazzeni, D & Ridder, B 2017, 'Opportunistic Planning in Autonomous Underwater Missions', IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, vol. 15, no. 2, 17685429, pp. 519 - 530. https://doi.org/10.1109/TASE.2016.2636662
[R2] Palomeras, N, Carrera, A, Hurtós, N, Karras, GC, Bechlioulis, CP, Cashmore, M, Magazzeni, D, Long, D, Fox, M, Kyriakopoulos, KJ, Kormushev, P, Salvi, J & Carreras, M 2015, 'Toward persistent autonomous intervention in a subsea panel', Autonomous Robots, pp. 1-28. https://doi.org/10.1007/s10514-015-9511-7
[R3] Piacentini, C, Alimisis, V, Fox, M & Long, D 2015, 'An extension of metric temporal planning with application to AC voltage control', ARTIFICIAL INTELLIGENCE, vol. 229, pp. 210-245. https://doi.org/10.1016/j.artint.2015.08.010
[R4] Cashmore, M, Fox, M, Long, D, Magazzeni, D, Ridder, B, Carrera, A, Palomeras, N, Hurtós, N & Carreras, M 2015, Rosplan: Planning in the robot operating system. in Proceedings International Conference on Automated Planning and Scheduling, ICAPS. vol. 2015-January, AAAI Press, pp. 333-341, 25th International Conference on Automated Planning and Scheduling, ICAPS 2015, Jerusalem, Israel, 7/06/2015. http://www.aaai.org/ocs/index.php/ICAPS/ICAPS15/paper/view/10619
[R5] Bajada, J, Fox, M & Long, D 2015, Temporal planning with semantic attachment of non-linear monotonic continuous behaviours. in IJCAI International Joint Conference on Artificial Intelligence. vol. 2015-January, International Joint Conferences on Artificial Intelligence, pp. 1523-1529, 24th International Joint Conference on Artificial Intelligence, IJCAI 2015, Buenos Aires, Argentina, 25/07/2015. http://ijcai.org/papers15/Papers/IJCAI15-218.pdf
[R6] Coles, A, Coles, A, Fox, M & Long, D 2012, 'COLIN: Planning with Continuous Linear Numeric Change', Journal Artificial Intelligence Research, vol. 44, pp. 1-96. https://doi.org/10.1613/jair.3608
Funding:
[F1] EP/J012211/1 Sustained Autonomy through Coupled Plan-based Control and World Modelling with Uncertainty (2012-2015, £237K) [EPSRC AIS Program]
[F2] EP/J012157/1 Automated Plan-Based Policy-Learning for Surveillance Problems (2012-2016, £371K) [EPSRC AIS Program]
[F3] EP/I031650/1 The Autonomic Power System (2011-2016, GBP160,142 to King’s) [EPSRC]
[F4] EUFP7: PANDORA (Grant ID: 288273)
[F5] EUFP7: SQUIRREL (Grant ID: 610532)
[F6] ASUR (Autonomous Systems Underpinning Research), DSTL-led consortium funding: Phase A project supported by BAE Systems.
Patents:
[PAT1] Creating and executing a well construction/operation plan (US Patent App. 15/541,381, 2018)
[PAT2] Method of Creating and Executing a Plan (US Patent App. 15/536,693, 2017)
4. Details of the impact
Background
Schlumberger is the world's leading provider of technology and digital solutions for reservoir characterization, drilling, production, and processing to the energy industry, with product sales and services in more than 120 countries and employing approximately 82,000 people in 2020.
Schlumberger’s global R&D activities related to drilling and cementing operations, which represent nearly a third of the overall company, generated a revenue of USD10,000,000,000 in 2019. The portfolio for this covers a wide range of technologies such as drill bits, directional drilling tools, formation evaluation while drilling tools, drilling fluids, pressure control equipment, and rig control systems. An integral part of this business is the services that Schlumberger provides to the wider industry where their technologies are deployed by highly trained field engineers and technicians. However, this model has become extremely challenging to sustain - the company considers that the key initiative for their future success and differentiation is to move their services up through levels of automation. [S1]
Involvement in the 2011-2016 industry-EPSRC Autonomous Intelligent Systems program [F1,F2] led to then Head of Schlumberger Research in Well Construction Automation meeting Professors Maria Fox and Derek Long, and discovering automated planning. Schlumberger’s Vice President for Technology Development (Well Construction) observes [S1]:
“Schlumberger Cambridge Research has always been at the forefront of innovation. Automation has been an area of research for us for many years. The real step change in our approach to automate highly dynamic and uncertain drilling process started when we engaged Derek and Maria first as consultants and eventually as full-time employees.”
Fox and Long joined Schlumberger, in 2016, to bring onboard their research knowledge and complete the deployment of plan-based automation in the DrillOps software, a commercial implementation of an automated drilling function.
Product development, testing, and initial rollout
The initial phase of this process involved product development, testing and initial deployments for field testing. The second phase, of larger scale commercial rollout, began in late 2019 and early 2020, but has been disrupted by COVID-19. Nevertheless, DrillOps has already been deployed on 28 rigs in North America, Saudi Arabia, Italy and also off-shore. As part of the second phase, all Schlumberger’s different businesses have started to deploy the relevant parts of DrillOps.
What this means so far “is that, [the] directional drilling service in the US is now deployed through a system called Connect BHA, a DrillOps product, while […] trajectory planning and optimization in the US is done using DrillPlan. They will be expanded to other countries in 2021 as [Schlumberger] builds the in-country cloud infrastructure. Additionally, [Schlumberger] has globally deployed [DrillOps] automation in 28 drilling rigs with plans to increase to over 100 rigs that operates in our integrated business”. [S1]
Initial benefits derived from Plan-Based Automation
Schlumberger’s Science and Technology Manager (Automation and Planning) explains [S2]:
*“The work has provided wide reaching business impact, […] through the DrillOps autonomous drilling service, which is not incremental but marks a fundamental shift in the way we approach automation both within our own operations and in solutions we are developing with major oil and gas operators (for example with Exxon Mobil as announced in Q2 2020). Automation, through plan execution enabled by the work of Fox and Long, **creates consistency of operations, which drives safety, reduces time-to-target, and the minimization of environmental footprint.*”
In their 2020 Q3 financial report [S4], Schlumberger reports:
“In Saudi Arabia, […] DrillOps* automation well delivery solutions surpassed 63,000 ft drilled, achieving a key milestone for our Integrated Well Construction LSTK operations. The on-bottom rate of penetration (ROP) with AutoROP was 17% higher than previous wells drilled by the same rigs' field average. Furthermore, DrillOps controlled the preconnection, reaming, and backreaming operations, significantly reducing nonproductive time, optimized well delivery time, and contributed to a 30% improvement in on-bottom ROP and shoe-to-shoe run in a recent section of a horizontal well.”
This highlights the incentives, both to Schlumberger and its clients, of the automated drilling function, in terms of improved efficiency, consistency and reliability. In Q1 2019, the typical well in the Permian took 27 days (an industry-wide average, not indicative of typical Schlumberger performance) and costs USD595/foot drilled [S6, figure 2]. A typical well is 6,000 to 7,000 feet and costs at least USD4,000,000 [S6]. With a 17% improvement in on-bottom ROP, as shown in the Saudi Arabia example above [S4] (although more recent figures in reference to Schlumberger's DrillOps solutions suggests about 20%-30% improvements in efficiency [S7]), each well would save at least USD500,000 (a low estimate).
Assuming that drilling a typical well takes 27 days, a rig would typically drill about 12 wells per year. As evidenced previously, the automated system has been in use on selected client rigs since 2016 and is now deployed on 28 commercial rigs. Based on these numbers it is possible to estimate that the automation system, built around the planner and plan-execution components, is currently worth at least USD150,000,000 per annum in savings (USD500,000/well savings x 28 rigs x 12 wells/rig/year) to Schlumberger.
A key part of the Schlumberger’s long-term strategy
As the industry recovers from the COVID-19 crisis, Schlumberger has positioned itself to move quickly with its new technology to offer efficient, consistent, and cost-effective drilling automation, in order to maximise benefits for its clients and shareholders. As part of the company’s long-term strategy and its commitment to the technology, Schlumberger’s Science and Technology Manager (Automation and Planning) observes [S2]:
“The approach to AI planning has subsequently been investigated in other aspects of Schlumberger’s services business. In particular the automation of measurement operations (known as Wireline) and well-site service delivery, we expect to embed the approach across the organisation as part of the digital transformation to further realise the benefits.”
Additionally, Schlumberger’s Vice President for Technology Development (Well Construction) states [S1]:
“[The] goal with this product is to make it available to the wider industry and become the industry standard for drilling automation. A good example of this is the recent agreement that [Schlumberger] signed with Honghua where every new rig that Honghua build will have the DrillOps system built in it.”
The importance of this work to Schlumberger is also highlighted in the 2020 Q2 financial report to shareholders [S3] in which it is observed:
“As announced last quarter, Schlumberger and ExxonMobil are jointly working on the deployment of digital drilling solutions around planning, execution, and continuous improvement through learning. As a next step, ExxonMobil and Schlumberger have finalized an enabling agreement for the deployment of DrillOps* on-target well delivery solution in ExxonMobil’s unconventional operations. The technology is expected to enable faster, lower-cost wells through drilling automation”.
The Q2 financial report goes on to say:
“Schlumberger and Honghua Electric Co., Ltd. entered into a memorandum of understanding (MOU) for the seamless integration of the DrillOps on-target well delivery solution with all new Honghua rigs. Under the MOU, Honghua will manufacture and sell rigs that have plug-and-play capability with the DrillOps solution, which integrates planning and operations while automating well construction tasks in order for the rig to operate at peak performance throughout the execution of the drilling plan.”
As the globally leading well services company (reporting revenues of over USD32,000,000,000 in 2019), Schlumberger has consistently been part of investment portfolios for a wide range of pension funds and other large institutional shareholders, so its public financial statements are widely analysed and thus a key indicator of the direction it is heading strategically.
The future of Drilling Automation
To conclude, it is worth quoting Schlumberger’s Digital Strategy Development Manager in her article about DrillOps and drilling automation [S3]:
“ Digital and automation technologies present the best opportunity for the drilling industry to innovate its way out of the current crisis. However, seizing the opportunity will require preparation, collaboration, participation and commitment to transform the way wells are drilled, to move away from working in isolation to working together to capitalize on each other’s strengths. Doing so will not only help the industry to survive this downturn but to thrive in the future. When it comes to automation, all players have a key role to play, including operators, service companies and equipment manufacturers; however, rig contractors, as the owners and operators of the rig, are at the center of it all.”
5. Sources to corroborate the impact
[S1] Testimonial from Vice President for Technology Development (Well Construction), Schlumberger Cambridge Research, Schlumberger
[S2] Testimonial from Science and Technology Manager (Automation and Planning) Schlumberger Cambridge Research, Schlumberger
[S3] Schlumberger Q2 2020 Press Release, July 2020
[S4] Schlumberger Q3 2020 Press Release, October 2020
[S6] Oil&Gas Journal: “Permian basin operators cut drilling time, lower expense”, July 2019
[S7] World Oil Magazine: “Beyond automation: Driving advances in autonomous drilling”, January 2021
- Submitting institution
- King's College London
- Unit of assessment
- 11 - Computer Science and Informatics
- Summary impact type
- Technological
- Is this case study continued from a case study submitted in 2014?
- No
1. Summary of the impact
Research led by Professor Jian Dai on the design of robot manipulators has led to the development of robotic therapy devices for the treatment of lower limb injuries. Two companies have been founded to develop cutting-edge healthcare technologies based on this research: Movendo Technology (Italy), a large biomedical company which provides rehabilitation devices and effective rehabilitation treatment, and AiTreat Pte Ltd. (Singapore), a cutting-edge robotic massage start-up.
An independent evaluation has found that the robotic devices developed by these companies provide treatment equal to or better than conventional physiotherapy, but at lower cost and more reliable consistency, in particular for diagnosing susceptibility to falls in older people, rehabilitation for patients with Parkinson’s disease after spinal injury and stroke, and rehabilitation after lower limb trauma.
In total, more than 11,000 patients have been treated with these robotic devices at over 100 clinics and hospitals in countries including Singapore, Italy, Germany, Czech Republic, Netherlands, Spain, US, Dubai, Canada, Greece, Saudi Arabia, Puerto Rico, Switzerland and Ukraine. Over 800,000 physiotherapy treatments have been delivered.
Movendo Technology has 34 employees and 1,500 agents worldwide, and its annual turnover was EUR2,764,000 in 2019. Its market share is approximately 2%. AiTreat has completed three rounds of VC/Pharma funding, and its 2019 turnover was USD460,000, 60% of which is attributed to Prof. Dai’s technology.
2. Underpinning research
Professor Jian Dai has been working on the mechanism design and orientation and dexterity of robot manipulators ever since he joined King’s College London in 1999. His research, conducted together with his PhD students, post-doctoral researchers and collaborators, has included both fundamental contributions on relevant modelling and computing methodologies and development of applications in various contexts. This has led to designing novel technologies for rehabilitation and massaging robots. The development and utilization of such robots by Movendo (rehabilitation robots) and AiTreat (massaging robots) are the subject of this impact case study (ICS).
Dai’s research underpinning this ICS began in the early 2000s with theoretical work on computational procedures for kinematics and control theory. Dai and Rees Jones [R1] presented a new approach for constructing the null space of a linear system of homogeneous equations using the cofactors of an augmented coefficient matrix. Their approach improved on the computational efficiency of the previous methods, by avoiding the Gauss–Seidel elimination algorithm, and on their numerical accuracy. [R1] showed how the new method can be applied in kinematics to calculate reciprocal screw systems, providing a theoretical framework for analysis and synthesis of robotic mechanisms.
Dai et al. [R2] used the method from [R1] in designing robotic platforms for ankle rehabilitation and analysing their stiffness. The proposed devices had fewer degrees of freedom than the competing designs, hence fewer actuators and simpler control leading to reduced costs. From the technological viewpoint, the devices integrated various stages of rehabilitation and could be tailored to requirements of physiotherapy. The results from [R2] provided technology for the development of robotic rehabilitation systems by Movendo.
Saglia et al. [R3] proposed control algorithms for redundantly actuated parallel mechanisms. The algorithms were based on inverse kinematics, relied on the accuracy of the proposed mathematical models and numerical procedures for Jacobian matrices, and were calibrated and validated in simulations and experiments. The control algorithms in [R3] together with the prototype presented in [R4] provided, via the patent US20110306473, the basis for the final design of the Movendo Hunova robotic platform.
The analysis of the relationship between force and motion initiated in [R1] was used in Zhang et al. [R5] in design and analysis of extensible continuum robots, and was further extended in Cui et al. [R6] for finger-rolling contact analysis. [R5] and [R6] laid foundations for the inverse kinematics formulated as a system of nonlinear algebraic equations in terms of the joint rates of the finger linkage mechanism and the parameters of the contact trajectories. This work provided the basis for designing the massaging device at AiTreat.
Li et al. [R7], starting again from the results presented in [R1], proposed a novel model-free method based on the adaptive Kalman filter to achieve path tracking for a continuum robot using only pressures and tip position. As the Kalman filter requires only a two-step algebraic calculation per one control interval, the low computational load and the real-time control capability were achieved. Simulations and experimental validation showed robustness of this control method against system uncertainties. This work was used by AiTreat in implementing control in their robotic devices.
The research underpinning this ICS as described above was supported by EPSRC and EU grants and attracted sponsorship from THK Co. Ltd. Japan for the investigation into robotic massage fingers. The current EPSRC grant EP/S019790/1, awarded jointly to King’s College London and Leeds University, extends further this research, aiming at development of new robotic healthcare devices. Jian Dai received the 2015 ASME Mechanisms and Robotics Award and the 2020 ASME Machine Design Award for his lifelong contribution to the fundamental theory, design and applications of mechanisms and robotics systems.
3. References to the research
[R1] Dai, JS & Rees Jones, J 2002, 'Null-space construction using cofactors from a screw-algebra context', Royal Society of London. Proceedings A. Mathematical, Physical and Engineering Sciences, vol. 458, no. 2024, pp. 1845 - 1866. https://doi.org/10.1098/rspa.2001.0949
[R2] Dai, JS, Zhao, T & Nester, C 2004, 'Sprained ankle physiotherapy based mechanism synthesis and stiffness analysis of a robotic rehabilitation device', Autonomous Robots, vol. 16, no. 2, pp. 207 - 218. https://doi.org/10.1023/B:AURO.0000016866.80026.d7
[R3] Saglia, JA, Tsagarakis, NG, Dai, JS & Caldwell, DG 2009, 'Inverse-kinematics-based control of a redundantly actuated platform for rehabilitation', PROCEEDINGS- INSTITUTION OF MECHANICAL ENGINEERS PART I JOURNAL OF SYSTEMS AND CONTROL ENGINEERING, vol. 223, no. I1, pp. 53 - 70. https://doi.org/10.1243/09596518JSCE622
[R4] Saglia, JA, Tsagarakis, NG, Dai, JS & Caldwell, DG 2013, 'Control Strategies for Patient-Assisted Training Using the Ankle Rehabilitation Robot (ARBOT)', IEEE ASME TRANSACTIONS ON MECHATRONICS, vol. PP, no. 99, pp. 1-10. https://doi.org/10.1109/TMECH.2012.2214228
[R5] Zhang, K, Qiu, C & Dai, JS 2016, 'An extensible continuum robot with integrated origami parallel modules', Journal of Mechanisms and Robotics, vol. 8, no. 3, 031010. https://doi.org/10.1115/1.4031808
[R6] Cui, L, Sun, J & Dai, JS 2017, 'In-hand forward and inverse kinematics with rolling contact', Robotica, pp. 1-19. https://doi.org/10.1017/S026357471700008X
[R7] Li, M, Kang, R, Branson, DT & Dai, JS 2018, 'Model-free Control for Continuum Robots Based on an Adaptive Kalman Filter', IEEE ASME TRANSACTIONS ON MECHATRONICS, vol. 23, no. 1. https://doi.org/10.1109/TMECH.2017.2775663
4. Details of the impact
Two companies have been founded to exploit Prof. Dai’s underpinning research: Movendo and AiTreat. In each company, one of the co-founders was a former research student of Prof. Dai. Each company has developed its own therapy robot. While much of the underlying research is similar, they use two quite different applications of robot technology with Movendo concentrating on a rehabilitation platform whereas the AiTreat robot delivers massage and acupuncture.
An independent evaluation has found that research at King’s led directly, and via the careers of the two former students and the technology of the two firms, to high quality products which benefits thousands of patients globally per annum. The larger part of the underlying technology is directly attributable to King’s research, although both firms also have added substantially to that core, for example in diagnostics and supporting clinical workflows, as well as extending clinical applications. However, it is unlikely that either firm would ever have been started or developed its technologies without the research inputs from King’s. [S1, p.19]
Movendo: technology development, founding, investment and company growth
Research conducted by Prof. Dai and his then PhD student Dr Jody Saglia [R3] (based on [R1, R2]) led to a key patent. The work attracted investment of over EUR10,000,000 from the Italian National Institute for Insurance against Accidents at Work (INAIL), that was helping to develop the ankle rehabilitation device “Arbot” and ran a successful RCT (Randomised Control Trial) on orthopaedic post-traumatic patients with lower limb injuries. The clinical trial and the patent led to the development of a new device for total body rehabilitation, Hunova, based on the core technology of the ankle rehabilitation device “Arbot” and the Patent. Hunova was been developed and clinically validated at the Italian Institute of Technology by a team including Dr Saglia, who raised EUR10,000,000 in corporate venture funding from Dompé Pharmaceuticals, one of the main Italian biopharmaceutical groups [S7] to start a new company, Movendo Technology, in late 2016. [S2]
Movendo Technology acquired an exclusive license for the Arbot and Hunova technology and now operates globally to commercialise its solutions in the rehabilitation market. Hunova is Movendo Technology’s principal product. It targets rehabilitation of the lower limbs, from ankle to knee and hip, the pelvis, and trunk, by providing a large variety of training modalities. [S6]
Movendo Technology Group employs 34 people (as of 31 July 2020), and annual turnover rose from EUR1,292,000 in 2017 to EUR2,764,000 in 2019. Its market share is approximately 2% [S11]. To date, it has sold 69 units. In addition to its core staff, Movendo employs around 1,500 agents worldwide. [S10]
In total, more than 10,000 patients have been treated at over 100 clinics and hospitals in countries including Italy, Germany, Czech Republic, Netherlands, Spain, US, Dubai, Canada, Greece, Saudi Arabia, Puerto Rico, Switzerland and Ukraine. Over 800,000 physiotherapy treatments have been delivered. [S11]
Dr Saglia, Movendo Technology, co-founder and former CTO [S2]:
“Prof. Dai’s contribution has been significant along the entire journey - from the very beginning of the concept idea to the final product design passing through clinical validation. This is a clear example of world leading research being successfully transferred to the industrial world, making a real impact on society. [Today] Movendo is continuously fostering a research and technology development collaboration with Prof. Dai’s group for its future products.”
AiTreat: Technology development, founding, investment and company growth
AiTreat Pte Ltd. (Singapore) has developed EMMA, a massage robot based on the technology proposed in [R5, R6, R7] by Prof. Dai and his PhD student, Chen Qiu. Following the receipt of an ACE Grant from SPRING Singapore (2015), the company has now completed three rounds of fundraising with the last round of investments coming from Brain Robotics Capital LP (a prestigious US VC), Tasly Pharmaceutical Group Co Ltd (Shanghai) and Ogawa Smart Healthcare Technology Group Co Ltd (Shenzhen). [S8]
“EMMA is AiTreat’s primary development and the first successful one of its kind in the world. EMMA is a state-of-the-art soft tissue therapeutic treatment system that utilizes robotic technology and medical knowledge that is stored using Artificial Intelligence. […] EMMA is equipped with sensors to measure muscle stiffness and uses 3D vision technology to analyze the patient’s body.” [AiTreat website, quoted in [S1, p.6]]
AiTreat Pte Ltd. has produced several generations of the massaging robot EMMA in four clinics in Singapore, with over 1,000 customers and over 5,000 times of use [S9]. The company turnover is USD460,000, 60% of which is due to using Prof. Dai’s technology. [S9]
AiTreat Pte Ltd was the winner of the Microsoft Developer Day Start-up Competition (2016), winner of 'Best Product/Application Design Award' at Shanghai International Start-up Competition (2016), and recipient of a StartupSG Tech Grant from Enterprise Singapore (2017). [S8]
Benefits of the robotic systems to medical practitioners and patients
The key benefits cited by practitioners who use the robotic systems are [S1, p.9]:
Treating more patients, with better health outcomes. “Combining several types of fingering including thumbing, index fingering and palm motion in one for massaging to provide versatile massage treatments using one end-effector, multiple types of massage, ... what this does is improve efficiency, enabling a large number of treatments.” [CTO of AiTreat, quoted in [S1, p.16]]
Cost savings for clinics and patients
Treatment equal to the best physiotherapist, but cheaper and easier to reproduce consistently. "Treatment from our robots is about equivalent to the best treatment from the best physiotherapist but the latter is unreliable, expensive and hard to reproduce. Some traditional treatments require two to three persons to implement." [Director, Sales and Marketing, Movendo, quoted in [S1, p.16]]
Consistent standard of treatment. "For example, [… with traditional treatment …] we give them exercises, [some] people don't do it, or we prescribe some random massage, it helps for a few days. But maybe it [results in] a misalignment of ligaments. We prescribe sessions on Emma for the back, and then acupuncture, with very good results." [CTO of AiTreat, quoted in [S1, p.16]]
The flexibility of the robotic systems. “Adaptability - we can change from simulating rubber bands to swimming in a pool with the flick of a switch” [Director, Sales and Marketing, Movendo, quoted in [S1, p.9]]
The clinical efficacy of the Movendo devices is documented in published research as at least equivalent and sometimes better than conventional treatment, in particular for:
Diagnosing susceptibility to falls in older people [S3]
Rehabilitation for patients with Parkinson’s disease, after spinal injury and stroke [S4]
Rehabilitation after lower limb trauma [S5]
Clinicians report that patients respond favourably to the robot treatment. “Patients are intrigued…They like the consistency and the [lack of ] worry about who [they] will get. We get a lot of word of mouth; they bring their friends.” [AiTreat clinician, quoted in [S1, p.15]]
The independent evaluation concludes:
“The two firms provide robotic technology, based substantially on research from KCL, which is part of a wave of clinical robotics which is likely to massively influence or even revolutionise the way clinical massage and acupuncture are delivered, and may extend beyond those core applications.” [S1, p.19]
5. Sources to corroborate the impact
[S1] Independent evaluation by Impact Science: “Robotic rehabilitation and massage”, November 2020.
[S2] Testimonial from Jody Saglia, former Head of Rehabilitation Technology Laboratory, Movendo Technology Group
[S3] Cella, A., de Luca, A., Squeri, V., Parodi, S., Vallone, F., Giorgeschi, A., … Pilotto, A. 2020 'Development and validation of a robotic multifactorial fall-risk predictive model: A one-year prospective study in community-dwelling older adults', PLoS ONE, 15(6), 1–22. https://doi.org/10.1371/journal.pone.0234904
[S4] G. Marchesi et al., 2019 'Robot-based assessment of sitting and standing balance: preliminary results in Parkinson’s disease', IEEE 16th International Conference on Rehabilitation Robotics (ICORR), Toronto, ON, Canada, 2019, pp. 570-576. https://doi.org/10.1109/ICORR.2019.8779387
[S5] Taglione, E., Catitti, P., D’Angelo, M. L., Squeri, V., Saglia, J., Sanfilippo, C., & De Michieli, L. 2018 'Proprioceptive and motor training using the high performance robotic device hunova: Protocol of a randomized, controlled trial in patients with lower limb post-traumatic conditions',. Annals of Physical and Rehabilitation Medicine, 61, e497–e498. https://doi.org/10.1016/j.rehab.2018.05.1158
[S6] Iandolo, R., Marini, F., Semprini, M., Laffranchi, M., Mugnosso, M., Cherif, A., De Michieli, L., Chiappalone, M. and Zenzeri, J. (2019) ‘Perspectives and Challenges in Robotic Neurorehabilitation’, Applied Sciences. MDPI AG, 9(15), p. 3183. https://doi.org/10.3390/app9153183
[S8] AiTreat website
[S9] Testimonial from Yizhong Zhang, CEO of AiTreat Pte Ltd
[S10] Movendo Technology website
[S11] Testimonial from Giuseppe Betti, CFO of Movendo Technology Group
- Submitting institution
- King's College London
- Unit of assessment
- 11 - Computer Science and Informatics
- Summary impact type
- Technological
- Is this case study continued from a case study submitted in 2014?
- No
1. Summary of the impact
Mobile connectivity is truly pervasive and underpins a wide range of commercial and societal applications. Telco has thus become a trillion-pound industry with 2G-4G mobile networks serving more than eight billion connections today. 5G systems were seven years in the making and are now being deployed with an expected market value of GBP500,000,000,000 by 2030.
A key factor in the success to acquire parts of this market is the ability to enter the 5G market early with innovative offerings, and to cut operational costs through standardised solutions. To this end, King’s fundamental research contributions on 5G telecommunication systems impacted industry and standards developments alike:
Ericsson (5G market leader): King’s impacted [text removed for publication].
Konica Minolta (enterprise printing market leader): King’s impacted [text removed for publication].
3GPP (global telecommunications standards body, 700 industry members): King’s provided standards-essential contributions to two standards on converged 5G architectures, which provide significant benefits in terms of customer experience and affects all network and broadband providers globally.
GSMA (global operator alliance with 750 operator members): King’s introduced cost-saving Generic 5G Slice Templates, initially included in the de-facto industry guidebook and which are now being adopted by [text removed for publication] telecoms operators globally.
2. Underpinning research
5G provides significant improvements in performance over 4G in that capacity (data rate in bits/s), reliability (number of dropped calls) and latency (e.g. time for a website to load) are improved by several orders of magnitudes. This enables unprecedented applications, such as untethered industrial control for Industry 4.0 or remote robotic surgery. The improved performance is achieved through a departure from legacy design principles, some of which King’s has contributed to through pioneering research.
Notably, 2G – 4G telecommunications systems are cell/network-centric in that capacity and quality of experience (QoE) are dictated by the location of the basestation. Departing from this design approach, King’s has contributed to fundamental research enabling the transition to device-centric architectures where capacity and QoE are dictated by the degree of connectivity experienced from the mobile terminal’s point of view. This has become the guiding architecture approach for 5G systems. Specifically, King’s pioneered and contributed to i) decoupled up and downlinks; ii) fixed-mobile convergence; iii) application-centric edge-cloud design; and iv) software-enabled network slicing:
2.1 Decoupled up and downlink architecture
In 2014, we introduced a paradigm shift in cell-centric telecommunication networks by allowing up and downlinks to terminate at different basestations. We showed significant capacity gains but also reliability and latency gains. One research contribution was to design a viable protocol allowing for two prior non-related basestations to handle the same call flow; this has been achieved through a complete decoupling of both data and control channels. The biggest challenge was to formally analyse the new system, since analytical models to represent this system were entirely missing. We had thus introduced a novel analytical framework quantifying the gains of decoupling in the lower frequencies but, surprisingly, showing that gains for higher frequencies such as millimetre wave systems eroded unless highly directional antennas were used [1]. This fundamental body of research resulted in several milestone publications, including a Best Paper Award at IEEE’s flagship conference which laid the groundwork for [1], with many citations indicating the opening of a new field of research. [text removed for publication].
2.2 Fixed and 5G mobile convergence
Another important transition in 5G is reconciling the different wireless technologies under the same network management. We made a prominent contribution in changing the way fixed and mobile broadband networks operate by converging their capacity and enabling them to jointly deliver critical services. Given these networks follow different protocols and often reside on different infrastructures, the main challenge was how to seamlessly bring them together. Through theoretical research and experimentation, we have shown how aggregating the two networks at higher layers, and using Multi-Path TCP (MPTCP), can result in better performance and QoE than the combined capacity of two separate streams [2]. [text removed for publication].
2.3 Multi-tier 5G edge-clouds
We have pioneered the notion of an application-centric caching-as-a-service in cloud and edge-cloud 5G networks. We have formulated the service delivery as a formal trade-off between cost and performance [3]. The result was a clear multi-tier cloud strategy with analysis supporting a strong case for edge-cloud deployments. [text removed for publication].
2.4 Generic 5G slice templates
Another area of significance in the transition from 4G to 5G is moving to softwarised, virtualised, and cloud-supported systems. One of our major contributions in this area has been the development of a programmable multi-domain and national scale testbed [4,5]. That allowed us to research and contribute to the definition of network slicing. Network slicing is a means to establish and guaranteeing personalised end-to-end services through re-programming the 5G network. Our main research contribution was the introduction of Slicing Templates, as a solution to addressing technical and business aspects of network slicing over an infrastructure with multiple ownerships [6]. [text removed for publication].
2.5 Co-creation of 5G use-cases
The ensemble of above research contributions enabled a fresh look at mobile architectures able to deliver 5G applications. Enabled by [4,5], King’s engaged in interdisciplinary co-creation research in health, transport and the arts. This has inspired numerous societal 5G applications at global scale and has led to ample media coverage. Furthermore, it stimulated research in King’s pioneering concept of the Internet of Skills, a next-generation internet powered by 5G, AI and robotics, able to execute physical skills remotely. [text removed for publication].
3. References to the research
[1] Elshaer, H, Kulkarni, MN, Boccardi, F, Andrews, JG & Dohler, M 2016, 'Downlink and uplink cell association with traditional macrocells and millimeter wave small cells', IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, vol. 15, no. 9, 7493676, pp. 6244-6258. https://doi.org/10.1109/TWC.2016.2582152 (based on IEEE Globecom 2014 Best-Paper: Elshaer, H, Boccardi, F, Dohler, M, Irmer, R, ‘Downlink and Uplink Decoupling: A Disruptive Architectural Design for 5G Networks’, IEEE Globecom 2014, December 2014, Austin, Texas, US.)
[2] Condoluci, M, Johnson, SH, Ayadurai, V, Lema Rosas, MA, Cuevas, M, Dohler, M & Mahmoodi, T 2018, 'Fixed-Mobile Convergence in the 5G era: From Hybrid Access to Converged Core', IEEE NETWORK, vol. 33, no. 2, pp. 138 - 145. https://doi.org/10.1109/MNET.2018.1700462
[3] Ghoreishi, SE, Karamshuk, D, Friderikos, V, Sastry, NR, Dohler, M & Aghvami, A-H 2019, 'A Cost-Driven Approach to Caching-as-a-Service in Cloud-Based 5G Mobile Networks', IEEE Transactions on Mobile Computing. https://doi.org/10.1109/TMC.2019.2904061
[4] 5G Testbeds & Trials Programme, Hub 1 Project; Department for Digital, Culture, Media & Sport, Dohler, M., Holland, O., Friderikos, V., Sastry, N., Shikh-Bahaei, M., Mahmoodi, T. et al.; GBP3,000,000; 1/07/2017 30/03/2018.
[5] The UK Programmable Fixed and Mobile Internet Infrastructure (INITIATE); EPSRC - Engineering and Physical Sciences Research Council, Mahmoodi, T., Dohler, M., GBP1,676,408, 1/02/2017 31/01/2021.
[6] Jiang, M, Condoluci, M & Mahmoodi, T 2017, Network slicing in 5G: An auction-based model. in 2017 IEEE International Conference on Communications., 7996490, IEEE, 2017 IEEE International Conference on Communications, ICC 2017, Paris, France, 21/05/2017. https://doi.org/10.1109/ICC.2017.7996490
4. Details of the impact
King’s research has enabled a departure from legacy protocol and architecture design principles with significant impact on global telecoms industries and standards:
4.1 Impact on Ericsson’s [text removed for publication]:
Ericsson is the world’s second largest vendor in telco services, software and infrastructure, and 5G market leader with an annual revenue of USD28,000,000,000. “ 2.5bn subscribers and 40% of the world’s mobile traffic is carried over Ericsson’s networks” [A, page 1], thus powering half of the world’s mobile commercial and societal services. Ericsson and King’s have engaged in close joint research, innovation and deployment/experimentation over the past seven years. The underpinning research 2.1 and 2.5 achieved impact in four areas:
**[text removed for publication] use-case developments: “ King’s played an instrumental role in gauging early 5G architecture capabilities and translating them into viable societal use cases. As a result, Ericsson has spearheaded many use cases [text removed for publication] . These have been documented on Ericsson’s website with specific mention of King’s inception; notably in health, education and gaming, and the arts and culture” [A, p.2]. Important world’s first were the output of King’s work: “ the world’s first robotic surgery over 5G […] which impacted the industry vision in the use of 5G for medical interventions […]” and “ UK’s first 5G 3.5GHz deployment with Vodafone, 5G sliced drone using EE’s and Verizon’s networks, or the world’s first 5G music lesson with Jamie Cullum”. [A p.3, B, C]
Impact on [text removed for publication] : “Based on the pioneering use case developments and 5G prototyping, the work of King’s College London has* [text removed for publication] to address novel 5G use cases” [A, p.3] and “it […] is [text removed for publication] ” [A, p.2]. As a result: “ In confidence, [text removed for publication] ” [A, p.3].
Skills development [text removed for publication] : “Ericsson was able to [text removed for publication] ” [A, p.2].
Marketing, branding and societal outreach: “The work conducted by King’s […] has led to significant press and media coverage* [C] . Notably outlets, such as CNN, BBC, Wired, FT, have continuously reported on the technical advances of this work. [text removed for publication] ” [A, p.3].
4.2 Impact on Konica Minolta [text removed for publication]:
Konica Minolta (KM) offers a large variety of products in the form of office equipment, medical imaging, graphic imaging, optical devices and measuring instruments at a GBP7,400,000,000 annual revenue. It is market leader in enterprise printing products. King’s has engaged with KM over a years-long engagement via [text removed for publication]:
Establishment [text removed for publication] : “The roadmap developed by Prof M Dohler in 2014-2015 outlined a set of opportunities for Konica Minolta around [text removed for publication] capabilities. Furthermore, King’s College London […] advocated for a [text removed for publication] . […] Following the KCL report, Konica Minolta focused on [text removed for publication] known as Distributed Cloud Intelligence (DCI)” [D, E]. Further, “the ongoing collaboration has been specifically useful for DCI and has helped [text removed for publication] ” [D].
*[text removed for publication] : “An important outcome [of the joint projects] has been Konica Minolta’s [text removed for publication] [D, F]. The [text removed for publication] benefits are reported [text removed for publication] [D]; thus, giving an increased competitive [text removed for publication] [D].
4.3 Impact [text removed for publication] via 3GPP standards contributions:
The 3rd Generation Partnership Project (3GPP) is the world’s largest umbrella of seven standards organizations which develop cellular telecommunications technologies, including radio access, core network and service capabilities. More than 700 industry partners contribute to 3GPP, making it the largest standards body shaping the £-trillion telco market. King’s research on converged 5G telecommunications architectures was integrated into 3GPP with the support of BT. The standards contributions underpinned by the research have been fed directly into Release 16 and 17 Technical specifications [G], i.e. 3GPP TS 23.501 Sections 4.2.10. (Architecture Reference Model for ATSSS Support) and 5.32 (Support for ATSSS) as well as 3GPP TS 23.502 Section 4.22 (ATSSS procedures). The underpinning research 2.2 achieved impact in two areas:
*[text removed for publication] 5G technical specifications: “The work that King’s and BT carried out on 5G convergence has been [text removed for publication] to support both fixed and mobile access technologies, and in particular the ability for operators to make dynamic decisions to route traffic to customers to optimise quality of experience in the most efficient manner” [H]. The contributions are specification standards, and essential to be implemented, hence confirmed to impact all broadband providers who have both fixed and mobile network infrastructure [H].
**[text removed for publication] 5G feature rollouts: “This [convergence] feature is expected to be incorporated into vendor roadmaps and operator network deployments across the globe as 5G solutions evolve and mature towards a fully converged architecture, delivering significant benefits to customer experience” [G].
**4.4 Development of cost-efficient slicing template for largest telco association:**The Global System for Mobile Communications Association (GSMA) is the world’s largest telco association representing and influencing the business practice of more than 750 mobile network operators worldwide.
King’s underpinning research 2.4 impacted the pioneering concept of a Generic Slice Template that allows multiple operators to exchange the basic information of their network slice without revealing their technical details that could expose their competitive advantage. Hence, for the first time, user services can go across multiple operator domains with the exchange of slice templates between the involved operators. “These insights have provided independent thought-leadership for the GSMA to push for the creation of the Generic Slice Template (GST)” [H].
King’s work was thus included in the “The ‘5G Guide: a reference for operators’ [which] is the de-facto industry guidebook on the commercial, policy and technical considerations for the 5G business case. Following its completion, it was distributed to the GSMA Board for adoption at the February 2019 Board meeting and subsequently distributed to the CEOs of the 750 mobile operators in the world in April 2019” [H]. Importantly, “all the GSMA Board member companies, i.e. the [text removed for publication] largest telcos in the world, have now incorporated or are considering incorporating slicing into their 5G network plans, following a similar strategy as outlined in the document” [H].
King’s work “has impacted the industry’s position on how to optimise the creation and management of slices. Two specific recommendations from KCL stand out: i) optimise granularity of network slices so as to balance the need for more differentiation versus the costs of provisioning slices; ii) and [the creation of] standardised slice templates with predefined and optional fields that could bring down the cost and time to deploy of network slices significantly, guaranteeing interoperability and enabling automation of slice management on global scale” [H].
5. Sources to corroborate the impact
[A] Ericsson Testimonial: [text removed for publication] Ericsson Research; 23 Oct 2020.
[B] Ericsson and King's collaboration on 5G use cases:
i) Ericsson and King’s: Reshaping our world with 5G researchii) 5G Health: Access to remote healthcare specialists iii) 5G Education: Bringing learning to life with educational technology
iv) 5G Arts: Reaching new audiences through connected culture v) 5G Gaming: The future of educational gaming
[C] Media articles on King’s / Ericsson’s 5G collaboration:i) BBC: “Drones to the rescue!” 30 April 2018ii) CNN: “How 5G could change everything from music to medicine”, 5 Feb 2018iii) EIN Presswire: “Jamie Cullum leads world’s first music lesson for music-making charity”, 28 June 2019iv) Financial Times: “5G future: a world of remote colonoscopies”, 28 March 2018v) BBC: “Hologram phone calls sci-fi or serious possibility?” Sept 2019vi) Wired: “More UK cities are testing 5G. Here’s how it could be useful”, 5 Sept 2018.
[D] Konica Minolta Testimonial: [text removed for publication] Konica Minolta; 26 Oct 2020.
[E] Konica Minolta’s new DCI business unit
[F] Konica Minolta’s new product offerings from DCI: 1.) Smart cloud services 2.) Hybrid edge computing
[G] British Telecom Testimonial: [text removed for publication] British Telecom, 9 Sept 2020 & 11 Dec 2017.
[H] GSMA Testimonial: [text removed for publication] GSMA; 17 Sept 2020.
- Submitting institution
- King's College London
- Unit of assessment
- 11 - Computer Science and Informatics
- Summary impact type
- Technological
- Is this case study continued from a case study submitted in 2014?
- No
1. Summary of the impact
Provenance is a record of the processes by which data was produced, by whom, how, and from what other data. Research on provenance at King’s over a decade, and later significant and sustained world-wide adoption, often without King’s direct involvement, have led to the global recognition that provenance is a critical facet of good data governance for businesses, governments and organisations in general. The impact of King’s pioneering work has manifested itself in i) commercial, governmental and research organisations launching new products incorporating provenance functionality, ii) multiple standardisation bodies providing guidance to software engineers, iii) scientific communities coalescing around provenance to ensure trusted information exchange, and iv) regulators asserting that provenance is a technique to address regulatory requirements. Overall, due to King’s research, provenance is now widely regarded as an essential function of an IT system, to provide a trusted account of what the system performed and the data it manipulated.
2. Underpinning research
The World Wide Web Consortium (W3C), the standardisation body for the web, defined provenance as a record that describes the people, institutions, entities, and activities involved in producing, influencing, or delivering a piece of data or a thing. Back in 2013, W3C published PROV – a standard for expressing such a record to make it possible to store, exchange, query and manipulate it in interoperable ways.
King’s 2014 impact case study described Miles’s contribution to the requirements [1] underpinning provenance and the methodology required to put it in practice. Since 2014, new research has been conducted and novel significant impact has emerged. King’s researchers have conducted world-leading research that builds on PROV to deepen the understanding of provenance, to develop software engineering methodologies and techniques to deploy provenance, and to conceive provenance-based techniques for systems to produce explanations about their decisions. This research has been led by Miles (2007-present), Curcin (2014-present) and Moreau (2017-present). Their teams have worked with a range of applications that share a critical data governance imperative of demonstrating the quality of data, including health management systems (UKRI grants EP/N027426/1 and KTP-509790), automated decision systems in finance (UKRI grants EP/R511559/1 and EP/S027238/1), and command and control systems (US Navy grant N629091812079). Through this body of research, and specifically the following three strands, provenance emerges as a fundamental technique for data governance.
[1. Understanding] In collaboration with leading researchers who specified PROV, Miles conducted a retrospective analysis that explicitly characterised the scope, requirements, guiding principles, and design decisions that resulted in PROV [2]. This post-standardisation analysis demonstrated the broad consensus about the design of PROV, helped increase the understanding of PROV and its interoperability, and was a scientific output that has contributed to its widespread adoption.
[2. Creating and managing] A key practical challenge that hampers adoption of new technologies is the effort required to deploy them in practice; specifically, for provenance, a challenge was the effort involved in automatically creating provenance that accurately describes the actions performed by an application. With this concern in mind, research has taken place in two complementary approaches, both supporting the novel paradigm of declarative construction of provenance.
First, Curcin defined provenance templates [3] as abstract provenance fragments representing meaningful domain actions (UKRI grant EP/N027426/1). Templates were conceived to generate a model-driven service interface for domain software tools to routinely capture the provenance of their data and tasks. By exposing a domain-focused interface for provenance, Curcin demonstrated that provenance templates can capture the audit trail of a task and its resulting data. The use of provenance thereby enables users to place their trust in systems; in particular, it facilitates reproducible research, which was demonstrated by a range of provenance-based queries, in the context of Learning Health Systems. Curcin’s provenance templates were subsequently used in an Innovate-UK project (KTP-509790), where the provenance templates technology was demonstrated to be deployable within legacy applications in order to enrich a product with novel data governance functionality.
Second, Moreau instigated UML2PROV [5], a technique capable of producing provenance automatically for programs specified according to the industry-adopted UML modelling language. This technique also relies on an extensive set of provenance fragments describing common programming language patterns. The significance of this technique is that it shows that high-level program specifications can be the source of automatic provenance generation, thus reducing the human effort involved in creating provenance, and hereby facilitating provenance adoption.
[3. Exploiting] Provenance is routinely regarded as a technique by which trust can be endowed to systems, by enabling the tracing of data that flow through them, so that it can subsequently be inspected by users. All the information, data dependencies and processes underpinning a decision are collectively the provenance of the decision. In grants EP/R511559/1-EP/S027238/1, in collaboration with the Information Commissioner’s Office (ICO), Moreau and his team have developed an approach that exploits provenance to construct human-consumable explanations of decisions made by systems [4]. The King’s team demonstrated that some GDPR-related questions pertaining to automated decisions can be answered from their provenance: the solution relies on semantic mark-ups embedded in the provenance, which are exploited by queries to extract the relevant elements that have influenced a decision; these elements are then used in declarative specifications of explanations realised by a natural language generation engine. In addition, Moreau and collaborators have developed a technique called “provenance analytics” [6], which demonstrates that Machine Learning techniques can be applied to analyse provenance in an automated manner so that “quality” or “trust” assurance can be derived without manual human inspection.
To sum-up, research at King’s has consisted of (i) requirement engineering linking design decisions to their rationale, which help improve PROV understandability, (ii) declarative methods reducing human effort involved in producing provenance, and (iii) techniques to derive explanations and measures of data quality from provenance. This research has helped transition provenance, originally seen as a laboratory experimental concept, to a fundamental data governance technique, deployed in practical applications.
3. References to the research
Groth, P., Gil, Y., Cheney, J., & Miles, S. (2012), Requirements for Provenance on the Web, Int. Journal of Digital Curation 7(1), 39-56. https://doi.org/10.2218/ijdc.v7i1.213
Moreau, L., Groth, P., Cheney, J., Lebo, T., & Miles, S. (2015), The Rationale of PROV. Journal of Web Semantics. https://doi.org/10.1016/j.websem.2015.04.001
Curcin, V., Fairweather, E., Danger, R. & Corrigan, D. (2017) Templates as a method for implementing data provenance in decision support systems, J. of Biomedical Informatics, vol. 65, pp. 1-21. https://doi.org/10.1016/j.jbi.2016.10.022
Huynh, TD., Stalla-Bourdillon, S. & Moreau, L. (2019), Provenance-based Explanations for Automated Decisions: Final IAA Project Report. (Accepted in ACM journal Digital Government: Research and Practice, Nov. 2020).
Sáenz Adán, C., Pérez Valle, B., García Izquierdo, F.J. & Moreau, L. (2020), Integrating Provenance Capture and UML with UML2PROV: Principles and Experience, IEEE Transactions on Software Engineering. https://doi.org/10.1109/TSE.2020.2977016
Huynh, D., Ebden, M., Fischer, J., Roberts, S & Moreau, L. (2018), Provenance Network Analytics: An approach to data analytics using data provenance, Data Mining and Knowledge Discovery. https://doi.org/10.1007/s10618-017-0549-3
4. Details of the impact
Research on provenance at King’s, along with the influence of standardisation of provenance at the World Wide Web Consortium (W3C) (Miles, PROV-DM), technology transfer to business (Curcin), toolkits and services to promote take-up (Moreau), influence of guidelines for data protection (Moreau), and, later significant world-wide adoption without King’s direct involvement, have led to the global recognition that provenance is a critical facet of good data governance and an essential function of an IT system.
Given the openness of W3C standard PROV, it is impossible to track all the usages of PROV. Furthermore, as PROV is used in the background, generally, as part of a data management function, it is challenging to identify its impact in isolation to the rest of the functionality. Thus, below, we explore key strands of impact, including explicitly listing publicly documented usages of PROV or PROV concepts [G].
4.1. Impacts on public policy and services
Many governmental organisations have a duty to make data publicly available: open government data is regarded as creating value (worth billions of pounds world-wide) in many areas, including transparency, democracy, participation, innovation and efficiency. We focus on two illustrations of open government. In the UK, the Gazettes are the official journals of public record, whereas, in the US, the U.S. Global Change Research Program (USGCRP, a cooperation between 13 federal agencies) publishes a quadrennial National Climate Assessment. Both have independently adopted PROV as a mechanism to improve public navigation and access to information through the use of linked knowledge, thereby addressing an overarching aim of information transparency.
“The credibility, trust, and integrity of the [Gazette] data has been strengthened because of PROV” [D, p.11]. PROV “works in the background to provide a clear, ethical and transparent data source” [D, p.1], and “ensures that the official public record is credible and accurate” [D, p.2].
In the US, the use of PROV had an impact on the environment, as the “policy debate on climate change has been influenced through the work of USGCRP: PROV-linked research meant that all information was meticulously documented which led to less controversy” [D, p.16]. “There have been a lot of scandals and difficult thinking in the scientific community about reproducibility and about accuracy and integrity, and […] PROV is a structure that can really support advances in general scientific practice to address those concerns” (Climate Adaptation Lead, USGCRP) [D, p.5]. Overall, PROV helps increase trust in data and processes, and one of its tangible impacts is the reduction of FOI requests, “as the accuracy of PROV renders many FOI requests pointless” [D, p. 13].
4.2. Impacts on practitioners
4.2.1. Impact on standards and standardisation bodies
The impact on standards reported for the REF 2014 now goes well beyond the original standardisation of PROV at W3C, which was specified in 2013 as a domain-agnostic ontology for provenance, building on research at King's by Miles on the requirements for provenance. Since then, we can report secondary adoption of PROV, which has been referenced and key concepts directly imported in specifications published by other standardisation bodies (HL7, Allatrope Foundation, RDA, IVOA), reaching out to 500+ member organisations. All these specifications have an impact on a range of practitioners, including software engineers, working on scientific data management systems and health care systems. These examples of secondary standardisation build on King’s research but without direct involvement from King’s, which is evidence of broad and sustained adoption of provenance. Specifically, Health Level Seven (HL7) is the international body for healthcare standards (500+ corporate members), representing healthcare providers, government stakeholders, payers, and pharmaceutical companies. HL7’s Fast Healthcare Interoperability Resources (FHIR) model is the standard for programmatic communication of health data, and it includes a mapping to the core concepts of PROV [E]. The Allatrope Foundation (a consortium of 40 pharmaceutical organisations) defined a universal data format that standardizes laboratory experimental parameters in order to remove human error and enhance scientific reproducibility; its Audit Trail ontology is building on PROV [G].
The Research Data Alliance (RDA, 50+ research organisations, 11,000 individual members) aims to build the social and technical bridges to enable open sharing and re-use of data. The RDA Recommendation [C, p10-11] incorporates PROV as metadata for the management of research objects in data centres to support reproducible research. The International Virtual Observatory Alliance (IVOA), comprising 20 Virtual Observatory (inter-)national programs, has published its recommendation ProvenanceDM applying PROV to astronomical data [H, p.13].
4.2.2. Impact on guidance for AI practitioners
The General Data Protection Regulation (GDPR) is the European wide regulatory framework that codifies some rights for data subjects (the users who have provided data in return for services) and obligations on data controllers (the organisations that are providing these services). A key challenge is that regulatory frameworks remain high-level and do not specify practical ways for organisations to become compliant; this is a particularly salient problem for companies adopting AI in their new products. The Information Commissioner Office (ICO), the data protection regulator in the UK, published guidance around “Explaining decisions made with AI”, explicitly referring to provenance: “Such provenance information provides the foundations to generate explanations for an AI decision, as well as for making the processes that surround an AI decision model more transparent and accountable.” [F, p.59] The ICO report [F, p.59] includes a link (https://explain.openprovenance.org/loan/\) to King’s demonstrator for provenance-based explanations [4], which in collaboration with ICO was tailored to address seven key requirements of the GDPR. The software system, the first of this kind, provides a tangible artefact to which implementors can refer when implementing high-level guidance. The mark-ups and associated queries over provenance identified in the research were instrumental in delivering fit-for-purpose explanations and present a significant advance for AI practitioners. Appendix 4 [F, p.124] further refers to the King’s report [4].
4.2.3. Widespread adoption of PROV
Two fundamental characteristics of PROV is the simplicity of its core and its design based on a wide consensus, promoting interoperability, as exposed by King's research [2] and provenance toolkits and services hosted at openprovenance.org. For these reasons, several communities have coalesced around PROV (or PROV concepts) with a view to facilitating the interoperable exchange of provenance. The key driver for these communities stems from their work with complex workflows, involving multiple stakeholders, typically each with their respective IT systems, across which flows of data and decisions need to be documented to ensure their auditability or reproducibility. A growing list of adaptors (in excess of 50, counted November 1, 2020) is maintained at [G]. For instance, the environmental science community is adopting PROV across the world, as illustrated by a range of projects, in the USA (NASA, JPL, PNNL), Germany (DLR), Austria (EEA), and Australia (Geoscience Australia, CSIRO). There is adoption of FHIR provenance in the HealthCare community, including companies such as Astrazeneca, Smile CDR and Perspecta. In the Astronomy community, Applause, a collection of photographic plates with full provenance, is operationally deployed by the Leibniz-Institut für Astrophysik Potsdam (AIP). Finally, innovative commercial products are exploiting provenance capabilities, Surround Australia, Smile CDR, Perspecta, and Imosphere, which we specifically discuss in Section 4.3.
4.3. Impact on Imosphere Ltd’s commerce and practice
Imosphere Ltd is a company of 50 employees commercialising solutions for healthcare organisations. Their flagship Atmolytics product is a tool that produces interactive reports from patient cohort data. Thanks to an Innovate UK grant (KTP-509790), Atmolytics incorporated elements of Curcin's provenance template technology, to enable a range of new functionality in the product, such as a full audit trail of data sets, versioning, analytics and reports providing transparency and increasing the users' trust in the data findings. Provenance templates were deployed to model key Atmolytics behaviours, such as patient cohort management and decision making, and ensure a faithful provenance record in a standardised PROV format. The provenance module differentiates Atmolytics from the competition, by its functionality to maintain the integrity of data, supporting informed and trusted decision-making [A].
The new, provenance-enabled version was launched in the summer of 2018, and has been installed at customer sites in the UK, Europe and USA. There are in excess of 4,000 users in the UK and USA, managing data of over 1,000,000 patients (by November 2020). Among them, 60 councils in the UK are benefiting of provenance functionality to manage their individual care budgets. In the US, the National Cancer Institute “City of Hope” tracks over 100,000,000 medical events, whereas the medical school Meharry Medical College, Tennessee, tracks 250,000 patient’s data. For the end-users of the system the “network graph view of provenance data is a far more natural way of visualising and querying historical relationships between patient cohorts and analytical tasks”. [B]
Overall, due to PROV, Imosphere has benefitted significantly: “Using the W3C PROV standard and provenance templates for this task, saved us years in design and development time and ensured we are standard compliant for any further extensions, reducing time to market by approximately one year.” Furthermore, “the introduction of data provenance capabilities in the software, has also improved the software engineering aspect of our data analytics portals, as it promotes good practice in reusing and documenting analytical components across reports, avoiding duplication”. (Imosphere CEO, [B])
5. Sources to corroborate the impact
A. YouTube Atmolytics video, July 2018
B. Testimonial from CEO of Imosphere Ltd.
C. Weigel, T., Plale, B., Parsons, M., Zhou, G., Luo, Y., Schwardmann, U., Quick, R.,
Hellström, M., Kurakawa, K. (2018). RDA Recommendation on PID Kernel Information (Version 1)
D. Impact Evaluation of PROV - a provenance standard published by the World Wide
Web Consortium, by Impact Science, July 2020
E*.* HL7 FHIR version 4.0.1 (section 6.3.3), October 2019
F. Explaining decisions made with AI, Project Explain, ICO, May 2020
G. Adoption of provenance, page maintained by Luc Moreau, November 2020, (password: REF2021-kcl)
H. International Virtual Observatory Alliance Provenance Data Model, April 2020