Impact case study database
Designing and implementing a digital platform to reduce A&E peak time demand across the South West through the provision of real time information to empower patient decision-making
1. Summary of the impact
Pressures on Accident and Emergency (A&E) departments in the UK have led to patients waiting longer to be seen and treated. Interdisciplinary research at the University of Exeter (operational research and behavioural economics) led to the development of a digital platform and app ( NHSquicker), which, by showing real-time waiting times at A&E and Urgent Care Centre/Minor Injuries Units (UCC/MIUs), enables patients to make an informed choice of where to go for treatment.
The impact of the research has been: a) The development and adoption of a new service available to 1.7 million patients since 2017, leading to an enhanced patient experience of acute care services b) Improved access and use of NHS services which positively impacted A&E pressures during peak periods c) Influenced changes in practice in seven NHS Trusts in South West England with the subsequent adoption of the technology by the NHS.
2. Underpinning research
The Exeter team has a long history of research on theory and practice in health care modelling of decision-making. There are two particular strands of research underpinning the impact on reducing hospital waiting times. The first strand led to the development of the platform and the App, NHSquicker. The second led to the refinement of the App to incorporate behavioural insights to support patients to make an informed choice of where to go for treatment.
Mustafee’s research focuses on understanding patient flows in healthcare facilities. His work emphasises the use of techniques such as computer simulation and data-driven analytics to support decision-making in the NHS [3.1; 3.2]. This work was extended by Mustafee and Powell by integrating analytics with real-time data feeds, leading to the idea of ‘Hybrid Models’ [3.3]. The research has been central in demonstrating the potential of these types of models to healthcare planning. Following a collaboration with Torbay & South Devon Foundation Trust (TSDFT) in 2015, the team developed a framework using hybrid models to look at designing a solution for shaping the demand for urgent care. With further funding from the University of Exeter, the team developed the NHSquicker App. The app provides live waiting times data from A&E departments and MIUs, and estimated travel times to the A&E and MIU. It also provides information on alternative local health services like GPs and pharmacies based on the location of the user – helping patients make decisions on where to go for medical treatment. Their research identified a particular need for an open system based on linked administrative data and that a broad range of data standards were crucial to the delivery of the app. As such, the research team worked with the NHS technical teams (IT systems engineers, database administrators) to develop the NHSquicker data standard. This standard allows NHSquicker to receive data from A&E patient flow systems such as PatientFirst™, Symphony™, TrakCare™, Oceano™ and EDIS™, allowing any Trust to relay their waiting times via the NHSquicker platform. To test the efficacy of the app and to enable wider use of the framework, a network of NHS Trusts was established - the Health & Care IMPACT Network, comprising five Trusts from Devon and Cornwall (Royal Devon & Exeter, Torbay and South Devon, Northern Devon, Plymouth Hospitals and Royal Cornwall Hospital NHS Trusts). Today, the app also receives live feeds from 9 A&E departments and 18 MIUs/UCC, operated by Taunton and Somerset NHS Trust and University Hospitals Bristol NHS Trust, and an additional 12 centres of urgent care.
The second strand of research focuses on how to better understand patients’ behaviour when facing such health decisions. Kaplan’s work on decision-making under uncertainty analyses the communication of uncertainty so users can make the best decision for themselves [3.4]. The research shows that individuals’ decision-making improved with the more information they have available to them. Chakravarty works on decision-making problems where decision-makers do not know the probability of future states [3.5]. Together, Kaplan and Chakravarty designed experiments to evaluate whether NHSquicker could lead to reduced waiting times at the A&E. The study found that patients prefer A&Es over MIUs. The study also found that revealing exact waiting times (if feasible) would lead to more people going to A&E, particularly if waiting times are short, and fewer will go if waiting times are long. The researchers experimentally tested these two scenarios. The results [3.6] confirmed the need for more precise information when waiting times are longer and less accurate information when the waiting times are shorter. In early-2020, the team started preliminary work with the network on the granularity of information that would be made available through the app. However, the pandemic meant that the stakeholders focussed on the management of COVID-19, and therefore organising either a physical or an online workshop to discuss the revised algorithm was not feasible.
3. References to the research
Mustafee, N. (Ed.). (2016). Operational Research for Emergency Planning in Healthcare: Volumes 1 and 2. OR Essentials. ISBN: 978-1-137-53567-2 (Vol 1), 978-1-137-57326-1 (Vol 2). Basingstoke, Hampshire: Palgrave Macmillan. DOI: 10.1057/9781137535696
Mustafee N, Taylor SJE, Katsaliaki K, Brailsford S (2009). “Facilitating the analysis of a UK national blood service supply chain using distributed simulation2. Simulation, 85(2), 113-128.DOI: 10.1177/0037549708100530
Powell, J.H. and Mustafee, N. (2017). "Widening Requirements Capture with Soft Methods: An Investigation of Hybrid M&S Studies in Healthcare," Journal of the Operational Research Society, 68(10):1211-1222. DOI: 10.1057/s41274-016-0147-6
Marimo, P., Kaplan, T. R., Mylne, K., & Sharpe, M. (2015). “Communication of Uncertainty in Weather Forecasts”. Weather and Forecasting, 30(1): 5-22. DOI: 10.1175/WAF-D-14-00016.1
Chakravarty, S. and Kelsey, D. (2015). “Sharing Ambiguous Risks”. Journal of Mathematical Economics, 56, 1-8. DOI: 10.1016/j.jmateco.2014.11.001
Chakravarty, S., Kaplan, T.R., and Mustafee, N. (2020). “Altering Wait Time Information to Reduce A&E Overcrowding”. University of Exeter Business School Working Paper https://ideas.repec.org/p/exe/wpaper/2003.html.
4. Details of the impact
A&E waiting times in the UK have increased substantially over recent years and have become a major policy issue. This led to a waiting time standard that states that 95% of patients should be assessed, treated, then either admitted or discharged within four hours of presentation at an A&E or an UCC/MIU. The Kings Fund, 2020Footnote:
The King's Fund (2020). What’s going on with A&E waiting times? https://www.kingsfund.org.uk/topics/emergency\-care (last accessed Sept' 2020). claimed that since July 2013, A&Es have not met this standard.
In 2015, the Exeter team worked with the Health & Care IMPACT Network to implement the NHSquicker app so that patients can choose the appropriate type of treatment facility for their condition. The work demonstrated that while patients do not have a direct role in scheduling operations of an urgent care facility, the decisions they take have a bearing on its performance. The work showed that real-time data on A&E/MIU wait times influenced destination choices made by patients. These results benefited the NHS trusts in the South West in two ways. First, by improving the appropriateness of the choice of an urgent care centre, and second, by smoothing demand over inevitably stretched facilities, particularly those offering emergency treatment.
The importance of the app is reflected in the following statement by Associate Director of Delivery South West Academic Health Science Network: “ *Professor Mustafee has developed new analytic models (which integrated a ‘nudge’ algorithm) based on business intelligence and real-time data and applied that to the development of a real-world innovation which empowers patients to make better, more efficient decisions. The positive impact of his work involves not only the patients who use the app, but also non-users of the app (e.g., they could experience lower wait time as those using the app may have visited an alternative urgent care centre), the NHS staff, and the entire system […] Professor Mustafee’s work has shown real time NHS benefits in Devon.*” [5.1].
The impacts ascribed to the work are described below.
App available to 1.7million patients, with over 40,000 patients directly using it Better information has led to shorter waiting times and improved patient experience. Large numbers of patients are also accessing the service via GP and Trust websites. An in-app questionnaire integrated with NHSquicker received 543 responses between 24/03/2020 and 10/09/2020: 82% of users agreed that NHSquicker helped them at the point of need, whilst 78% responded that it helped them decide where to go. As patients have come to know of the app and download it, the analytics data showed that the vast majority (approx. 87%) retained the app on their mobile devices having found it very useful. One patient commented: “Used NHSquicker this morning .... went to Newton as there was only two people waiting ... booked in ... had consultation ... had three X-rays ... got treatment plan and crutches ... out in less than an hour ...”. Another said: “My GP used Quicker to tell me where I could go to a local MIU, which was 15 minutes’ drive past the ED. I was seen so much quicker despite the drive”.
Impact on A&E services in reducing pressure on A&E services with more patients choosing to go to MIUs in the trusts which first adopted NHSquicker.
A data summary facility developed by the research team is embedded within the app and is used to access summary patterns of patient movements. The evidence from this data from two early adopters of NHSquicker (TSDFT and North Devon Healthcare Trust) show that the app made an impact in directing patients away from busy A&E department at peak times. For example, for the period 2017- Feb 2019 for TSDFT, an analysis of data showed that there was a significant shift in peak time patterns of attendances (that is, between 11.00-16.00). First there was a reduction in A&E attendances, and second, an increase in MIU attendances compared to the same period in the previous year. The overall impact of the app was to reduce attendance in A&E and increase MIU attendance. The significance of the changes in patient patterns of behaviour is reflected in the following comments:
The Chief Executive of TSDFT said, “Quarterly changes in Minor Injury Units and A&E (MIU/A&E) presentations during busy hours (11am-6pm) for ages 18-50 years has shown that there is a significant shift from the start of 2018 in the pattern of attendances, with a reduction in A&E attendances and an increase in MIU attendances. The work of Professor Mustafee has had a significant impact on shaping demand for urgent care. The success of this intervention has impacted significantly upon the pressures faced within the Torbay A&E department and led to new uses of the app. An example that comes to mind is the use of NHSquicker by frontline staff to monitor our four urgent care facilities; the application was originally developed for patients but has now found another use!” [5.2]. The 2018 Care Quality Commission (CQC) report on Torbay and South Devon NHS Foundation Trust reported NHSquicker as a University-Trust Quality Improvement initiative, "The trust had a number of innovative programmes designed to improve services. Quality improvement methodologies were being used to support a number of improvement projects, including the NHS Quicker smartphone application." [5.3]
The Executive Medical Director and Deputy Chief Executive at RD&E Trust and Medical Director at North Devon Healthcare NHS Trust said, “The success of this intervention (NHSquicker) has impacted significantly upon the pressures faced by the A&E departments at both the RD&E and the North Devon Hospital. There has been a significant shift from the start of 2018 in the pattern of attendances, with a reduction in Emergency Department attendances and an increase in Minor Injury Unit attendances, which has led to better services for the patient and led to reduction of overcrowding in the Emergency Department” [5.4].
A senior member of NHS Horizons, which is a specialist team within the Improvement Directorate of NHS England and supports “leaders of change, teams, organisations and systems to think differently about large-scale change, improve collaboration, and accelerate change”, says: “ *From the outset, I believed Professor Mustafee’s work to be an exemplar of how innovative research can shape practice and have a significant impact on the lives of patients and staff. The success of this intervention has benefitted the entire system greatly and helped ease the pressures faced by many A&E Departments in the South West.... I believe the NHSquicker app has the potential to be developed on a national scale, in order to expand the positive impact of this work to the wider NHS network.*” [5.5]
3. Shaping practices in multiple Trusts, and the subsequent adoption of the technology by the NHS
To date, NHSquicker receives data from all acute Trusts in Devon, Cornwall plus one Trust in Somerset, and one in Bristol. In June 2019, following requests from TSDFT, the research team collaborated with NHS Digital to develop a new version of NHSquicker, which provides on-demand information on GPs, pharmacies and sexual health clinics by directly using the NHS Directory of Services (DOS) database.
NHSquicker v2 was launched in February 2020. The NHS Director from Torbay described how the app was having an impact across the Trust and led to better decision-making. He said: “It’s fantastic – I use this every day to see what is happening in my organisation and the status of the adjoining trusts. Seeing data in real time allows us all to plan ahead. This is a first nationally!” A Senior Manager (Digital Transformation) spoke of the use of real-time data for empowering patients “… what the Trust wants to use it for, is a tool to drive patient behaviour” and the advantages of a system-wide view where data is pulled-in from multiple Trusts, “It’s creating value from a wider point of view. So, it would be value for patients, but it will also be value for the system because it’s about allocating people into a more efficient place to be” [5.6].
The Executive Medical Director and Deputy Chief Executive of RD&E Trust and the Medical Director of Northern Devon NHS Trust, said, “Professor Mustafee’s intervention has been so positive that we will be happy to integrate our new single electronic patient record system (EPR), which will be rolled out later this year, to send data feeds to the NHSquicker app. This demonstrates our recognition that they have created something which is both valued and has the potential to improve the service the NHS offers” [5.4].
NHSquicker was featured in BBC (Spotlight) and ITV news (South West) and other media channels [5.7] and was also a finalist in the Health Service Journal HSJ2018 award in the category “Enhancing Care by Sharing Data and Information” [5.8].
4. Shaping future adoption
The ongoing pandemic has negatively affected some activities planned during the REF impact period. The next phase of the work is to develop and evaluate the future version of the app taking into account the research results from the behavioural economics research [3.6].
Future work will be focused on scaling up the use of the app from a regional to a national setting. Another stream of work is to improve the performance of services like Pathway (used by the NHS 111 service to triage) by utilising the data collected by NHSquicker [5.5].
5. Sources to corroborate the impact
Letter of testimony: Director of Delivery, South West Academic Health Science Network, stating the importance of the NHSquicker app. They highlight, “ Innovations such as these could benefit many international Health Systems, as locked-up data in healthcare information systems is made available through real-time technologies and which, in turn, can empower patients to make an informed choice.”
Letter of testimony: Chief Executive, Torbay and South Devon NHS Foundation Trust, evidencing the impact of the NHSquicker app on A&E services.
The 2018 Care Quality Commission (CQC) report on Torbay and South Devon NHS Foundation Trust, which reported NHSquicker as a University-Trust Quality Improvement initiative (page 13).
Letter of testimony: Executive Medical Director and A&E Consultant at RD&E and Medical Director at Northern Devon NHS Foundation Trust, highlighting how the app led to a better services for patients in Emergency Departments. They point out, “ As the Medical Director of two NHS Foundation Trusts, I would like to stress how the interoperability aspect of the app is of utmost importance. The system level, which allows the app to pull in data from not just one, but multiple Trusts at the same time, plays a crucial role in allocating patients more efficiently across multiple Trusts, therefore relieving A&E departments of some of the pressure, making this system truly beneficial for the entire NHS.”
Letter of testimony: Senior member of NHS Horizons Team, NHS England stating NHSquicker is an exemplar of how innovative research can shape practice.
Quotes from healthcare professionals (interviews), who detailed how NHSquicker has positively impacted their Trusts
NHSquicker was featured in BBC (Spotlight) and ITV news (South West): https://web.archive.org/web/20200421094405/https://www.health-impact-network.info/media/
List of finalists in the Health Service Journal HSJ2018 awards. NHSquicker was shortlisted as a finalist in the category “Enhancing Care by Sharing Data and Information” https://web.archive.org/web/20210301122126/https://www.hsj.co.uk/the-hsj-awards/hsj-awards-2018-finalists-announced/7023236.article
Additional contextual information
Grant funding
| Grant number | Value of grant |
|---|---|
| 0000 | £19,907 |
| 0000 | £2,945 |
| 0000 | £69,460 |
| 0000 | £7,500 |
| 0000 | £8,000 |
| 0000 | £15,000 |
| 0000 | £5,000 |