Impact case study database

1. The Wales “Reasonable Worst Case” Scenario: planning for hospital bed provision

“The importance and the impact of this will never be forgotten by many of the planning and operational decision makers in the NHS” [ C1].

The “Reasonable Worst Case” (RWC) was a set of challenging, but realistic, model scenarios for the Welsh winter. The “Most Likely Scenario” (MLS), was a short-term forecast based on data tracking. The Swansea model identified early warning of severe hospital demand, made at a time when COVID-19 case numbers were very low (only 25 per day in August), and the risk of a ‘second wave’ was uncertain, with many doubts it could be as large as the first. The result of using the RWC was successfully meeting winter demand, with an efficient provision. The RWC impacted tactical (3-6 months) and operational (1-42 day) decisions across all Welsh hospitals, providing the basis for staff estimates for: test-trace-protect, in-hospital testing and scheduling, laboratory staffing requirements, and local and government budget allocations. The RWC/MLS was the central assumption for hospital bed and critical care capacity planning, elective operating schedules, and procurement decisions. It was used in every health board in Wales, and the NHS Wales Informatics Service, as part of standardised planning assumptions by Q3 of 2020.

Quantitative analysis by Cwm Taf Morgannwg University Health Board (CTM UHB) showed the Swansea Model enabled accurate estimates of inter-arrival rates for COVID-19, and determined the need to increase critical care capacity by 150% by December. Since this increase had associated cost in more nurses (1:1 care) and doctors (1:6), prior warning of timing and volume was essential for reorganisation. As a consequence, there was: “ minimal requirement for emergency rates or service closure” [e.g. a pre-emptive enhanced pay for critical care nurses was adopted across Wales to ensure staffing numbers] *. “Because of the accuracy, there was very minimal overprovision of the service, or knee jerking when faced with steep in-day increases in demand, in effect optimising clinical and cost effectiveness” [ C1].

This was in direct contrast to the response during the first peak, before the availability of the Swansea Model. Trends in the Welsh epidemic were not always in sync with other areas of the UK, yet initial planning had only UK-level scenarios available to rely on. The result in CTM UHB was considerable (167%) over-provision of costly critical care staffing over the Easter weekend [ C1]. The impact is supported by the Chief Information Officer, CTM UHB [ C1]:

“At an operational level, the models and the advice of the Swansea Team have supported NHS organisations to very accurately predict critical care and hospital bed requirements. Furthermore by being able to provide accurate forecasts and track the observed versus actual, organisations were able to maintain and give hope to extremely fatigued clinical staff, that we were close to, and then over the peak. This confidence has undoubtedly helped individuals and organisations through one of the most challenging periods the NHS faced in recent times”.

And further, by UK Deputy Chief Scientist [ C2], who presented our RWC, alongside SPI-M models for the 4 home nations, to the UK Cabinet Office in October 12^th 2020:

“Without it, it would have been very difficult for Health Board planners in Wales, and overarching policy makers to set scenarios that were realistic, given great uncertainties at the time, in both transmission and the effect of interventions. Congratulations on an excellent and important piece of work. Thank goodness you were there”.

2. Impact within Welsh Ambulance Service NHS Trust (WAST)

In September 2020, WAST carried out its tactical seasonal planning for Winter, using a discrete event simulator to combine available capacity (rosters) and other resources, with an input of call volume, to predict expected performance. There was therefore a requirement for a long-term input of likely COVID-19 hospitalisations, which could be translated into expected call volume, and then used in the WAST simulations and planning. This was provided by the Swansea Model RWC. The modelling indicated (by 1^st October) that in every scenario for December, WAST would find it difficult to meet demands without additional resources [ C3].

In response, the WAST CEO shared predictions with all health boards, commissioned support from St John’s Ambulance, was psychologically prepared for what was coming, and was able to move quickly to re-engage fire-rescue and military when needed. The predictions proved accurate, with ‘Red Call < 8 min response rate’ realised at 54% against an October predicted 53%, and ‘Amber 1 Call’ median was 60 mins against an October predicted 48 mins. The impact can be supported by Assistant Director, Commissioning & Performance, WAST [ C4]:

“Sharing these scenarios has been critical in our winter modelling. Without the early warning modelling scenario, there would have been a significantly higher compromise to patient safety”.

3. Design of the Firebreak and subsequent non-pharmaceutical interventions (NPIs)

The model provided WG with evidence for the timing, level and duration for Wales’ winter NPIs. As cases rose sharply in October, we estimated the type and duration of intervention required for an efficient, short-sharp reduction in transmission. The scenarios were developed on 12/10/20, and delivered to the First Minister on 16/10/20, who announced the ‘Firebreak’ on 19/10/20. Important for public scrutiny and engagement, our evidence was published simultaneously [ C5].

The impact was one of the most effective short-term interventions to date in the UK, with R_t reduced from approximately 1.4 to 0.8 [ C6] and likely lower during the first week, resulting in a prevalence 50% lower than England, which Wales had been closely tracking during the autumn [ C7]). Notably, a ‘re-bound’ was avoided. There was a 6-week period before return to pre-firebreak incidence, closely following predictions. Vital time was gained to prepare for winter seasonal peaks in hospitals, which would have been harder to manage if cases had risen from a higher November base prevalence. Crucially, cases, deaths, and hospitalisations were reduced (see Figure 1).

Following the Firebreak, we prepared follow-up NPI scenarios for the winter peak. This proved especially challenging after the emergence of a SARS-CoV-2 variant of concern, with a higher transmission rate. We delivered scenarios on the introduction of Level 3, detailed analysis of school closure, and ultimately the need (and duration) of a second full Firebreak [ C8].

Figure 1: *Impact of Firebreak (red bar), estimated by comparing observed events (blue) and counterfactual (red line), assumed to be level 1-2 restrictions for 5 weeks (instead of 17 day firebreak, all other realised NPIs (Level 3, Schools Closure and Firebreak 2) included in both). Note the different impact periods for cases/deaths/ admissions due to different clinical event lags. *

The Firebreak impact can be quantified by comparing observations to a counterfactual. One counterfactual would be no action in October/November (but with all December measures included). This would result in an estimated >120,000 additional cases, that would have overwhelmed ICU across Wales (with no December measures it would, of course, have been far worse again). In Figure 1, we show another counterfactual for the Firebreak, that includes control measures at level 1-2 intervention (Medium-High Alert): control measures that allowed certain activity, with an approximate 0.2 reduction in R_t (level 3 plus school closures were required to reduce R_t below 1 in December). In Figure 1, it was assumed Wales enacted these controls for 5 weeks (twice as long as the Firebreak) before exactly the same December measures were enacted. In comparison, the Firebreak (factual) scenario resulted in approximately 85,000 fewer cases, 5000 fewer hospital admissions, 350 fewer ICU admissions, a reduction in peak ICU occupancy from 150 to 100, and 1100 fewer deaths (a 30% reduction) between 23^rd October and 31^st December. If school closures (a key recommendation of our modelling work in Nov/Dec) were omitted from the counterfactual, a further 16% more cases would have been expected. Economic health cost estimates are at an early stage, but are crucial to offset against costs of lockdowns. Current estimates of quality-adjusted-life-year plus simple direct healthcare cost savings for the firebreak compared to counterfactual are approximately GBP600,000,000 [ C9].

Our work was directly referenced in numerous press conferences, and in the Senedd by the First Minister of Wales, including during his announcement of measures on 15^th December: “ The modelling does demonstrate that if we were to have a position in which large numbers of people would come together for boxing day sales then that would be another form of mixing and whenever people get together the risk of coronavirus rises, we are not in a position in Wales today where we could knowingly allow those risks to take place… we will have to act to minimise those risks whenever we can. We will do that over the Christmas period through the joint advice that we will publish today” [ C10].

The overall impact timeline (Figure 2) of work and impact can be corroborated by Co-Chairs of TAC, and the Chief Scientific Advisor for Health in Wales [ C11].

*“An indisputable and substantial impact on determining Welsh Government policy and decision making at the highest level, supporting NHS health board and ambulance service capacity planning as well as advice around the implementation and release of interventions such as lockdown and the firebreak…the work they have done has gone on to directly influence national policy and decision making, making a real difference and ultimately saving lives”.*