Impact case study database

French’s emphasis on multi-disciplinary and participatory approaches to solving real problems and the innovative use of technology in supporting decision-making has helped politicians and policymakers, business leaders, public health officials, local authority and community representatives, and emergency planning officers in the UK and internationally.

COST (European Cooperation in Science and Technology) Action IS1304, 2013-19: French served on the management team of this EU-wide multidisciplinary network of scientists and policy makers to promote structured expert judgement (SEJ) to quantify uncertainty for evidence-based decisions, and improve the use of scientific expertise by policy makers. He led the development of training courses, co-ordinated two workshops and the final conference: around 50 ESRs were exposed to SEJ methods, and training materials are widely available through the Action’s website. Knowledge of SEJ methods was transferred to intermediary organisations, e.g., TNO ( www.tno.nl) in the Netherlands and VTT ( www.vttresearch.com) in Finland. The action was judged in the highest category of success. [5.1]

UK Government's Uncertainty Toolkit for AnalystsUK government departments, including the Food Standards Agency, the Medicines and Healthcare Products Regulatory Agency (MHRA), Department for Business, Energy & Industrial Strategy (BEIS), the Defence Science and Technology Laboratory (Dstl), the Nuclear Decommissioning Authority (NDA), OFGEM, Public Health England (PHE) and the Met Office have benefited from French’s research and expertise. His work across many areas has “provided real insight across government… and has resulted in several successful studies for government and industry which are informing policy and future direction,” says Professor Veronica Bowman, senior principal statistician at Dstl. [5.3] The basic tenets of his ideas [3.1,3.4,3.7] form part of a cross-government Uncertainty Toolkit for Analysts (published in early 2020, aimed at analysts across all departments and agencies) which sets out “ good, not best, practice, as analysis and communication must always be tailored to the audience and decision being made” [5.4]. Antony Bexon, head of radiation assessments at PHE, says that the toolkit “ fits well with a growing move across Government to acknowledge and address uncertainty more formally.” [5.5].

The AU4DM NetworkIn 2015, jointly with Mark Workman (Imperial College London), French co-founded Analysis under Uncertainty for Decision-Makers (AU4DM), a community of academics, policy makers and industry representatives seeking to develop best practice around analysis for decision-making under uncertainty. Through workshops and meetings, and novel tools such as the Visualisation of Uncertainty catalogue as well as other online resources, AU4DM has become a national leader in this field: organisations testifying to the practical value of the network include Anglian Water, "The network differs from other initiatives that seek to promote better uncertainty handling in decision making in that it listens to user needs instead of simply explaining academic modelling tools… AU4DM organises two or three excellent professional development events based around the exploration of realistic scenarios annually” [5.9]; Network Rail, “The events hosted by AU4DM ran a number of training simulations using very realistic and probing scenarios to catalyse discussions about uncertainty. Short pop-up style presentations… allowed examples of complex decision-making under uncertainty and potential solutions to be shared between attendees representing a wide range of decision contexts” [5.10]; and Dstl, “the Decision Tools catalogue that he edited from input across the network has been incredibly successful in guiding users to research and tools and the visualisation catalogue he co-edited has provided similar benefits.” [5.3]. The visualisation catalogue is now going in to its second edition.

AU4DM has also had an international impact: Sunny Modhara (Network Rail), “The adoption of these ideas has attracted interest from a large number of other international railways, in particular their incorporation into industry guidance on decision making.” [5.10] The Decision Tools catalogue has helped shape Chapter 17 Decision Making Options for Managing Risk in the next IPCC Report (due 2021). French is a contributing author to section 17.3 and a peer reviewer of the entire Chapter. Professor David Viner, a director at Macquarie Capital and a coordinating lead author for Chapter 17, who met French through AU4DM, says: "He has provided many insights and recognises political sensitivities that arise from suggestions about how communities, regions and governments should think about risks and decisions… IPCC reports are influential across governments in facing up to the impacts of climate change. His contributions will help the next report continue to be so.” [5.11]

Uncertainty handling in nuclear accidentsFunded by the UK's Atmospheric Dispersion Modelling Liaison Committee (ADMLC), in 2014 French led a project with the Met Office and PHE to improve the presentation and treatment of uncertainties during a nuclear accident to the UK's Scientific Advisory Group for Emergencies (SAGE). To SAGE's general approach which considers a single reasonable worst-case prediction of the course of the accident, he added a ‘novel’ second approach [3.3,3.6,3.7] which used several scenarios including one or more reasonable worst cases as well as ones with lesser impacts [5.5]. Matthew Hort, ADMLC Chair, says that the project “facilitated in building an improved, shared understanding and realistic expectations between decision-makers, scientists and communicators of what will be known in the early phase of a radiological emergency and how this knowledge… will evolve.” The project led to discussions with Robin Grimes, the FCO’s Chief Scientific Adviser, on how this work can be used to “inform and improve the current practices of presenting uncertain information in radiological emergencies, and thus improve the provision of health protection advice." [5.6].

French's work on the EU-funded CONFIDENCE project [3.9] on categorising and presenting uncertainty has helped "shape the understanding of the international community regarding the level of confidence that might be gained through the application of process-based models in assessment of human exposure to radiation, and in particular, their possible use in communicating risk and gaining trust of the public and other stakeholders following a nuclear or radiological emergency." [5.7] The Slovak Republic project member, Tatiana Duranova, who was trained by French in workshop facilitation, says: "Simon’s contributions in decision analysis and uncertainty handling influenced the process of the development of radiation protection culture in Slovakia and so made it possible for stakeholders to be knowledgeable, active, open and democratic and better prepared for decision making in case of possible nuclear accident when uncertainty handling is a key issue.” [5.8]. Less tangibly but possibly more effectively, the work is shaping thinking and practice within NERIS, a European network of agencies, communities around nuclear sites, regulators and research institutes that promote emergency preparedness and share good practice. Public Health England’s Bexon says: “Our thinking on uncertainties and the use of scenarios to convey complex and deep uncertainties has a route to wider application than just the UK.” [5.5]

Decision analysis in remediation of radiation-contaminated sitesFrench was invited to join the International Atomic Energy Agency's (IAEA) programmes on Modelling and Data for Radiological Impact Assessments (MODARIA I and II, 2012-2019) in the working group on site remediation. With the NDA's permission, he contributed a 2011 report on the Geological Disposal Facility (GDF) for high-level radioactive waste in the UK, along with many other inputs from the perspective of decision analysis. The output was finalised in December 2019. These reports "will provide important information and tools for use by Member States in the development of pragmatic remediation strategies, and the subsequent planning and implementation of remediation…(to) ultimately ensure and demonstrate protection and safety of people and the environment to the benefit of current and future generations." [5.7].

Probability of Black StartIn 2017, French suggested to the UK's Acting Chief Scientist, Professor Chris Whitty, that SEJ methods [3.8] could assess the risk of a complete failure (Black Start) of the National Grid, which would result in significant disruption to businesses and households. To minimise the impact of this risk, BEIS considered the development of a legal obligation on electricity companies setting a timeframe within which restoration should be achieved. BEIS required a robust likelihood assessment of total electricity failure; French was a leader of the project in 2019, and "(t)he output from this SEJ has fed into the cost-benefit analysis that underpins the proposed legal obligation." [5.12].

The background: Exit polls at UK General Elections since 2005

The major UK broadcasters fund and organise an election-day exit poll, in order to inform their coverage (on TV, radio and internet) on the night of the UK General Election, a time when current-affairs broadcasting attracts large audiences.  For the election in 2005, David Firth as consultant for BBC developed (with John Curtice) a completely new method [3.1] for the exit poll, to improve the accuracy of the headline prediction made at 10pm on election night.  The method proved so successful in 2005 — with an exact prediction of the 66-seat Labour majority — that it was adopted jointly by all of BBC, ITV and Sky for the following election in 2010.  After a similar success in 2010, the method has been used jointly by BBC, ITV and Sky at all subsequent general elections (2015, 2017 and 2019).  Corroborative statement [5.1], written after the 2010 election by the then Editor of BBC Political Research, confirms the pre-2015 background and emphasises the importance of the new method to the broadcasters. By 2010 Firth had moved on to other projects, and he passed the operation of the exit-poll method and associated software to Jouni Kuha (LSE), who replaced Firth as the statistician in John Curtice's election-day team at every General Election from 2010 to date.

Fresh impact in 2015–2019

The impacts described below are of two main kinds:

1. Impact of the underpinning research in conveying timely and accurate information to a large worldwide audience, via conventional media (TV, radio, press) and the internet. While this kind of impact was previously generated in connection with the General Election of 2010; the impact between 2015 and 2019 is around three times as big since it relates to three further elections.

2. Additional impact via the public track-record of accuracy of the new exit-poll method since 2005, in the form of substantially increased trust in the exit poll and the 10pm election-night prediction that comes from it. By 2019 the 10pm exit-poll prediction was presented (by broadcasters and other media outlets) without the scepticism that had routinely accompanied such predictions previously (in 2005 and 2010 for example); and even politicians now typically accept the exit-poll prediction as a reliable indication of the election outcome.   Moreover, the world's financial markets also now exhibit essentially 100% trust in the accuracy of the exit poll (whereas previously, even as recently as 2010, the markets' response at 10pm to a surprising but ultimately accurate exit poll was rather more muted because exit polls back then were not trusted to be accurate).

The Curtice-Firth methodology developed in the underpinning research [3.1] was used unchanged, by a team of academics working jointly for the three major broadcasters (BBC, ITV, Sky), to design and analyse the exit polls for the three General Elections in 2015, 2017 and 2019. The corroborative statement in [5.2], written in 2020 by Professor Jouni Kuha who is the statistician in the exit-poll team led by Sir John Curtice, fully confirms this. Kuha states straightforwardly: "The methodology of the exit poll in all of these years has been the one which was developed by David Firth and John Curtice for the 2005 election, as described in Curtice and Firth (JRSSA, 2008)." Kuha's statement letter also comments on the remarkable accuracy of exit-poll predictions in the five elections between 2005 and 2019, and states that "This success would … have been impossible without the current exit poll methodology."  The article in [5.2] by Curtice, Kuha and other exit-poll team members, published after the 2017 election, gives a more detailed account.

The broadcasters (BBC, ITV and Sky) disseminate their exit-poll prediction to the whole world at 10pm on election day via TV, radio and the internet.  In the UK alone, the number of BBC1 viewers at 10pm on each election night exceeds 6,000,000, according to reports [5.3] on the 2015, 2017 and 2019 elections from the BBC and Press Gazette; the corresponding number of viewers on ITV and Sky is reported to be 1–2 million.  The exit poll's global reach via the internet is not measured but is very substantial: the exit-poll prediction is picked up immediately by news media across the world, and by global financial markets.  The background letter [5.1] indicates the high value that the BBC and other broadcasters attach to the exit poll.  According to paragraph 29 of the 2018 report of the House of Lords Select Committee on Political Polling and Digital Media (in **[5.3]**), the cost of running the exit poll at a General Election is around GBP300,000 — a figure which further corroborates the high value that the broadcasters attach to the exit poll.

Interest in the exit poll from the general public is strong (as indicated by the quoted audience figures, for example), and there is evidence that the public at large now appreciates that modern-day ("John Curtice") exit polls are much more reliable than pre-election polls.  Evidence [5.4] collects some representative social-media posts from members of the public in December 2019, e.g., " There is only one poll to trust and it's the exit poll".  From these social media posts, it seems also that the exit poll influences individual behaviour, for example the decision to stay up on election night: typical tweets include " No point - exit poll is always right", " I didn't sleep a wink on referendum night. Wouldn't have missed it for the world. I don't bother on General Election nights though. I trust John Curtice enough to watch the exit poll and get off to bed" and " JC's exit poll is normally bang on, at 10pm tomorrow I'll know whether to go to bed early".

To support wider understanding of the exit poll and its methods, David Firth created (in 2010) the public-facing "Exit Poll Explainer" page at warwick.ac.uk/exitpolling — which sees thousands of visits around the time of an election (access statistics [5.5] show more than 15,000 distinct visitors in the week of the 2017 election, for example).  Sir David Spiegelhalter, then President of the Royal Statistical Society, highlighted the Explainer page along with this public comment [5.6] immediately after the 2017 election: " Clear outright winners of this election (and others): statisticians who designed the exit poll and its model".  In his 2019 popular book The Art of Statistics, Spiegelhalter even highlighted Firth's work on exit-poll methodology as the book's final chapter " An Example of Good Statistical Science".

In all three of the most recent elections (2015, 2017, 2019) the exit poll's prediction was markedly different from what had been expected from pre-election polls (as indeed it had been also in 2010 and 2005).  Whereas in 2010 the surprising exit poll was downplayed by TV broadcasters and on-air pundits (as noted also in **[5.1]**), by 2017 its track-record of accuracy had forced the commentators and politicians to accept the 10pm exit poll as a reliable prediction of the election outcome.  Evidence [5.7] collects some media items which show this.  Even in 2015, former LibDem leader Paddy Ashdown rubbished the exit poll in his interview on BBC1 and famously said he would eat his hat on TV if his party lost the large number of seats that the exit poll had predicted (their actual loss was two seats more than the prediction).  By 2017, though, the tone had changed markedly towards acceptance of the exit poll's accuracy.  In her BBC interview a few weeks after the 2017 election, PM Theresa May admits to having been "shocked...devastated" upon hearing the exit poll that signalled the loss of her House of Commons majority.  By 2019 we even see John McDonnell, Labour's shadow chancellor, conceding the election on BBC1 just 15 minutes after the 10pm exit poll has been announced.  Whereas in 2015 The Independent still asks " Just how did the exit poll manage to get it so right?", in 2019 The Economist reports simply that " Britain's exit poll has an exceptional record".

In the peer-reviewed paper [5.8], written after the 2017 election and published in one of the most prominent international journals of political science, the evidence just described is solidified through formal text-analysis of election-night broadcasts across UK general elections since 1955.  Paper [5.8] provides many new insights, and a key conclusion is that " The remarkable accuracy of the 2010 exit poll, and the subsequent success of the 2015 and 2017 exit polls, have been associated with a growing focus on them in the early part of the broadcasts."

Evidence [5.9] collects clippings about the exit poll in relation to financial markets, clippings which show the markets' increased trust in the 10pm exit poll prediction. In 2017 a CNN Money article stated: "The exit poll has typically been much more accurate than Britain's notoriously shoddy opinion polls".  A 2019 pre-election article from Pound Sterling Live quotes Samuel Tombs (Chief U.K. Economist at Pantheon Macroeconomics): "Abandon all your priors when the UK exit poll is released at 22:00 — it has a very good track record since its current methodology was introduced in 2005".  A Financial Times online article published shortly before close of polls in 2019 assesses the increase in trust directly, and states: "The exit poll's recent run of prescient results has meant that traders are putting increased stock in its predictive power, with the pound fluctuating more aggressively following the poll's release from one election to the next."  The sequence of graphs shown below amplifies the message of that Financial Times article; it shows the GBP-USD exchange rate, through election nights 2010, 2015, 2017 and 2019 (using data from forexite.com).  In each panel the red line is the average price in the half-hour before the 10pm close of polls, while the green line is the average for the half-hour after 10pm.

In 2010 the 10pm fall in Sterling was only a relatively small fraction of what the currency would ultimately lose by the next morning (even though the surprising exit poll was highly accurate): just like the political pundits, the market did not really trust the exit poll by 2010.  As the accuracy of exit polls became more firmly established, though, the market's trust increased.  By 2017 the market's 10pm reaction to a surprising (but ultimately accurate) exit poll was as big as its full response to the actual result of the election the next morning.

John Curtice — who is by now very much the enduring public face of election polls in the UK — received a knighthood in 2018.  David Firth's underpinning technical contribution to the accuracy of modern exit polls has been recognised in public less formally, through mentions in media articles and occasional interviews for BBC TV and radio.  Ahead of the election in 2017 the BBC acknowledged David Firth's exit-poll work through a 1-minute feature video [5.10] which was broadcast via BBC TV news bulletins in the UK and syndicated to global media via the internet.

Over GBP200,000,000 per year is spent in the UK on vaccines and vaccine delivery, and infectious disease and health economic models are used to assess whether any change in the immunisation programme is cost-effective. The UK Department of Health and Social Care Joint Committee on Vaccination and Immunisation (JCVI) advises UK health departments on immunisation. The JCVI recommends a vaccination policy only if it is considered both safe and cost-effective. HPV is the world’s most common sexually transmitted infection and is strongly associated with the development of cervical cancer in women and other health conditions in both sexes including anal, oropharyngeal, mouth, and throat cancers, and genital warts. In 2008 the UK introduced bi-valent HPV vaccination of girls to safeguard against cervical cancer, changing to a quadra-valent vaccine in 2011 to provide additional protection for genital warts. On the basis of both gender equality and the risk of disease in males, there was considerable social and political pressure to introduce HPV vaccination of boys. Gender-neutral vaccination against HPV was discussed with the committee at a number of JCVI meetings from 2014 onwards.

As a member of the JCVI from 2010, with substantial experience modelling the spread and control of many infectious diseases, and with input from a Patient and Public Involvement (PPI) panel established at Warwick to advise on model assumptions and other details of research, Keeling was uniquely placed to provide expertise and evidence to the dialogue surrounding HPV vaccination. Keeling presented Warwick’s modelling on HPV to the JCVI HPV subcommittee in February 2016, January 2017 and May 2018. Commissioned work from the JCVI on HPV was originally intended to form a second opinion to parallel work undertaken by researchers at Public Health England (PHE) [5.1].

The predictions from the University of Warwick were the main quantitative results used to assess the cost-effectiveness; Keeling and co-workers determined that at 1.5% discounting adding boys to the programme would be cost effective if the vaccine could be procured for less than £36-47. From the meeting minutes: "The subcommittee agreed that it did not need to go further with the PHE model and that it could reach a decision based on the Warwick modelling work, which was robust, and the findings made so far by PHE alongside the meta-analysis of published models by Brisson et al." The Warwick model therefore became the primary model used in the HPV vaccination policy decision making process [5.1].

The JCVI Statement on HPV vaccination in July 2018 recommended extending the HPV vaccination programme to include adolescent boys in the UK for the first time. Their recommendation was based on the findings from the Warwick model: "Using a 1.5% discount rate it is likely that a gender neutral programme would be cost-effective, and on the basis of these findings JCVI would advise extending immunisation to adolescent boys" [5.2]. The statement contributed to the Republic of Ireland’s Health Information and Quality Authority health technology assessment (HTA) on extending their own HPV vaccination to boys in December 2018 [5.3]. In spite of the relatively recent release of the JCVI statement, this is a clear indicator that the analysis is already informing cost-effectiveness modelling and the wider international debate on gender-neutral HPV vaccination.

Concerning this move by the JCVI Shirley Cramer CBE, Chief Executive of Royal Society for Public Health, stated, "The JCVI's decision to advocate for a gender neutral vaccination programme against HPV is a victory for the public's health. Boys have been left insufficiently protected against HPV for too long and it is good news that the UK is following in the footsteps of the other 20 countries already vaccinating boys against HPV" [5.4].

Following the JCVI recommendation, from September 2019 Public Health England (PHE) extended the HPV vaccination programme to adolescent boys [5.5], with similar announcements made for Scotland, Wales and Northern Ireland. The extension protects men who have sex with men because they do not benefit from the herd effects of female vaccination. News of the HPV vaccination programme was referenced over 100 times in the media across 6 countries. For UK media outlets covered, a readership of approximately 26,000,000 was estimated from statistics by PAMCo.

Between September 2019 and March 2020, gender-neutral HPV vaccination started in 83% of the 150 local authorities in England, with 5% of these fully completing the programme before school closures due to the COVID-19 pandemic in March 2020 which led to a temporary pause in the campaign [5.6]. The JCVI subsequently re-affirmed the ongoing importance of the campaign when it restarted in June 2020, issuing a statement in July 2020 prioritising “the first HPV vaccine dose for all eligible children and the catch up of those who missed the first dose” [5.7]. As a result, the entirety of NHS England and NHS Improvement (NHSEI) commissioned school aged providers were asked to implement HPV vaccination restoration and recovery plans, with some catch-up having been carried out from all providers [5.6].

Despite these difficult circumstances, the gender-neutral HPV vaccination programme succeeded in covering an impressive 54.4% of year 8 boys in England with the recommended priming dose [5.6]. This represents approximately 170,000 12- to 13-year-old boys based on Department of Education statistics [5.8]. This is in addition to 59.2% of year 8 girls who were also vaccinated (approximately 177,500), and 64.7% of year 9 girls completing a 2-dose HPV schedule (approximately 188,000) [5.6, 5.8]. These values all represent minimum coverage, as data from at least 19 local authorities out of the 66 total NHSEI-provider respondents only captured values up to March 2020 without subsequent data from programme restoration beginning July 2020. A drop in coverage for girls based on 80% coverage in previous years was reasonably concluded to be “ mainly due to the impact of the COVID-19 pandemic” [5.6] .

The public health benefits of HPV vaccination have been quantified in a recent independent meta-analysis that reviewed 65 studies covering 60,000,000 people after 8 years following initial vaccination for both bi-valent and quadra-valent girls-only vaccination campaigns [5.9]. It was found that cervical intraepithelial neoplasia decreased by 51% and 31% for girls 15-19 and women 20-24 respectively, and for the quadra-valent vaccine genital warts cases decreased 67%, 54% and 48% in girls 15-19, women 20-24 and boys 15-19 respectively. Extending vaccination to include boys will further improve these benefits. The switch to a gender-neutral national policy on vaccination against HPV in the UK is predicted over the next 50 years to lead to the prevention of approximately:

• 5800 cases of oropharyngeal cancer,

• 4000 cases of cervical cancer,

• 2000 cases of anal cancer,

• 1000 cases of penile cancer,

• 25,000 cases of cervical intraepithelial neoplasia, and

• 230,000 cases of genital warts.

This is in addition to the 60,000 cases of cervical and 40,000 other cancers prevented by the previous policy of vaccination of girls only.

Several statements from leading public health figures and politicians help quantify the magnitude of the gender-neutral vaccination programme [5.5]. Head of Immunisation at PHE Dr Mary Ramsay stated, "This universal programme offers us the opportunity to make HPV-related diseases a thing of the past and build on the success of the girls' programme." Public Health Minister Seema Kennedy added, "The success of the HPV vaccine programme for girls is clear and by extending it to boys we will go a step further to help us prevent more cases of HPV-related cancer every year. Through our world-leading vaccination programme, we have already saved millions of lives and prevented countless cases of terrible diseases. Experts predict that we could be on our way toward eliminating cervical cancer for good."

The scale of the challenge linked to neglected tropical diseases (NTDs) remains staggering: over 1,000,000,000 people remain at risk from around 10 NTDs (London Declaration on Neglected Tropical Diseases, 2012), yet to date only a single human transmissible disease – smallpox – has been eradicated by public health measures. Since 2015 Warwick’s NTD modelling research has had an impact on policy formulation and implementation of gHAT elimination strategies in the DRC and Chad, two of the worst-affected countries in the world, saving or improving the quality of an approximate 740,000 lives. In the case of LF, work at Warwick was implemented in the revised 2017 WHO guidelines, which promotes mass drug administration (MDA) via a triple drug regimen. This guideline has resulted in 45,200,000 people across 11 countries being treated with triple drug therapy in 2019.

Impact on national governments: optimal strategies for gHAT elimination in DRC and Chad

Public health interventions against gHAT have been very successful: the annual number of cases fell from 25,841 in 2000 to 6,973 in 2010, and to 864 in 2019 [5.1]. Given this impressive progress, the WHO moved from a target of elimination of gHAT as a public health problem by 2020 to a target of elimination of transmission by 2030 [5.1].

Because transmission is not directly observable from case and screening data, it must be inferred by combining this data with an appropriate model [3.2-3.4]. Since 2015, Rock has worked directly with the responsible public health agencies in Uganda, Cote d’Ivoire, Guinea, DRC and Chad to allow them to quantitatively gauge the impact of existing interventions on gHAT transmission. Of note, productive partnerships began in 2015 with the respective health ministries’ Programme National de Lutte contre la Trypanosomiase Humaine Africaine (PNLTHA) in DRC and Chad, two of the countries worst-affected by gHAT.

DRC: PNLTHA is a specialised programme set up in the early 1990s to organise public health approaches to the disease control throughout the country. Its integrated approach includes active screening and treatment; vector control (reduction of human-tsetse fly contact); and epidemiologic surveillance, training and communication at community level.

The University of Warwick’s modelling examined what combination of medical interventions (active and passive screening, as well as vector control) in the DRC would be required to meet elimination of transmission by 2030 in 168 health zones (each with an average population of 100,000) [3.3, 5.2]. The findings revealed six new priority health zones where large-scale vector control would be needed (Bagata, Bandundu, Bolobo, Kikongo, Kwamouth and Masi Manimba in former Bandundu province), in addition to continuing existing vector control in the Yasa Bonga health zone, in order to meet the elimination of transmission target. This contributed to PNLTHA’s policy review in 2018 and the decision to implement scaled-up vector control between 2019 and 2022 [5.2]. These 7 health zones have some of the highest levels of gHAT in the world [5.2], with an approximate total at-risk population of 700,000.

Vector control interventions for gHAT use ‘Tiny Target’ technology – a small (50x25cm), low cost (<$1), eco-friendly insecticide-impregnated panel of blue cloth that attracts tsetse and is planted using stakes near a body of water. In 2014 there were no such large-scale interventions in DRC, however numbers of Tiny Targets were intensified to their highest ever levels in 2020 in accordance with Warwick’s modelling, with 43,000 deployments made per year spanning an area of 11,700km². Additionally, these deployments are planned to increase to 80,000 by 2022 [5.2]. The Director of PNLTHA-DRC says: “ The modelling results from [Dr Rock] and the HAT MEPP team have played a decisive role in guiding and verifying planning localised interventions against gHAT across the DRC and our confidence in achieving the elimination goal of 2030. We hope to continue this precious collaboration… and be guided by other results” [5.2].

Chad: Vector control was first implemented in 2014 in the swampy Mandoul region, which has an approximate population of 40,000, and where most gHAT cases occurred between 2000 and 2013. Adaptation of the Warwick model to Mandoul made it possible to quantify the effect of interventions (both improved passive screening and vector control). Modelling suggested that these combined measures led to a 72% reduction in transmission. Continuing implementation of these interventions saw cases gradually reduce from 715 in 2002 to fewer than 20 cases reported in 2018 and 2019 [5.3]

The Warwick modelling for Chad provided important validation that the approach in Mandoul was an optimal strategy [5.3]. The Director of PNLTHA-Chad highlighted the significance of such verification: " *Cost effective public health interventions are essential in all countries due to the huge demand for medical resources, which is especially important in a low-income country like Chad.*” With regards to meeting the 2030 elimination of transmission target for gHAT, the findings demonstrated it was " essential to continue interventions every year in order to lead our country on the path of elimination" [5.3].

Continued BMGF funding: In a 2018 review meeting, ‘Progress and Plans for Elimination and Eradication Programs for HAT and Guinea Worm’, Rock met with Bill Gates and his team to present and discuss the HAT modelling work [5.1]. In 2020 Rock secured further grant funding, ‘Modelling for the HAT Endgame’ [G4], with BMGF stating this was to “ assist in finding the most efficient strategy to success" for gHAT elimination, in the context of the WHO 2030 elimination target [5.1].

Impact on international organisations: revised WHO guidelines on LF elimination

On the basis of her high-profile modelling work through the NTD Consortium [3.1], in January 2017 Hollingsworth joined a guideline development group convened by WHO to re-examine its guidelines pertaining to LF elimination. Prior to 2017, mass drug administration (MDA) with double drug therapy of diethylcarbamazine and albendazole (DA) was recommended by the WHO as the preferred approach to LF elimination in endemic areas. In 2017, this group produced revised guidelines [5.4] that incorporated Hollingsworth’s findings in recommending a switch to the ‘IDA’ triple drug therapy (ivermectin, trade name Mectizan, with diethylcarbamazine and albendazole). Warwick modelling allowed for the quantification of the number of annual triple therapy rounds required to meet elimination targets [3.6], an assumption that is fundamental in evaluating treatment cost and hence the overall viability of the proposed regimen.

In response to the new WHO guidelines, the global pharmaceutical giant Merck Sharp & Dohme committed to donating an additional 100 million Mectizan (ivermectin) treatments annually up to 2025 to support the elimination of LF by triple-drug therapy [5.5]. In 2019, 201,000,000 treatments for LF were approved by Merck's Mectizan Donation Program (MDP), allowing the triple drug IDA programme to extend its coverage into 13 new countries and territories: American Samoa, Egypt, Fiji, Guyana, India, Kenya, Madagascar, Malaysia, Papua New Guinea, Samoa, Sao Tome & Principe, Timor Leste and Tuvalu [5.5]. In MDP’s 2019 Annual Highlights, Program Director Dr Yao Sodahlon writes in his message: “ As of December 2019, the number of people no longer needing treatment with Mectizan and albendazole for elimination of LF as a public health problem increased to 163.3 million [163,300,000], from 150.9 million [150,900,000] the previous year… We will continue working [with our partners] to sustain high Mectizan treatment coverage and ensure that no one is left behind – regardless of who they are and where they live” [5.6].

In October 2020 WHO revealed that in 2019 45,200,000 people were treated with the triple-drug IDA regiment across 11 countries based on the revised guidelines [5.7]. This represents around 6 in every 1,000 people (based on the world’s total population of 7.7 billion in 2019).

Impact on Government policy: informing USDA’s strategic guidelines and operating procedures

In the US, outbreak preparedness and response strategy, including vaccine prioritisation, is the responsibility of USDA’s Animal and Plant Health Inspection Service (APHIS). As recognised experts in infectious disease modelling, Tildesley and Keeling have collaborated with the USDA and US Department of Homeland Security for over 10 years to inform US policy in this area [5.1]. The [text removed for publication] USDA Centre for Epidemiology and Animal Health, calls this collaboration “ an excellent example of an effective Federal-Academic partnership.” She continues: “ The transfer of knowledge from researchers in the UK such as Dr Tildesley has improved the ability to plan for and respond to agricultural diseases in the United States.” [5.1]

Both USDOS and USAMM early model versions were first featured as part of the USDA’s Foreign Animal Disease Preparedness and Response Plan (FAD PReP) in 2014, following the development of USDOS at Warwick in 2013 [3.1]. For the plan, both models were incorporated in the Overview of Modeling and Assessment Tools standard operating procedure (SOP), with SOPs constituting formalised step-by-step guidance to be followed under specific circumstances [5.2a]. In this context, this overall series of SOPs provides the USDA – one of the USA’s federal executive departments – with operational details ‘for conducting critical activities, such as communication and biosecurity, that are essential to effective preparedness and response to an FAD outbreak’ [5.2b].

Series of SOPs are often incorporated into larger strategic documents, and this is no different for the USDA. First published in 2014 and reissued in 2020, the FAD PReP Foot-and-Mouth Disease (FMD) Response Plan: The Red Book provides the holistic approach taken by US government officials in response to an FMD or other foreign animal disease outbreak. The Red Book acknowledges the importance of mathematical models as part of this approach, mandating that the USDA should bring together “ epidemiologists, disease agent experts, economists, and affected commodity experts to undertake risk assessments to inform policy makers as required” [5.3]. Further, the FAD PReP series of SOPs are featured throughout the 2020 Red Book, with the 2014-published Overview of Modeling and Assessment Tools SOP spotlighted as outlining appropriate modelling contingencies, including its provision of “ current models and assessment tools” which encompasses USDOS and USAMM [5.3].

Overall, the USDOS and USAMM models have remained relevant and useful to US agricultural policy from their initial integration into the still active 2014 SOP [5.2a], through further refinement in 2016/2017, and finally with the models being officially transferred to the USDA [5.1] in 2019 for use in the prediction of future disease risk through livestock movements. The [text removed for publication] USDA Centre for Epidemiology and Animal Health says: “ *Use of [the USAMM and USDOS] models provide information for various analyses important to mitigating the risk of disease spread within the US livestock industry and have been applied to support the Federal Emergency Management Agency Food, Agriculture, and Veterinary response exercise in collaboration with USDA.*” [5.1]. The models were extended in 2018 to simulate bovine tuberculosis outbreaks.

The 2020 version of the Red Book reflects knowledge gained and lessons learned during APHIS’s testing of its response capabilities with FAD outbreaks in 2016, 2017, 2018 and 2019, as well as the planning and execution of the 2018 ARMAR (Agriculture Response Management and Resources) foot-and-mouth disease (FMD) functional exercise (see Impact on response planning, below). Among other things, compared to prior versions: it includes policy that explicitly recognises vaccination as a likely response tool in an FMD outbreak, as well as new surveillance sections revised by the Center for Epidemiology and Animal Health; it highlights available training resources and often overlooked planning considerations; and it incorporates policy guidance prepared for ARMAR on managing a national movement standstill [5.3].

Impact on response planning: highlighting the importance of a national-scale analysis

USDOS gives the USDA the capacity to determine the optimal control policy that should be introduced dependent upon where the disease might start and how it might spread in the early stages of an outbreak. Run on a laptop or supercomputer, USDOS efficiently simulates outbreaks by local (aerosol) and long-range transmission (animal movement) [5.4]. USDOS can investigate the impact of a suite of movement restrictions, ring culling, vaccination and diagnostic testing strategies. Warwick’s novel real-time modelling approach for FMD is able to use the most up-to-date data during an ongoing outbreak to advise policy makers on the most effective intervention strategies when there is significant uncertainty during the early stages of an outbreak, and adapt policies as more is learned about epidemic behaviour. Shortening the duration of an epidemic could mean significantly lower economic losses.

Warwick and colleagues’ real-time modelling of disease outbreaks contributed to the development of an app, USAMM-Shiny App, to make it easier for public officials and livestock farmers to predict cattle shipments and prepare for disease outbreaks [5.5a]. The interactive app allows users to create heat maps to visualise the movement of cattle across the US: the visualisation of movement data can help to inform risk assessments, develop scenarios and trace back efforts to predict where cattle shipments may occur in the United States using the USAMM model [5.5b]. This information allows the USDA to determine high-risk counties for disease introduction; it also underpins the outbreak simulation capability provided by USDOS, as shipment-based disease spread is a required input into the model and there are no complete records of animal shipments in the US to otherwise rely on.

Disease preparation exercises remain a core activity of APHIS, which sponsors or participates in an average of 3 or 4 such exercises annually (41 exercises over 11 years) for FMD [5.6]. Prior to the transition of the models into the USDA, governmental uptake of the USAMM model was demonstrated by APHIS and other collaborators in a 2017 assessment exercise [5.7]. This marks an important use-case for USAMM which the USDA actively participated in, highlighting the integration of real-world data on cattle imports with the model priorities for surveillance of specific state counties.

In 2019, the US Government Accountability Office (GAO) was asked by Congress to review USDA’s efforts to prepare for an FMD outbreak. The resulting report [5.6] recommends that USDA follow its procedures to prioritise and monitor the completion of corrective actions that the agency had previously identified for FMD preparedness. Efficient allocation of limited vaccine resources in the event of an outbreak was identified as an area of concern. The GAO notes that APHIS has “ used predictive models to evaluate different vaccination strategies” at the state level [5.6] and recommends that “ a similar analysis at the national level … could help inform USDA’s vaccine prioritization decisions in advance of an outbreak”. Such national-scale analyses are only now possible due to the novel capabilities provided by the USDOS/USAMM models.

Although there has not been an outbreak of FMD in the US for almost 100 years, a constant risk of reintroduction remains. The importance of the capability provided by the USAMM/USDOS model is reflected by the fact that the country has over 882,000 cattle farms, with over 93,600,000 beef and dairy animals. According to the GAO, “ an FMD outbreak would have serious economic impacts, in part because trade partners would likely halt all imports of U.S. livestock and livestock products until the disease was eradicated. These imports were valued at more than $19 billion [USD19,000,000,000] in 2017” [5.6]. This is expanded on further in the 2020 Red Book: “ A U.S. outbreak contained to California could cost $6–14 billion; a nation-wide agroterrorism attack could reach $228 billion [USD228,000,000,000]” [5.3]. These stark figures reflect the importance of effective contingency planning for US agriculture in the face of an outbreak: the need to reduce the risk of large-scale transmission through national-scale modelling cannot be overstated.

The main pathway to impact for this research is through membership of the advisory bodies directly responsible for providing scientific input into government policy. Keeling, Dyson, Hill and Tildesley are active members of the SPI-M modelling group which provides scientific advice to SAGE and directly to the Department of Health. Keeling and Tildesley also sit on the Interdisciplinary Task and Finish Group on the Role of Children in Transmission (TFC), an interdisciplinary body of SPI-M, and the Scientific Pandemic Influenza Group on Behaviours (SPI-B). Keeling also sits on the JCVI.

In the year 2020, the Warwick group have authored or co-authored 57 modelling reports to SPI-M, with 19 being sent onto SAGE as a result. The majority of these reports are publicly available alongside the published minutes of SAGE meetings. However, in keeping with civil service norms, the authorship of the papers is not stated. The contributions of the Warwick group to the relevant reports are confirmed in a statement from the UK Government Chief Scientific Adviser [5.1].

Estimation of the R number: The R number is the crude yardstick against which the effectiveness of the public health response to the epidemic is measured. The Warwick model is one of 6 models from 7 UK institutions that have been used throughout the pandemic to form SPI-M’s weekly combined estimate of the R number for 10 UK regions, together with short and medium term forecasts of the progress of the epidemic [5.1, 5.2]. Credible estimates of the R number were critical in the timing and sequencing of the relaxation of social distancing measures after the first lockdown. The need demonstrated in [3.6] for a cautious exit from lockdown to avoid a second wave, including the potential for reintroduction of local lockdown measures where ICU occupancy exceeded a given threshold, was largely included in the recommendations [5.3] of the 33^rd SAGE meeting on 5 May and subsequently adopted by the Government when exiting the first lockdown. Local lockdowns were later introduced in Leicester on 17 July and in the North of England on 31 July on the basis of these recommendations.

Specification of Reasonable Worst-Case Scenario: On 10 March, the 14^th SAGE meeting recommended that, for planning purposes, it is not useful to produce a "most likely" scenario given the lack of available data. The Reasonable Worst-Case Scenario (RCWS) was recommended as being the most useful scenario for government planning. Warwick and three other modelling groups subsequently developed several RWCS, which provided similar results once normalised. When the RWCS was reviewed in May at the request of the government, the Warwick model was selected by SPI-M at the 38^th SAGE meeting on May 21 as the single model to provide the consensus [5.4]. Keeling was subsequently asked to explain the rationale for the RWCS in oral evidence to the House of Lords Select Committee on Science and Technology on 2 June [5.5].

Protocols for reopening of schools in England after the first lockdown: The findings of [3.7] were incorporated, along with findings from 3 other models, into a paper [5.6] presented at the April 30^th SAGE meeting by the TFC. This paper covered 9 separate scenarios from minimal attendance of schools by the children of key workers to full attendance. The main point of consideration was the finding from the Warwick model, supported by other modelling groups, that “the impact of opening schools is a lot less than any changes to the population-wide policy of lockdowns”. A follow-up SAGE meeting was arranged the next day to fully evaluate the detailed findings. At this meeting, the consensus ranking on the 9 different scenarios was supported by SAGE – with precautionary measures advised ahead of re-opening schools [5.7]. In June 2020, a phased reopening of schools was announced, focusing initially on early years and primary school settings and smaller class sizes in line with modelling recommendations.

JCVI vaccine prioritisation list: The Chief Scientific Advisor confirmed the Warwick researchers’ contribution to scientific prediction of the impact of vaccination presented to JCVI (Joint Committee on Vaccines and Immunisation), SPI-M and SAGE, stating: “The early elements of this research considered the benefits of targeted vaccination and helped inform the JCVI priority list, while later models have focused on how non-pharmaceutical interventions can be relaxed as the vaccine is rolled out across the population.’’ The mathematical modelling in [3.8] indicated that the optimal strategy for minimising future deaths or quality adjusted life year (QALY) losses is to offer vaccination to older age groups first. These findings were subsequently cited by the JCVI on 30 December as providing the scientific rationale for the committee’s recommendation that vaccine priority groups should be ranked primarily by age [5.8]. This is reflected in the implementation of the Covid-19 vaccination programme.

Introduction of social bubbles as a soft exit from lockdown: The findings of Warwick’s modelling demonstrating the relative safety of social bubbles [3.9] were included in a report presented to the 36^th SAGE meeting on 13 May [5.9]. All models indicated low risks of increase in R from bubbles involving single-occupancy households and single parents with primary school age children. Although SAGE advised against the introduction of bubbles in the short term when other measures have just been lifted, on June 10^th the Prime Minister announced that single adult households who live alone or with dependent children would be permitted to form a ‘support bubble’ with one other household [5.10], a policy that remains in place today.

Circuit breakers in Wales and Northern Ireland: The results of [3.10] on circuit breakers were included in a paper presented to the 59^th SAGE meeting on 24 September [5.11]. SAGE noted the estimate of R of 1.2 to 1.5. would result over time in hospitalisations exceeding scenario planning levels unless interventions brought R back below 1. SAGE also advised that a two-week circuit breaker could achieve this. Although the government decided against a proposed circuit breaker in England over half-term, the policy was adopted by the Welsh and Northern Ireland governments who entered circuit breaker lockdowns on 23 October and 27 November respectively.

Hutton’s research has international economic and social impact by improving lawyers’ use of statistics and by underpinning expert witness reports, benefiting plaintiffs, defendants, companies and lawyers. Three examples of Hutton’s impact in litigation are highlighted, after a note on dissemination.

Hutton is the current Chair of the Royal Statistical Society (RSS) Statistics and the Law section. As Chair, she responded to a House of Lords Inquiry into Forensic Science and Forensic Science Regulatory concerns. She co-authored best practice guides for advocates and judges and gave 6 professional development seminars on statistical issues for barristers and judges since 2016 for, inter alia, American Association of Justice, Welsh Judicial Society and Inns of Court College of Advocates [5.1].

1. Gee & Others v DePuy International Limited [2018]

Statistical evidence prepared for the metal-on-metal hip trial Gee & Others v DePuy International Limited [2018] was key to the outcome that DePuy was not liable; the ruling is a prime example of what defines a defect under the Consumer Protection Act (CPA) 1987, and has since been cited.

This case was brought under the Consumer Protection Act (CPA – 1987). Plaintiffs claimed that DePuy Pinnacle MoM hip prostheses were defective, tending to cause adverse reaction to metal debris (ARMD), hence further surgery. Hutton's review of the research and registry information focused on whether the statistics were robust, and was submitted as evidence for DePuy's defence. Hutton’s key contributions [3.4] were assessing the potential effects of registry completeness and missing confounding factors, and how the low threshold for diagnosing ARMD might inflate failure rates.

Finding DePuy not liable, Justice Andrews stated “one cannot possibly conclude” there was a “materially greater risk of failure than the comparator” [5.2]. Any pay-out would have been a significant proportion of DePuy’s equity (£130m, DePuy International Limited 2018 Annual report). In contrast, USA, plaintiffs with Pinnacle hip plants were awarded over $1bn [5.3].

Hutton’s evidence directly informed the process for the legal team and interacted effectively with other experts' work. Her joint report provided a focused basis for cross-examination. She provided live responses to the Claimants’ experts' evidence under cross-examination [5.4].

The impact of Gee has already been extensive. The ruling is now an authoritative example of how medical defects are considered under CPA, “starkly different” from previous rulings [5.5]. The High Court and other Senior courts have endorsed this precedent on the standard of evidence required to prove causality for medical devices. [5.5].

Gee has also been cited as an exemplar product liability case in the textbooks and practitioner guides [5.6].

2. Allergy Therapeutics PLC (ATL) v Inflamax Research (IR) Settlement

Hutton’s expert reports were "invaluable to us and no doubt a factor in this settlement" between ATL and IR, worth $11.7m to ATL [5.8]).

Cooley LLP, acting for ATL, instructed Hutton in a claim against IR regarding the quality of a trial of hayfever treatments, which required exposure of participants to pollen concentration within a target range. However, pollen measurements were variable and often out of range. Hutton evaluated the opposing expert evidence, which claimed that failure to maintain correct pollen concentration did not affect outcomes. Drawing on her methodological research on the potential influence of missing data plus how averages of measurements on a person over time can hide important effects, Hutton prepared a joint report with the IR expert, and explained to ATL's barristers why the trial results were not reliable.

The settlement was key for ATL achieving profitability in 2019: “The strong operating performance and the settlement of the legal case with Inflamax led to a net profit of £3.5m” [5.7].

3. Informing international life expectancy settlements

Hutton’s reports made a difference of about £150m in 200 personal injury cases, out of total awards of £500m - £700m. Awards cover ongoing care costs for those with debilitating permanent conditions, impacting directly on quality of life.

Canada: “These cases and settlements are life changing for the clients, and an accurate life expectancy opinion is crucial to ensure that the child has enough funding to care for his or her lifelong needs. We have requested Hutton as an expert witness on 3 occasions due to her well-known expertise on life expectancy, including the nuances of the statistics in the area. She is very knowledgeable in understanding the debate over the various databases on life expectancy for children with cerebral palsy and talented in writing reports in this regard.”[5.9]

UK: “We have requested Hutton as an expert witness on six occasions due to the impeccable quality of her statistical evidence and reputation as a leading life expectancy expert witness.” [5.9].

Australia “We have requested Hutton as an expert witness on at least 15 occasions due to the deserved reputation as leading legal consultant for life expectancy in catastrophic injury cases. Overall in every instance that Slater & Gordon has worked with Hutton, she has provided a detailed analysis of great quality.”[5.9]

The range of injuries affecting survival include traumatic and acquired brain injuries (51 reports), spinal cord injuries (19) and cerebral palsy (70). Cases come from the UK (141 reports), Australia (34), Canada (7) and Eire (18). The few UK neurological injury life expectancy witnesses typically cite Hutton's research, as do the California-based Life Expectancy Project experts.

Regulatory toxicity testing using animals remains an important part of chemical hazard identification and characterisation across the world.  In the UK in 2019, 145,000 animal procedures were conducted for regulatory toxicity and safety testing with 14,000 of these used for acute toxicity assessment [5.1]. Warwick research has been fundamental in the development and refinement of the Fixed Concentration Procedure (FCP) for the assessment of acute inhalation toxicity leading to development and adoption of a new OECD test guideline (TG433) in June 2018 [5.2]. The implementation of this guideline results in a substantial reduction in the number of animals required for testing, in improved animal welfare and economic savings, and aligns the test with EU Directive 2010/63/EU which states: “ methods selected should avoid, as far as possible, death as an end-point due to the severe suffering experienced during the period before death”.

Development and adoption of a new OECD test guideline TG433

International guidelines on toxicity testing are approved by the Organisation for Economic Co-operation and Development (OECD) for its 36 member countries including UK and all EU founder countries, USA, Canada, Australia, New Zealand and Japan. As approval requires the agreement of all individual OECD member countries, overcoming this obstacle for any new guideline can be considered “ a major achievement” [5.3]. One area of toxicity testing is the testing of acute inhalation toxicity for dusts, gases and mists described by OECD Guidance document GD39, which referred to specific test guidelines TG403 and TG436.

For the FCP, there were three primary concerns by the Regulators which initially prevented approval: the lack of evidence for the comparable performance of the FCP method with existing OECD guidelines TG403 and TG436, the impact of suspected sex differences in the level of toxic effects, and poorly defined nature of evident toxicity. On Warwick’s contribution to guideline acceptance and the concerns above, Dr Fiona Sewell, Head of Toxicology at NC3Rs, stated: “ Statistical simulations conducted by Nigel Stallard and his team, addressed the first two areas of concern, and were vital for the final acceptance of the method by the OECD” [5.3]. Further research expanding on this point was reported in Sewell et al. (2015) [5.4], with co-authors drawn from a range of global stakeholders such as the French National Institute for Industrial Environment and Risks (INERIS), the US NTP Interagency Center for the Evaluation of Alternative Toxicological Methods (NICEATM) and the UK Home Office.

Additionally a 2019 RSPCA report Avoiding Mortality in Animal Research and Testing [5.5] highlights the importance of contrasting FCP to existing methods itself [3.1-3.2] as a component which helped to tackle evident toxicity doubts: "This concern [evident toxicity] surfaced repeatedly in efforts led by the UK NC3Rs to gain approval for the inhalation toxicity equivalent, the Fixed Concentration Procedure (FCP). Only extensive retrospective analysis of a large historical data set, and detailed comparison of the classifications made by the FCP with other methods succeeded in overcoming mistrust of evident toxicity and led to acceptance of the draft guideline in 2017 (OECD TG433) 13 years after its first publication". Following publication of the work by the Warwick group, a revised FCP was proposed in a draft test guideline TG433 in 2015, and this was adopted by the OECD in 2018 [5.1]. The adopted test guideline includes Warwick's work [3.1- 3.2] as two of the ten supporting citations with the diagrams illustrating how the procedure is to be conducted taken directly from one of the two research papers [3.1]. The overarching OECD guidance document (GD39) on inhalation toxicity testing was revised in 2018 [5.6] to reflect the adoption of TG433 and also to cite the Warwick papers [3.1-3.2]. Overall, the test is considered a “ major refinement” on the previous methods [5.3].

Improving animal welfare and reducing numbers of animals required for acute inhalation toxicity testing by changing OECD guidance

Traditionally, testing of acute inhalation toxicity has been conducted by estimating the LC₅₀, the concentration level that would lead to a death rate of 50%, following OECD test guideline TG403 or TG436. Depending on the initial concentration used, a standard LC₅₀ test requires testing of 30-40 animals [5.3], typically with at least a quarter of these exposed to lethal concentration levels.

The FCP, based on [3.2], and adopted by OECD from June 2018 on the basis of Warwick work [3.1-3.2], leads to animal welfare and use improvements.  First, the number of animals required is reduced to 7-11, a decrease of approximately 75% compared to the 30-40 animals used in a typical LC₅₀ test [5.3].  Second, the use of the endpoint of 'evident toxicity', defined as signs of toxicity that predict severe toxicity or death at higher test concentrations, rather than death, leads to testing at lower concentration levels. This leads to improved animal welfare compared to the other test methods, with 0-5 animals exposed to lethal concentrations [3.1].

Regarding the benefits of the new procedure, Dr Sewell stated, “ Acceptance of this test guideline has the potential to reduce the number of animals used for the classification of chemicals worldwide, whilst simultaneously reducing the levels of suffering, through the use of evident toxicity in place of death as an endpoint” [5.3].

The work by Stallard [3.1-3.2] was also included as two of the six publications cited as examples of progress towards developing the 3Rs (Reduction, Refinement and Replacement of animals in research) by signatory organisations to the Concordat on Openness on Animal Research in the UK in their 2015 Report [5.7].

On the current state of inhalation testing in the UK, Dr Fiona Sewell explained, “ In the UK, the Home Office Animals in Science Regulation Unit (ASRU) is now actively enforcing the FCP in preference to the other accepted approaches”, which reflects a substantiative nationwide shift in policy. While global harmonisation for FCP is still ongoing, the NC3Rs are “ working with the UK’s representatives to the OECD to build a case to remove the two existing Test Guidelines that use death as an endpoint so that the only accepted OECD method is the FCP” [5.3]

Regarding the journey towards acceptance, the Chief Executive of the NC3Rs stated that "The new OECD test guideline for inhalation toxicity has massive 3Rs benefits. The route to acceptance has been a difficult one even when we felt the data was water-tight. I am extremely proud of the work and commitment that the NC3Rs team and others in the UK, including our OECD national coordinator have put into making this method with its benefits for animal welfare a reality." [5.8]

Economic benefits from reducing animal numbers required for acute inhalation toxicity testing by changing OECD guidance

In addition to animal welfare improvements, the reduction in animal numbers required in TG433 leads to an economic benefit. The average cost of acute inhalation testing for a substance using the conventional method is estimated by US Environment Protection Agency to be over USD32,000 [5.9].  The reduction in the number of animals by around 75% (7-11 compared to 30-40) would therefore lead to considerable economic savings.