Impact case study database

**(1) Impact on NICE guidance/national policy:**York HTAs and economic decision models for PS and PsA have supported national decision-making and NHS prescribing policy since 2006, continuing through the REF period. Decision models are the fulcrum of NICE technology appraisal committee deliberative process and produce estimates of value in a standardised cost per QALY ‘currency’ that allows comparison across healthcare interventions. Decisions have rapid and direct impact because NHS England is legally obliged to provide funding for drugs and treatments recommended by NICE within 90 days, and the right to access these is enshrined in the NHS constitution.

“Decision models are central to NICE process and national decision-making as they provide the framework for synthesising available evidence and generating estimates of clinical and cost effectiveness in a format relevant to the NICE committee's decision-making process. The decision models that the York team developed for psoriasis and psoriatic arthritis within multiple technology appraisals (MTAs) therefore directly informed and supported NICE committee decisions about use of the agents considered in those MTAS”. NICE Deputy CEO [1]

In addition to the specific NICE evaluations for which they were developed [A-E], the York models provided the backbone for subsequent NICE single technology appraisals (STAs). STAs comprise independent evaluations of single therapeutic agents based on academic critique and evaluation of pharmaceutical company submissions to NICE. Both company submissions and academic appraisals have utilised the York models.

“The models developed within the MTAs were then applied and used as a basis for new models in subsequent STAs, which resulted in a series of biologic agents being made available to NHS patients with psoriasis and psoriatic arthritis during the period 2014 to 2020.” NICE Deputy CEO [1]

Specifically, since 2014 the York PsA model [C] formed the basis of company models submitted for ustekinumab [TA313] and apremilast [TA433]; the more recent York PsA model [D] for ustekinumab [TA340], ixekizumab [TA537] and tofacitinib [TA543]. The original York PS model [A] provided the basis for secukinumab [TA350], apremilast [TA419], ixekizumab [TA442], dimethyl fumarate [TA475], brodalumab [TA511], certoluzumab pegol [TA574], and tildrakizumab [TA575]. In addition two fast track appraisals of guselkumab [TA575] and risankizumab [TA596] drew on efficiency and cost assumptions from previous appraisals. All agents were approved for use in the NHS. [2]

Thus, since 2014,York research has informed 15 separate national guidance statements on psoriasis and PsA: two directly from underpinning York MTAs [D][E], seven from York STAs plus six STAs/ fast-track assessments completed by others. Together these led to 11 biologic agents being made available within the NHS, as were two small molecule drugs (dimethyl fumarate and apremilast), evaluated using the same model.

“I can therefore confirm that your research has supported national decision-making that has had direct impact within the NHS”. NICE Deputy CEO [1]

(2) Impact on UK clinical practice guidelinesAs well as underpinning NICE guidance, the York models informed the NICE overarching clinical guideline for the assessment and management of PS (2012, updated 2017), the Scottish Intercollegiate Guidelines Network guideline for the diagnosis and management of PS and PsA (2010) and the British Association of Dermatologists guidelines for biologic therapy for psoriasis (2015, updated 2020). All remain current and have cited/referred to the underpinning York HTA reports, associated NICE guidance or the “York models”. [3]

(3) Impact on prescribing The impact of York research acting through mandatory NICE guidance and clinical guidelines is corroborated by national prescribing data available from NHS digital, analysis of which illustrates that prescribing patterns of approved drugs for autoimmune disease (data are not available by specific condition) align with positive recommendations from NICE [4]. An example is given in the figure below.

Figure 1. Ixekizumab prescription in England and Wales (2017-2018) showing dates of issue of NICE Final appraisal documents (FAD; 1st and 3rd vertical lines) and European Medicines Agency (EMA) licencing (middle vertical line), illustrating rise in prescriptions following the issue of each FAD.

(4) Impact on patients’ lives Psoriasis and PsA are debilitating chronic conditions that have a serious adverse impact on patient health and wellbeing. The 11 new biologic agents approved during the REF period provide patients who fail on conventional therapy with access to life-enhancing treatment, improving their symptoms and quality of life, whilst ensuring value for the NHS.

“In my experience as an expert patient at NICE technical appraisals of biologic agents for the treatment of psoriasis and psoriatic arthritis, evidence from the University of York HTAs and the York models has been both informative and essential in the understanding of the range of benefit which has provided choice to patients, but also has guided and provided healthcare professionals with confidence, certainty and knowledge to help in their discussions with patient about care plans and pathways. This has also aided patients to see that treatments are of benefit and increased the choice available. It is clear that without the thorough and consistent evaluations carried out at York to inform national decision making, the advancement of care for people with psoriasis and psoriatic arthritis would not have been as comprehensive and would not have provided the life-changing benefit that many patients are currently experiencing.” Chief Executive Psoriasis and Psoriatic Arthritis Alliance (PAPAA) [5]

(5) Estimated population benefit to the UK It is estimated that 21,000 PS and 6,500 PsA patients who fail conventional therapy are eligible to receive biologic agents annually. Using an average of the quality adjusted life years (QALYs) in NICE committee preferred modelled scenarios quantifies the impact of the underpinning research on UK population health as generating an estimated gain of approximately 692 QALYs for psoriasis and 3,408 QALYs for psoriatic arthritis 2014 - 2020. This derives mostly from improved quality of life rather than extended survival [6].

(6) Wider impact on NICE evaluations by example in NICE technical guidance The York psoriasis model provides a detailed example (with code) in the NICE DSU technical document on generalised linear modelling frameworks for network meta-analysis. This forms the guidance that all NICE technology appraisal teams are encouraged to follow, including materials produced and submitted by pharmaceutical companies (York is part of the DSU). Thus, our research on PS indirectly supports the production of all submissions to NICE, committee decisions and national guidance across a broad range of clinical topics [7].

“The inclusion of York’s psoriasis model as one of the key models and as a worked example with ready to use code, highlights its importance to NICE technology appraisals thereby indirectly supporting the production of NICE submissions which underpin committee decisions and NICE guidance.” NICE DSU Director [7]

(7) International impact through international guidelines

Evaluations and economic models developed by the York team have also had wider international impact beyond the expectation of the original research and been used in clinical guidelines in several countries. These include the Dutch Society of Dermatology and Venereology (2011), Italian Society for Rheumatology (2009) and guidelines issued by the Japanese Dermatological Association and the Japanese Ministry of Health (2018), all of which remain current. York models have also been used as the basis of HTAs conducted in other jurisdictions including the USA, thereby generating indirect impact and population health gain in those jurisdictions. All have cited/referred to the underpinning York HTA reports, the associated NICE guidance or the “York models”. [8]

Impact on policy statements, guidelines, toolkits and care pathways

Research undertaken at York has informed international and national guidelines on enteral feeding strategies for very preterm and VLBW infants.

Three sets of World Health Organisation (WHO) guidelines cite York research:

• “Donor Human Milk for Low Birth Weight Infants”;

• “Feeding of Very Low Birth Weight Infants”; and

• “Standard Formula for Low Birth Weight Infants”.

All are subsets of the WHO’s guidance on feeding of low birth weight infants [1].

Research output [B] is cited as the underpinning evidence for recommending that VLBW infants “ who cannot be fed mother's own milk should be fed donor human milk [in] settings where safe and affordable milk-banking facilities are available or can be set up”. Seven outputs authored by McGuire (including **[D][E][F]**) are cited as informing feeding guidelines for VLBW infants, including specific recommendations on the rate of enteral feeding and the use of nutrient fortifiers.

York research is cited in other national and international guidelines and policy documents by the American Academy of Pediatrics (AAP), the Canadian Consensus Group (CCG), and the European Association of Perinatal Medicine (EAPM) [2]:

• The AAP 2017 policy statement cites output [B] in recommending the use of donor human milk: “ recent studies support health benefits for its use in [VLBW] infants, especially in decreasing rates of necrotizing enterocolitis”.

• The CCG 2015 guidelines refer to output [E] to underpin key recommendations for VLBW infants, including to “ start trophic feeds preferably within 24h of life”; and “… if the feeds are tolerated for around 2-3 days, consider increasing faster”.

• The EAPM 2017 consensus guidelines recommend early initiation of human milk feeds, and their rapid advancement, for VLBW infants, stating that “ there is no evidence of increased NEC [necrotising enterocolitis] with early initiation of feeds or advancing feeds more rapidly”, based on McGuire’s finding [C][E]. Output [B] is cited to support the recommendation that “ donor breast milk is better than formula as it reduces the risk of necrotising enterocolitis”.

Impact on training and education

Through its underpinning of WHO and other guidance and policy statements, York research has informed training and education in enteral feeding for very preterm and VLBW infants in the UK and internationally, leading to the wider use of donor human milk and progressive enteral feeding. For example:

• The Interpractice 21st project (Oxford Maternal and Perinatal Health Institute) provides training for healthcare professionals to promote the best international standards in nutrition for preterm infants. The online module, “Feeding Recommendations for the Routine Care of Preterm Infants”, available in English, Spanish, Italian, Russian and Portuguese, cites output [E] as the evidence for recommendations on the initiation, advancement, and volume and frequency of enteral feeds [3].

• The European Standards of Care for Newborn Health, a practical standards-based resource for neonatal healthcare practitioners, cites outputs [B][C][E] in its advice on establishment of enteral feeding in preterm infants [4].

Impact on adoption and practice

The volume and use of donor human milk, and establishment of donor milk banks, has increased substantially in the UK and internationally during the past decade [5]. For example, Human Milk Banking Association of North America data indicate an expansion from 16 non-profit milk banks in 2013 to 30 as of 2020. Collectively, these processed about 85,000L of milk in 2013, rising to ~210,000L in 2019 [5]. The expansion is predicated on research including the York Cochrane reviews, and the WHO guidance that draw on these. Milk banks internationally base their existence and practice on the WHO guidance, as evidenced in sources such as the Australian Government’s healthcare resource pages, and official recommendations for the adoption of milk banks in Spain which cite both the recent WHO guidance and York research [5]. The Health Scotland 2017 report for NHS practitioners and policy makers notes that “ access to donor breast milk in all neonatal units has resolved some of the issues of making breast milk equally available to all babies who need it” [5].

Further evidence of impact in neonatal care facilities is provided by recent studies in North America. A quality improvement project in Massachusetts (USA) that examined the effect of newly-introduced feeding regimens on growth for very preterm infants cites outputs [C] and [E] as underpinning two of its key elements: “earlier initiation of trophic feeds [and] accelerated daily feeding” [6]. A study examining the expansion in US neonatal units using donor human milk for supplemental feeding of very preterm or VLBW infants (74% increase since 2011) cited output [B] as the unpinning evidence that allowed practitioners to become more confident in recommending adopting this practice [7a].

Impact on patient outcomes and cost savings

The changes in practice based on York research have improved care and outcomes for very preterm and VLBW infants. The Massachusetts initiative concluded that the new enteral feeding regimens reduced time to establish feeds and reduced central vascular catheter use, a major risk factor for infection and sepsis in very preterm infants [6]. Another US study showed that “ a progressive standardized, evidence-based feeding protocol was associated with improved growth without increased risk for necrotizing enterocolitis” in VLBW infants [7b]. In the UK, the Human Milk Foundation, a charitable umbrella organisation for NHS milk banks, connects increased use of donor human milk with reduction in necrotising enterocolitis and the saving of preterm infants’ lives [5].

Cost savings may be associated with adopting enteral feeding regimens informed by the York reviews [8]. The Massachusetts study concluded that “ the change in practice also resulted in cost savings” because infants spend a shorter time spent in neonatal intensive care units [6]. Economic modelling from a UK NHS perspective suggests that interventions to reduce the use of formula for feeding preterm infants are likely to achieve substantial cost savings for the UK NHS (lifetime quality-adjusted life year gain >10,000), principally by reducing rates of necrotising enterocolitis, sepsis, and neurodevelopmental impairment [9]. In Southern Africa, analysis suggests that donor human milk is a highly cost-effective intervention for very preterm infants, costing ("worst case") USD619 per Disability Adjusted Life Year averted. This analysis concluded that there is a “ compelling argument to increase the supply of donor milk in middle-income countries” [10].

(1) Impact on UK clinical practiceYork research directly informed the NICE Guideline on the Prevention and Management of NS (for all ages of patients) [NICE CG151]. This included preliminary and unpublished findings that were shared by Phillips as part of his membership of the guideline development group; specifically providing evidence for adoption of a risk stratified (low-risk) approach [1.5.1 and 1.5.2] [A], use of a single inflammatory biomarker [1.4.1.2] [F], and discontinuing the use of routine chest X-rays on admission [1.4.2.2] [E]. The guideline was first published in 2012, but remains current; in 2020 NICE decided that recommendations required no substantial changes [1]. Subsequent York research including [B] and updates to [A] and [E] provided further and specific evidence on managing NS in CYP. Thus, York’s underpinning research has supported clinical decision making since 2012 and throughout the current REF period.

“*Research from the University of York on neutropenic sepsis (NS) in children and young people (CYP) with cancer directly informed NICE CG151, the NICE Guidelines on the Prevention and Management of NS, in particular providing supporting evidence for adoption of a risk stratified approach, which subsequently results in both benefits for patients and resulting cost savings for the NHS in terms of earlier discharge lower intensity antibiotic therapies and discontinuing use of routine chest x-rays on admission.*” CEO, Children’s Cancer and Leukaemia Group (CCLG) [2]

Repeated 14-day snapshot audits of UK NS admissions by the national network of heath care professionals treating CYP with cancer (the CCLG) demonstrated increasing alignment with NICE guideline recommendations and with our research findings, which were specific to paediatric patients. Published audits showed that:

• Implementation of risk stratification programmes increased from 36% in 2012 to 75% in 2017, reducing (in-patient) stays from a median of 5 to 3.5 days.

• Broad spectrum intravenous antimicrobial use decreased; the proportion of patients who either transitioned from intravenous to oral antibiotics or stopping at 48 hours after admission increased from 43% to 76% in the same period.

• Centres no longer undertake routine chest X-rays.

• Despite increasing use of serum inflammatory biomarkers in other paediatric specialities, use in this patient group has been minimal. [2]

The CCLG CEO noted how this impacts on patients and families:

“The impact of this important work is multifactorial, from informing guidance to supporting its implementation and changing practice across the UK, to cost savings for the NHS, and informing international clinical practice. Ultimately of course, I would argue the benefits to young patients and families are the most important impact of this work. A childhood cancer diagnosis is a distressing, isolating and challenging time for any family, and any work that improves provision of supportive care will improve quality of life. We know from our work with patients and families that seemingly small things make a huge difference – a shorter hospital stay, not undertaking additional tests and procedures, or being able to go home with oral medication rather than a sustained admission for IV therapy – impacting in a number of domains on quality of life including mental wellbeing and stress, finances, and family life.” [2]

Implementation accelerated in 2020 with the emerging SARS-CoV2/COVID-19 pandemic. UK paediatric oncology services were prioritised to continue delivering therapy but urgently required a strategy to safely minimise hospitalisation and switch to lower-toxicity approaches, including full implementation of risk-stratified management of NS, and to shorten treatment duration even further. Within two weeks, the York team validated the PICNICC Australia ‘AUS’ risk prediction score for a UK population [C] and evaluated use of very short duration admission, leading to a new agreed national strategy through the CCLG. Acknowledging provider fears about risks of home-based care [D] with clear messaging and consistent information, and supporting the use of a ‘virtual ward’ with daily structured phone calls to parents was also part of the response.

Health care provider implementation tools and patient-information materials based on York research were developed, as was an evaluation package to assess breadth and quality of uptake. The importance of York input to the pandemic response was confirmed by both the Chair [3] and CEO [2] of the CCLG:

“…the York team validated the PICNICC Australia ‘AUS’ risk prediction score for a UK population and evaluated its use within two weeks. In turn this led a new agreed national strategy through the CCLG. This extremely short turn around period would not have been possible without the underpinning programme of research carried out by the York team who had validated the approach to risk stratification and empiric treatment of febrile neutropenia.“ [3]

“Full implementation of risk stratified management of NS, and a further reduction in treatment duration, was made possible by the rapid validation by the York team ……This enabled a newly agreed national strategy to be delivered through the CCLG.” [2]

Data from CCLG audits comparing length of stay and antibiotic use in 2017 and preliminary data from 2020, combined with NHS National Cost Collection data illustrates that implementation of risk stratified management has resulted in a substantial cost saving for the NHS. The proportion of NS episodes discharged within 24 hours increased from 0% to 27%, and average inpatient stay reduced from 3.5 days to 1.8 days for those who were hospitalised. Given costs of GBP997 per inpatient bed day and GBP538 for day case management, this generates savings of GBP2,010 per episode. A further small saving of GBP21 per episode accrues from low risk patients switching from IV to oral antibiotics. Using 2020 estimated annual incidence of 3,011 NS episodes, this equates to freeing up 5,703 bed days and reducing NHS costs by GBP6,115,563 per year [4][5].

(2) Impact on international clinical practice: York research has also directly informed and “ been instrumental to the development and completion” [5] of international guidelines, which have in turn changed practice and delivered international impact as the clinical lead/corresponding author of the International Society of Paediatric Oncology guidelines notes:

“… research on the management of fever and neutropenia in children and young people being treated for cancer undertaken by the team at the Centre for Reviews and Dissemination at the University of York was an important source of evidence for the international clinical guidelines that were produced by an international panel of pediatric oncology experts, including Dr Bob Phillips. The guidelines were subsequently adopted by The International Society of Paediatric Oncology (SIOP). During development, the York team’s published and not yet published research … was particularly important in informing recommendations about the initial management of FN. The guidelines … have since been adopted by the American Society of Pediatric Hematology/Oncology, the Pediatric Oncology Group of Ontario, the American Society of Clinical Oncology, C17 Council (an organisation including institutionally appointed heads of the 16 paediatric hematology, oncology and stem cell transplant programmes across Canada), the Multinational Association of Supportive Care in Cancer (MASCC) and Children’s Oncology Group (COG). [5]

Guideline recommendations are now included in every new anti-cancer study undertaken across the North American COG network and widely within Europe [5].

Studies carried out in children’s hospitals in the US have also reported benefits arising from guideline-congruent low risk management. In Oklahoma costs of NS management in low risk patients were halved: “ The mean total cost of an LRFN episode was $12,500 per patient pre implementation and $6168 post implementation, a decrease of $6332 (51%) per patient”. In Massachusetts “40% of [NS] episodes were defined as low risk and managed either entirely in the outpatient setting ...or with a step down strategy involving a very brief inpatient stay” and in Missouri over 60% of CYP presenting with NS were discharged early (“ 188/299 FN admissions”). The Missouri study noted that “ by reducing the overall length of stay in a subset of patients, early discharge can improve quality of life and reduce costs. This practice may also benefit patients in terms of safety by decreasing the risk of hospital-acquired infections and avoiding prolonged antibiotic exposure that leads to resistant bacterial and fungal infections” [6].

The Melbourne based PICNICC-Australia team demonstrated that adopting risk stratified management saved “290 in-hospital bed days in 18 months” and reported that with in-hospital management of paediatric low-risk NS costing AUD2,200 per day and home-based NS care approximately AUD830 per day , “the cost benefit …is likely to be substantial” and that the “program is currently being scaled nationally, thereby increasing the clinical, economic and quality of life impact of this model of care” [7].

[Impact I] Developed clinical standards and National Certification Programme

1. Research by Doherty developed key performance indicators and an analytic approach leading to the foundation of a National Certification Programme for CR (NCP_CR). This is run jointly by the British Association for Cardiovascular Prevention and Rehabilitation (BACPR) and national audit team (3.1 & 3.2). The National Certification Programme monitors and reports on the quality of CR delivery against published clinical minimum standards. Longitudinal national audit data shows that CR quality has improved significantly from only 27 programmes (12%) in 2014 to 93 programmes (42%) achieving full certification status in 2020 representing a 30% improvement. British Journal of Cardiology (2016) (5.1a).

2. The same research informed clinical standards and core components used by over 230 clinical programmes across the UK. ( 3.1 is reference 81 in the quotation below):

“The ultimate goal is for all CR programmes to deliver services in line with the Standards and Core Components in this document, however at present most programmes are working towards the minimum standards as outlined in the NCP_CR. ⁸¹” Page 19 BACPR Standards and Core Components (2017) (5.1b).

1. National reporting of key performance indicators as part of the NCP_CR showing that the quality of CR has increased since this research was conducted. BHF Quality and Outcomes Report 2020 page 23 (5.1c).

2. UoY research on the development and success of a national certification programme and minimum standards was viewed, by the European Association for Preventive Cardiology, as the first to implement and evaluate minimum standards which were then used to inform the development and implementation of European standardization and quality improvement of secondary prevention: “The use of minimum standards for the evaluation of the quality of CR has been tested elsewhere ²¹” Page 2 Standardization and quality improvement of secondary prevention through cardiovascular rehabilitation programmes in Europe: Eur J Prev Cardiol. 2020. Reference 21 in the quote is 3.1, (5.1d).

[Impact II] Improving national CR wait times and patient outcomes

1. Research presented at national and international conferences in 2015 and 2016 showed that timely CR led to improved mental health and physical health outcomes (3.2). The BACPR standards writing group reviewed Doherty’s research on waiting times and CR delay and used it to support timely CR as part of their standards. Using published data collected as part of NHS Digital audits the quotation below highlights more than a 50% reduction in waiting times driven by UoY research:

“In 2020, UK median CR wait times have reduced to 33 and 21 days for surgical and non-surgical patients, respectively. This represents a reduction in waiting time of 21 days for surgical patients and of 19 days for non-surgical patients compared with 2014 surpassing national targets and yielding significant improvements in service delivery and patient benefit by avoiding delay (Hinde et al 2020). This change in clinical practice owes much to sustained BHF-funded studies at the University of York on wait times. Before these studies, average wait times had only decreased by 2.5 days between 2011 and 2014.” Page 20 BHF Quality and Outcomes Report 2020 (5.2a).

1. UoY research on waiting times has continued to inform new versions of clinical standards evidenced by the following quotation where reference 28 in the quotation is our research on the benefits of early CR: “There is continued emphasis on the importance of early CR which is both safe and feasible, and improves patient uptake and adherence.21–28” Page 511 BACPR, Standards and Core Components. Heart 2019;105:510-515 ( 5.2b).

2. The quotation below cites two UoY studies, on the impact of early rehabilitation on psychological outcomes and physical outcomes, as the only references used to inform the decision to include early CR as a minimum standard for CR services across Europe: “The timing of CR has a significant impact on fitness⁵³ and psychological outcomes ⁵⁴” page 3 of Standardization and quality improvement of secondary prevention through cardiovascular rehabilitation programmes in Europe: Eur J Prev Cardiol. 2020 (5.2c).

3. Beyond the recognised improvement in physical and mental health outcomes for patients, evidenced through UoY research on timely CR, there is also a service level benefit achieved through enhanced uptake. Timely CR increases the likelihood of patients starting whereas delayed CR decreases the likelihood. Health economic modelling of timely CR has quantified a 15.3% benefit in CR uptake (i.e. ~ 20,786 more patients based on 2019 national audit data) and improved long-term health with a cost per quality-adjusted life-year of GBP3,286 (5.2d).

[Impact III] Development and implementation of a new self-management intervention to address a known gap in CR delivery for patients with heart failure (HF).

1. The REACH-HF programme of research including Doherty’s chair based exercise intervention was developed with extensive patient and carer involvement. Clinical and cost effectiveness was established (3.3) after which REACH-HF was rolled out through four NHS Beacon sites and four Scottish Health Boards. Roll-out was supported by NIHR, NHS England/Scotland. NHS Digital also introduced REACH-HF as a new mode of delivery option in Jan 2019 allowing clinicians to routinely record this intervention as part of an NHS provision. The REACH-HF service implementation approach, outlined above, won the BMJ Services Award for Stroke and Cardiovascular Services in 2020 . The BMJ award recognised REACH-HF for its excellence in healthcare provision and for successfully implementing leading research into NHS clinical practice (5.3a).

2. REACH-HF was adopted by the South West Academic Health Sciences Network (5.3b).

3. The REACH-HF intervention was adopted nationally by the BHF and BACPR as part of the Covid-19 initiative to enable older patients with heart failure to exercise safely at home. In response to Covid-19 NHS service changes and an urgent need for online training of NHS staff, Doherty and REACH-HF colleagues changed their face-to-face facilitator course to online training. In doing so they successfully delivered training to 100 staff leading to an increase in home-based CR services in the UK. Doherty also adapted his chair based exercise programme making it freely available to NHS staff and patients online. National audit data pre-Covid (2019) vs Covid era (2020) confirms that the proportion of patients with heart failure taking up hospital-based CR dropped by 47% whereas home-based CR increased by 52%. Based on the success of our Covid-19 response along with development and implementation of online training for NHS staff, NICE endorsed REACH-HF as a quality assured shared learning example under the theme of Covid-19-ready-rehabilitation-for-heart-failure (5.3c).

[Impact IV] Policy and practice: NHS Long-term Plan; British Heart Foundation Strategy and international clinical guidance.

1. In 2019 UoY researchers (Doherty, Harrison, Hinde, Bojke) were asked by NHS England and BHF executives to investigate, as part of NHS Long-term Plan preparations, the cost benefit of increasing CR uptake. Our findings and calculations ( 3.6 also incorporating 3.5 in the cost analysis), were used to aid decision making and set targets as part of the NHS Long-term Plan and BHF Strategy:

“Scaling up and improving marketing of cardiac rehabilitation to be amongst the best in Europe will prevent up to 23,000 premature deaths and 50,000 acute admissions over 10 years” (Page 63 section 3.72 of the Long Term Plan) (5.4a).

BHF strategy document (The Big Picture 2018) used our research findings and UoY based National Audit of Cardiac Rehabilitation data to set its 65% and 85% targets: “Achieving an uptake rate of 65% would result in 8,500 fewer deaths and 21,000 fewer hospital readmissions over 10 years. And reaching 85% uptake could save a remarkable 20,000 lives and avoid nearly 50,000 admissions over the next decade, as well as saving the NHS tens of millions of pounds. Source: Hinde S, Bojke L, Harrison A, Doherty P. (2018) Modelling of potential CR uptake scenarios for the BHF vs 2015/16 NACR data” Page 3 (5.4a).

1. Our joint European collaborative research project entitled Cardiac Rehabilitation Outcome Study (CROS) was used to inform Scottish National Clinical Guidance SIGN 150:

“While CR meets the definition of a complex intervention, with studies including some or all of the elements described in the BACPR pathway, systematic reviews have concluded that the reduction in cardiovascular mortality associated with attending CR can be attributed to the exercise component.^5,6” Page 1, Reference 6 is 3.4, (5.4b).

The CROS project systematic review and meta-analysis also informed European standards on the ability of CR to benefit patients by reducing premature death:

“Previous data, including recent meta-analysis have shown the efficacy of CR ^3,5–7 to reduce mortality” Page 2, reference 5 is 3.4, (5.4b).

Size of the NHS budget

1. Annual national estimates of productivity influence policy and affect decisions on NHS resources. The DHSC competes with all other government departments for a share of national resources. In recent years the relative “protection” of the health budget has sharpened the focus of DHSC negotiations with the Treasury, requiring evidence of efficient resource use in the NHS. The former Chief Analyst of DHSC noted, “By detailing the amount and quality of care secured from NHS resources this work provides evidence about what the NHS is doing with the budget it receives and helps identify opportunities for better use of funding.” [1a] Health expenditure in England is around £127 billion per annum and demonstration of overall productivity is key to achieving a more favourable settlement for the NHS. York research provides evidence of this productivity. The Office for National Statistics (ONS) uses the York quality adjustments each year in their estimate of productivity which in turn feeds into the National Accounts. York’s research is referenced in ONS documents which explain the quality adjustments used since 2005 to adjust productivity measures (retrospectively at first, from 1998/99 onwards) [1b]. Annual publications from the ONS acknowledge the research, e.g. in 2020: “The English financial year productivity figure is produced on a similar basis to an alternative healthcare productivity measure produced by the Centre for Health Economics … [they provide a link to CHE research]. …the largest element of the quality adjustment is produced by the University of York and used in both publications” [1c]. The ONS also refers to ongoing research at York that is improving the quality measures further, including the development of criteria to assess suitability of quality measures across all public sectors [1d]. The quality adjustments make a distinct difference to the estimate of productivity, accounting for an average of 28% of the increase in productivity between 2014 and 2018, and for 57% of the growth in the most recently available year (2018) [1e]. York research is recognised widely; e.g. cited in 2015 in Parliament: “The most comprehensive and reliable estimates of productivity for the National Health Service in England are compiled by the Centre for Health Economics at the University of York. Their published series provides data from 1998/99 onwards…” [1f].

The York productivity measures have been used in other contexts to support and inform important policy decisions. First, the trends in both total factor and labour productivity assist policymakers in determining appropriate NHS pay awards. Annual reports of the NHS Pay Review and the Doctors and Dentists Pay Review reproduce evidence submitted by the DHSC about York’s research, as illustrated in examples from 2018 [2a][2b]. Pay is a major element in NHS spending, hence using the trends in underlying productivity of labour to help decide the level of pay awards each year, is vital to determining the resources available for the NHS. Second, the Office for Budget Responsibility uses the productivity estimates to inform their long-term projections of health spending and fiscal sustainability which in turn affects decisions made on NHS spending by the Treasury [2c]. Third, York research was part of the evidence on hospital efficiency submitted by NHS England to the Health Select Committee in 2016 (only 3 non-government sources of evidence were cited, and York’s was one of these). The data were used to estimate efficiency savings that could be expected from NHS trusts, hence feeding into the overall calculation of the net funding requirements and reflected in the national settlement for the NHS budget [2d]. Street was specialist adviser to the 2016 Comprehensive Spending Review and the national and trust level research was cited, the latter as evidence of scope for efficiency improvements [2e].

In summary, York research has strengthened the evidence underpinning spending decisions and in turn this has determined the size of the budget made available for the NHS and hence the amount of health and care services that can be delivered to the whole population. York expertise (Castelli) has been sought by those seeking to improve the methods of measuring productivity outside the health sector: e.g., Ministry of Justice; and outside the UK: e.g., World Health Organisation; China and Malaysia [3a]. The Deputy Director General of the Malaysia Productivity Corporation noted that the York research (and hands-on training provided to them by Castelli) helped them to develop successfully the healthcare measurement framework for Malaysia [3b].

Distribution of the budget

2. The PBRA method (sometimes called the “Nuffield Formula”) was adopted in 2013 by the Advisory Committee on Resource Allocation (ACRA), an independent expert body that advises the Secretary of State for Health on how national resources are allocated to commissioners. ACRA is currently chaired by Smith and Cookson is also a member - both are at York and other York economists (Gravelle, Rice) are/have been members of ACRA and its Technical Advisory Group in the past, with a combined input of 38 years of service. The new PBRA formula recommended by ACRA was accepted by NHS England in December 2013, and they referred to the changes as helping to “ensure that funding matches the needs of local populations” and is “equitable and fair” [4a].

This formula has been used every year since 2014/15 to allocate resources to CCGs [4b]. In principle, if every individual had a budget allocated personally to them that met their individual healthcare needs, this would be the most accurate allocation. In practice, individual allocations are aggregated across practice lists/CCGs to create a flexible budget that pools risks. PBRA uses a wealth of individual level data to increase the accuracy of the formula which predicts needs and hence ensures that resources are allocated appropriately to better meet the healthcare needs of local populations. The National Audit Office noted: “ By using newly available data at the level of individual patients to create a more detailed model of healthcare utilisation, NHS England’s new approach is better at predicting relative needs” ; and, with reference to the implementation of PBRA : “NHS England adjusted 90% of each clinical commissioning group’s target allocation for 2014-15 for related need. The adjustment ranged from a 27.9% increase to a 25.0% decrease, compared with what target allocation would have been based on population size alone” [4c].

Although the PBRA formula was first used for 2014/15 allocations it now has even more influence: it forms the basis for 57% of the CCG annual allocations by NHS England (£59.4 billion from a total £104.3 billion allocated to CCGs to meet healthcare needs in 2019/2020), including an extension of the PBRA method to mental health (£9.4 billion) (latter by University of Manchester based on the original methods) and specialist services (£8 billion by formula). Current plans will extend it to prescribing and maternity allocations (£9.5 billion) [5a]. CCG allocations have most recently been set for the period 2019/20 - 2023/23 using this formula [5b]. Hence the methods developed by York have been used since 2014/15 to produce a formula that allocates an increasingly large proportion of the NHS budget in a fair and accurate way, maximising the congruence between healthcare needs and the services commissioned for, and provided to, the entire population.

Using the budget effectively

3. York research on health opportunity costs in the NHS was described by the former Chief Analyst of the DHSC as “the single most important piece of work” being done for the DHSC” [6]. Through frequent meetings with policy makers and analysts at DHSC, NICE and the Health & Care Alignment Working Group (a cross departmental group tasked with aligning how economic evaluations are undertaken), the research has been influential in policy analysis by the DHSC. The DHSC now routinely uses £15,000 per Quality Adjusted Life Year (QALY) as an empirical estimate of the health the NHS generates (loses) with increases (decreases) in funding, because of York’s research. This evidence is used in DHSC impact assessments which are a mandatory requirement for all new policies introduced across government. Since 2014, DHSC has undertaken 23 impact assessments using the £15,000 per QALY estimate, considering policies ranging from dental charges regulation to accelerated access to new medical technologies [7]. The total financial impact of these policies was estimated in the assessments at £1.9 billion but, prior to the York research, the implications for population health were not routinely considered. This is now included in the 23 impact assessments and was estimated in total as 125,846 QALYs which the DHSC values at £7.6 billion, a four-fold difference. It has also been used in specific policy decisions: eg, the reform of the Cancer Drugs Fund following the NAO review which requested evidence from York on the health opportunity costs versus benefits [8] and resulted in removal of less cost-effective drugs.

The methods were extended to public health and social care and have influenced Public Health England’s (PHE) approach to the evaluation of the health gained from spending on public health interventions. The York research demonstrated that spending extra money on public health interventions could produce greater overall health gains than spending it elsewhere in the healthcare system, leading PHE to make a “prevention over cure” argument [9a]. The Chief Economist of PHE stated that this argument was the central message in discussions with the Treasury on public health spending and “… was influential in securing a better settlement than in previous years”, as well as feeding “directly into the deliberations” of the Health & Social Care Taskforce, a joint group between the Treasury and Number 10; and the York research team was commended for the “policy relevance and impact of this work” [9b].

Impact beyond the NHS has been achieved by motivating research in other countries (by York researchers as well as others) which has informed international decision-making, most notably by governments in Norway and Canada which have based recommendations for pharmaceutical pricing and regulatory reform directly on York’s threshold estimates [10a][10b]. The Canadian Patented Medicines Prices Review Board stated the York research was “instrumental” in them advancing price regulatory reform and without it they would have been unlikely to advance reforms that are “expected to save the health care system billions of dollars” [10c]. In the USA the research informed lower benchmark drug prices adopted by the Institute for Clinical and Economic Review (ICER - known as an independent drugs “watchdog” in USA), allowing Medicaid and private insurers to negotiate lower prices (reflecting health opportunity cost), making drugs more affordable and maximising the health gained from spend. ICER stated that the research “directly informed” their approach and the reduction of their value based benchmark [10d]. In each case, overall population health is improved as a result of better allocation of health resources.

Nature of impact

The research programme has had a major impact on the development of QOF. Changes to QOF are agreed in annual negotiations between the Department of Health (DH) and the British Medical Association (BMA), working on recommendations by the National Institute for Health and Care Excellence (NICE). Since 2009, members of the research team have directly advised NICE, instigating changes to the structure and scope of QOF, including:

- Removal of indicators: performance frameworks cannot cover all clinical areas, and achievement on indicators eventually reaches a point beyond which further improvement is not feasible and a rigorous approach to removal is therefore required. We developed key criteria on which indicator removal decisions were based and 46 indicators, accounting for over GBP403,000,000 in incentive payments, were removed from the programme between 2013 and 2018.

- Adjustment of achievement thresholds: under QOF, payments are scaled between a lower threshold, which sets the minimum level of achievement required, and an upper threshold, which indicates a high level of achievement. We developed a formula to calibrate achievement thresholds against historical performance levels, and this method for setting targets was implemented in 2013/14 for coronary heart disease indicators ( 5.1). This raised upper thresholds, and led to increases in average practice performance, for example: blood pressure and/or cholesterol was controlled for an additional 38,000 patients with coronary heart disease in England following the threshold changes ( 5.1).

Our work continued to inform development of the QOF programme, and in 2017 Tim Doran was appointed to the QOF Review Technical Working Group ( 5.2) , which was tasked with conducting a fundamental review of QOF and producing recommendations for reform or replacement of QOF ( 5.3, 5.4, 5.5). The Working Group reported in August 2018, recommending a fundamental restructuring of elements of QOF, including:

• targeting indicators at specific population segments

• introducing a personalised care adjustment to align indicators with clinical decision making and patient choice

• retiring ineffective indicators

• introducing a quality improvement domain to address clinical priority areas ( 5.4).

The review incorporated evidence from research on QOF and other physician incentive schemes conducted by a range of research groups, but most of the key evidence was derived from the work of our group ( 3.4). In the words of the Chair of the Working Group:

“The evidential review was heavily dependent on the research outputs of Professor Doran’s team, and his input to the subsequent working group discussions was instrumental in making the case for substantial changes to the framework, including the removal of ineffective quality indicators and better targeting of remaining indicators to appropriate patient groups.” ( 5.3)

As a direct consequence of the review, several changes were implemented for 2019/20 ( 5.6) representing a major reallocation of NHS resources, including: the retirement of 28 indicators (worth over GBP200,000,000 in incentive payments); the introduction of 15 new indicators (worth over GBP130,000,000) and the introduction of a new Quality Improvement domain (worth over GBP90,000,000).

The research has also achieved impact beyond academic and policy audiences, with national print ( 5.7) and broadcast ( 5.8) media using our research to raise concerns about the limited cost-effectiveness of incentive programmes. Reports of our research highlighting potential negative impacts on unincentivized aspects of care prompted responses from the Chair of the British Medical Association's GP Committee and the Chief Executive of the Patients Association pressing for reform of the incentive programme, ( 5.9) leading the Department of Health and NHS England to undertake the national QOF Review in England ( 5.2). In Scotland, the QOF was discontinued altogether in 2016.

Innovations

As QOF was implemented simultaneously across all UK practices, precluding the use of randomised trial approaches, a range of novel quasi-experimental study designs (including interrupted time series and synthetic control approaches) were developed to estimate the causal effects of the scheme. A range of novel data linkages were also required for the studies. Our approaches and linkages have subsequently been adopted by other international research groups investigating these issues. This work has also had implications beyond the investigation of incentives, and helped to legitimise the use of quasi-experimental methods in health services research. Research on QOF was also based on novel datasets that were generated to support implementation of the programme, requiring the development of new methods and definitions of quality/performance, and of extensive coding sets to enable interrogation of existing primary care clinical computing databases. The research team created the online Clinical Codes Repository to enable sharing of code lists between research groups, and this now holds 84,049 codes.

Our research has been geared towards impact at various levels: 1. National, especially in Malawi; 2. Regional adoption of the HBP work in Malawi as best practice; and 3. Global policymaking. The thread that brings these impact areas together is that the outcome of decisions made at higher levels can only be fully understood by determining impacts on those countries ultimately intended to be the beneficiaries of international health decisions.

(1) Designing the Essential Health Package (EHP) of interventions to be prioritized for funding in Malawi (national level)

Like many countries, Malawi has chosen to prioritize spending by developing a health benefits package (which in Malawi is referred to as an Essential Health Package; the EHP). York methods and applications have been incorporated into Malawi’s plans for healthcare provision through sustained engagement between the Ministry of Health (MOH) and York researchers, including an initial health economics workshop with the MOH and other Government departments held in June 2016 [5.1a] and annual meetings since 2016. In July 2017, Malawi published its medium-term strategic health plan for 2017-2022 with a chapter on EHP that is underpinned by York research [5.1b]. As the Deputy Director - Planning Department in the MOH puts it: “CHE’s research has impacted on Malawian policy development in two ways: by providing an analytic framework which enables robust determination of the EHP around which the health budget is formulated; and by providing new estimates of Cost-Effectiveness Thresholds which allow us to prioritize those interventions which deliver health gains at lower costs” [5.1c; A, B]. The strategic health plan notes that previous EHPs could not be implemented partly because “the cost-effectiveness threshold that was used for determining whether an intervention would be included in the EHP did not reflect the opportunity cost of health spending in Malawi” [5.1b, p. 33]. Instead, the plan cites York research (including earlier work that was subsequently published as **[A]**) as providing more appropriate benchmarks [5.1b, pp. 33-5]. As a result, the revised EHP costs 31% less than the cost of providing its predecessor package, and less per DALY averted: “while implementing the previous package would have cost USD7.91 per DALY averted, with the revised EHP it only costs USD5.97 per DALY averted” [5.1b, p. 40] . York research has thus contributed significantly to a revised EHP which “provides better value for money than the previous package” [5.1b, p. 40] .

As well as informing the allocation of Malawi’s USD247million revised EHP, York’s analytic framework and cost-effectiveness thresholds are also used by the Foreign, Commonwealth and Development Office (FCDO, formerly DFID) to guide allocation of funds in FCDO’s Umoyo-Wathu Health Systems Strengthening Programme, which provides GBP130,000,000 funding for health assistance in Malawi. The FCDO Team Leader of the Human Capital Team describes the Umoyo-Wathu programme as “firmly grounded on the EHP’ and explains that the programme ‘focuses on scaling up access and impact of the EHP, and makes use of York’s research to inform and guide investments in cost-effective health services” [5.1d].

(2) Regional adoption of the HBP work in Malawi as best practice

York’s analytic framework and cost effectiveness benchmarks developed for Malawi [A, B] subsequently influenced health policy regionally in the East, Central and Southern Africa Health Community (ECSA-HC) - the regional intergovernmental organization representing all Ministries of Health in its 9 member states (Kenya, Lesotho, Malawi, Mauritius, eSwatini, Tanzania, Uganda, Zambia, Zimbabwe). As confirmed by the Director of Programmes, ECSA-HC formally recognises the Malawian EHP as best practice [5.1e]. As he writes: “ The revised benchmarks on cost-effectiveness developed by York’s Centre for Health Economics enable more focused and effective spending, providing better value for money and improved health outcomes ” [5.1e]. In 2018, at their annual meeting, the 9 ECSA-region Ministers of Health wrote a regional Ministerial Resolution on Priority Setting and Health Benefits Package Design (HMC67/R2) [5.1f]. The resolution reflects the principles of the York research by proposing methods that reflect the realities of resource constraints, particularly the need to consider the supply side constraints to HBP implementation, and is now adopted as policy throughout the region. Following this, training on HBP design was provided by York researchers to MOH representatives and academic partners from 9 ECSA states (4-6 February, 2019, Lilongwe, Malawi). As a result, other countries in the region, including Uganda, Zambia and Kenya, have embarked upon efforts to revise their health benefits packages [5.1e]. York’s approach to HBP design has also influenced health planning in Ghana, a country of 29,000,000 people and annual health insurance expenditure of ~USD250,000,000. In 2018, the MOH commissioned an economic evaluation of the national health insurance scheme. That evaluation notes that “the research methods were inspired by research in which an EHP was developed for Malawi)” [5.2, p.14; A].

The York approach to EHP design has also been adopted as an example of best practice by the UK FCDO. As the Team Leader in the Human Capital Team at FCDO Malawi explains: “the Malawi example of the EHP is part of the FCDO Best Buys guidance to country offices, for helping develop effective health services” [5.1d]. This document is provided to all FCDO in-country health leads.

(3) Global health policy-making and prioritization

In June 2015, the WHO announced that it would stop recommending the use of 1-3 times the GDP per capita benchmark in its guidance [5.3a]. Through sustained engagement with the WHO [5.3b], research from York on the most appropriate benchmarks to use to evaluate the cost-effectiveness of health interventions in LMICs influenced this decision (earlier work, subsequently published as **[B]*). A WHO Economist at this time notes the York research was: “instrumental in shifting thinking about economic evaluation at WHO’ and ‘directly influenced a series of decisions at WHO to clarify its position about cost-effectiveness thresholds for LMICs...and enabled in turn the increased prioritization and scale-up of interventions that could deliver higher health gains for lower cost” [5.3c].

York methods research has also influenced recommendations by WHO clinical departments, especially in HIV. Our research (with the HIV Modelling Consortium, for which we lead economic analyses), has been incorporated into the last 3 HIV Treatment Guidelines and major updates [5.4a-c, 5.5a]. One study [C] argued for the adoption of a differentiated care approach to monitoring patients on HIV treatment and this was incorporated into the 2016 WHO HIV Treatment Guidelines [5.4a], leading to the approach now being available for the majority of the 17,000,000 people receiving HIV treatment worldwide. The Director of the HIV Modelling Consortium states: “The policy changes have reduced the frequency of visits required by patients to see healthcare providers, and led to increased health gains at a lower overall cost than previous treatment models used” [5.5b]. Differentiated treatment approaches have also been emphasised in the new UNAIDS HIV targets which will guide global HIV responses between 2020 and 2025 [5.6]. Additional research [D][E] has informed the 2017 WHO Advanced HIV Disease Guidelines recommendation for an enhanced package of care to those living with HIV and presenting at clinics with advanced disease [5.4b]. An independent review of the Joint Global Health Trials Scheme, funded to GBP138,800,000 in rounds 1-9, included the REALITY trial study [D] as its first example of impact and policy influence [5.7].

The Portfolio Holder for Advanced HIV Disease at WHO summarises: “The 2013 Consolidated HIV Guidelines included for the first time extensive disease modelling, cost-effectiveness analysis and emphasised a public health approach to providing care to people living with HIV...UoY researchers (had) a major input to (the) policy shift towards providing ART for all people living with HIV in 2016...UoY has continued to provide important inputs into HIV guidelines processes (including) differentiated care, enhanced prophylaxis for advanced HIV disease, and changes in drug regimens” [5.5a]. Since then, there has been a major shift in the emphasis placed on economic analysis: “ Economic analysis is now recognized as being a critical component for Guidelines development. The work of the Centre for Global Development and the international working group convened by (CGD) and co-chaired by Paul Revill ...provided WHO and other international health agencies with recommendations for the use of economic analysis in health policy formulation” [5.5a].

The approaches that York has developed to economic analysis have additionally informed investments in health technologies of the future, especially by the BMGF who fund development of many products tailored for LMICs. York researchers have assessed the likely impacts of new HIV diagnostics, in particular. One example is the research on the potential of HIV self-testing [F], if it were to come to market and become widely available. This led to investment in continued product development by the BMGF [5.5b]. The research also acted as a catalyst for further research. Most notably, research on feasibility and implementation was conducted through the HIV Self Testing in Africa (STAR) initiative, funded by UNITAID with support from the BMGF and others [5.8a]. York researchers’ work on self-testing also led to a positive recommendation in the 2016 WHO Guidelines on HIV Self Testing Supplement [5.8b], in which [F] was cited, and its availability was subsequently scaled-up [5.5b]. By July 2018, HIV self-testing was used by an estimated 4,700,000 people, with the global market expected to reach 20,000,000 annually by the end of 2020 [5.8c].