Impact case study database

Nature of impact

The research programme has had a major impact on the development of QOF. Changes to QOF are agreed in annual negotiations between the Department of Health (DH) and the British Medical Association (BMA), working on recommendations by the National Institute for Health and Care Excellence (NICE). Since 2009, members of the research team have directly advised NICE, instigating changes to the structure and scope of QOF, including:

- Removal of indicators: performance frameworks cannot cover all clinical areas, and achievement on indicators eventually reaches a point beyond which further improvement is not feasible and a rigorous approach to removal is therefore required. We developed key criteria on which indicator removal decisions were based and 46 indicators, accounting for over GBP403,000,000 in incentive payments, were removed from the programme between 2013 and 2018.

- Adjustment of achievement thresholds: under QOF, payments are scaled between a lower threshold, which sets the minimum level of achievement required, and an upper threshold, which indicates a high level of achievement. We developed a formula to calibrate achievement thresholds against historical performance levels, and this method for setting targets was implemented in 2013/14 for coronary heart disease indicators ( 5.1). This raised upper thresholds, and led to increases in average practice performance, for example: blood pressure and/or cholesterol was controlled for an additional 38,000 patients with coronary heart disease in England following the threshold changes ( 5.1).

Our work continued to inform development of the QOF programme, and in 2017 Tim Doran was appointed to the QOF Review Technical Working Group ( 5.2) , which was tasked with conducting a fundamental review of QOF and producing recommendations for reform or replacement of QOF ( 5.3, 5.4, 5.5). The Working Group reported in August 2018, recommending a fundamental restructuring of elements of QOF, including:

• targeting indicators at specific population segments

• introducing a personalised care adjustment to align indicators with clinical decision making and patient choice

• retiring ineffective indicators

• introducing a quality improvement domain to address clinical priority areas ( 5.4).

The review incorporated evidence from research on QOF and other physician incentive schemes conducted by a range of research groups, but most of the key evidence was derived from the work of our group ( 3.4). In the words of the Chair of the Working Group:

“The evidential review was heavily dependent on the research outputs of Professor Doran’s team, and his input to the subsequent working group discussions was instrumental in making the case for substantial changes to the framework, including the removal of ineffective quality indicators and better targeting of remaining indicators to appropriate patient groups.” ( 5.3)

As a direct consequence of the review, several changes were implemented for 2019/20 ( 5.6) representing a major reallocation of NHS resources, including: the retirement of 28 indicators (worth over GBP200,000,000 in incentive payments); the introduction of 15 new indicators (worth over GBP130,000,000) and the introduction of a new Quality Improvement domain (worth over GBP90,000,000).

The research has also achieved impact beyond academic and policy audiences, with national print ( 5.7) and broadcast ( 5.8) media using our research to raise concerns about the limited cost-effectiveness of incentive programmes. Reports of our research highlighting potential negative impacts on unincentivized aspects of care prompted responses from the Chair of the British Medical Association's GP Committee and the Chief Executive of the Patients Association pressing for reform of the incentive programme, ( 5.9) leading the Department of Health and NHS England to undertake the national QOF Review in England ( 5.2). In Scotland, the QOF was discontinued altogether in 2016.

Innovations

As QOF was implemented simultaneously across all UK practices, precluding the use of randomised trial approaches, a range of novel quasi-experimental study designs (including interrupted time series and synthetic control approaches) were developed to estimate the causal effects of the scheme. A range of novel data linkages were also required for the studies. Our approaches and linkages have subsequently been adopted by other international research groups investigating these issues. This work has also had implications beyond the investigation of incentives, and helped to legitimise the use of quasi-experimental methods in health services research. Research on QOF was also based on novel datasets that were generated to support implementation of the programme, requiring the development of new methods and definitions of quality/performance, and of extensive coding sets to enable interrogation of existing primary care clinical computing databases. The research team created the online Clinical Codes Repository to enable sharing of code lists between research groups, and this now holds 84,049 codes.