Impact case study database
Improving the quality of NHS Primary Care through measuring and monitoring patients’ experiences
1. Summary of the impact
The NHS needs clear, comprehensive feedback from patients to monitor and improve the performance of primary care. Exeter-led research directly informed the initial creation and the ongoing development and use of the NHS’s national GP Patient Survey (GPPS) - one of the world’s largest annual patient surveys, covering all 6,900 general practices in England. Since 2013, ten national surveys have been conducted of over 5 million NHS patients. This has directly informed national policy, changed service provision, improved service planning and the regulation/inspection of general practice, and directly informed the public and the healthcare watchdog about patient experiences of general practices. For example, The Care Quality Commission have used the survey data to target inspections of all GP practices in England and many Clinical Commissioning Groups are regularly using it within their decision making.
2. Underpinning research
In 2008, the University of Exeter was funded by the Department of Health and Ipsos MORI to develop a patient survey instrument for evaluating general practice [£600k, 2008-2010; J Campbell, joint Chief Investigator]. This collaborative research was instrumental in designing and establishing the national General Practice Patient Survey (GPPS), which has now run once or twice a year since 2011. [3.1]
The early development of the questionnaire and survey methods closely engaged the main policy and professional stakeholders: Department of Health, BMA, Royal College of General Practitioners, Royal College of Nursing and the Healthcare Commission. Four stages of cognitive testing of early versions of the survey instrument were followed by an extensive pilot survey (of 590 people) which informed both the clearer wording, structure and specific choices of the 51 questions across 11 domains of service quality, access to care and patient experience. [3.1]
Exeter also led four of the seven component studies of the ‘IMPROVE’ programme: an NIHR-funded Programme Grant for Applied Research into measuring and improving patient experience of primary care [£2 million (£929,000 to University of Exeter), 2010 to 2015; J Campbell, joint Principal Investigator]. The programme focussed primarily on two aspects of patient experience: (a) communication in consultations and (b) the ability of the patient to see a doctor of their choice, especially as a means of providing actionable information for improving service delivery [3.2].
This included analysis of the GPPS from 2011-13 which showed that the quality of doctors’ communications could not be reliably assessed at a practice-level; lower performing practices for this aspect of care might include some GPs whose communication was highly rated by their patients [3.3]. Similarly, a mixed-methods study revealed how ‘good’ patient experiences reported via questionnaires, might conceal more varied and some negative experiences when evaluated through conversation (using qualitative interviews) [3.4]**.
Another study within the IMPROVE programme of studies evaluated patient experience of out-of-hours GP services [3.5]. This research showed that relatively poor experiences of out-of-hours care were experienced by service users from Asian ethnic backgrounds, by service users who were unable to take time away from work, and when the out-of-hours service was from a commercial provider. Research using GPPS data also showed the substantial impact on quality of life of having multiple chronic conditions; those with three or more chronic conditions reported a reduction in quality of life that was greater than the sum of the reductions associated with each chronic condition when experienced alone [3.6]. Finally, the IMPROVE programme of studies has shown how the patient surveys were critically important for monitoring national trends, such as (from 2010 to 2015) the increasing difficulty people experienced in seeing a doctor of their choice. They revealed the importance of data on patient experience (and so the GPPS) as a guide to improving care, and highlighted the need for action on the quality of care for minority ethnic groups [3.2].
3. References to the research
(Exeter authors in bold text)
Campbell J, Smith P, Nissen S, Bower P, Elliott M, Roland M. The GP Patient Survey for use in primary care in the National Health Service in the UK – development and psychometric characteristics. BMC Family Practice. 2009 Dec;10(1):57. DOI: 10.1186/1471-2296-10-57
Burt J, Campbell J, Abel G, et al. (+12 other University of Exeter co-authors). Improving patient experience in primary care: a multi-method programme of research on the measurement and improvement of patient experience. Programme Grants for Applied Research (NIHR Journals Library). 2017. No. 5(9). DOI: 10.3310/pgfar05090
Roberts MJ; Campbell JL; Abel GA; Davey AF; Elmore NL; Maramba I; Carter M; Elliott MN; Roland MO; Burt JA. Understanding high and low patient experience scores in primary care: Analysis of patients’ survey data for general practices and individual doctors. 2014. BMJ (Online) 349(nov11 3):g6034. DOI: 10.1136/bmj.g6034
Burt J; Newbould J; Abel G; Elliott MN; Beckwith J; LLanwarne N; Elmore N; Davey A; Gibbons C; Campbell J et al. Investigating the meaning of ‘good’ or ‘very good’ patient evaluations of care in English general practice: A mixed methods study. 2017. BMJ Open 7(3):e014718. DOI: 10.1136/bmjopen-2016-014718
Warren F, Abel G, Lyratzopoulos G, Richards S, Barry H, Elliott M, Roland M, Campbell J. Characteristics of service user and provider organisations associated with experience of out-of-hours general practitioner care in England: population based cross sectional postal questionnaire survey. BMJ 2015; 350: h2040. DOI: https://doi.org/10.1136/bmj.h2040
Mujica-Mota RE; Roberts M; Abel G; Elliott M; Lyratzopoulos G; Roland M; Campbell J. Common patterns of morbidity and multi-morbidity and their impact on health-related quality of life: evidence from a national survey. 2015. Quality of Life Research 24(4):909-918. DOI: 10.1007/s11136-014-0820-7
4. Details of the impact
General practice underpins the globally recognised efficiency and effectiveness of UK healthcare, with UK primary care health professionals delivering 1.1 million consultations each working day. Delivering high quality primary care and improving patients’ experiences of healthcare are central goals of current UK national health policy. Research by the University of Exeter’s Primary Care Research Group has directly informed the initial creation, ongoing development, and improved use of the General Practice Patient Survey (GPPS). The survey empowers patients by enabling them to share their experience of their GP practice, and the data is publicly available, allowing them to make informed choices about their preferred GP practice. There has been continuous use of GPPS data by Parliament, the Department of Health and Social Care, and other NHS and local commissioning organisations to improve service delivery and patient experience.
[ Since the creation of the GPPS pre-dates the REF2021 impact period, we only claim impacts related to its conduct, development and use since August 2013.]
4.1 Continuous use of the National GP Patient Survey
Since August 2013, the GPPS has been conducted ten times, and analysed responses from over 5.6 million NHS patients [5.1]. It forms the basis for all six current core national metrics on patient experience of care (Department of Health and Social Care, Care Quality Commission, Public Health England). GPPS is unique in that it is now an England-wide annual survey that provides rich data for a wide range of audiences, including the public, through widely accessed websites e.g. NHS, and Public Health England. Data is reported at national, Clinical Commissioning Group (CCG), and practice levels, and informs a wide range of primary care organisational and management uses.
The research that underpins the survey is a highly valued element of the GPPS and its various versions. For example, the Director of NHS Commissioning (for England), testified [5.2] the following:
‘The evidence base published by Professor Campbell and his colleagues over the last 10 years has been invaluable in adding credibility to the outputs and impact of the survey. That credibility extends to the highest levels of the NHS and directly affects our team-decision making in respect of patient experience of care provided by NHS organisations, including all general practices in England.’
4.2 Influencing national policy
Confidence in the routine use of the GPPS is underpinned by Exeter’s research and their expertise has been used to inform the further design and use of national metrics based on its data [5.1; 5.2]. GPPS survey findings are used in Parliamentary discussions to inform policy making [5.3; 5.4]. For example, analyses of GPPS data influenced the introduction of ‘Improved Access’ arrangements, for delivering routine evening and weekend appointments in primary care [5.3; 5.4]. This policy was introduced as a direct result of the surveyed decline in patient satisfaction with patients increasingly reporting being unable to access GP appointments. As a result, the Nuffield Trust was commissioned by NHS England to investigate the impact of Improved Access on continuity of care, and analysed data from the GPPS to produce a report and make recommendations for commissioners and policymakers. The Nuffield report [5.5] drew significantly on Exeter’s research on the GP patient survey and resulted in the new primary care services. Data from the survey at GP practice level is also routinely used by the Department of Health and Social Care in evaluating practice performance across a range of policies including access to services (including out-of-hours GP services) and managing winter pressures [5.6].
4.3 Informing planning of local services
GPPS data are routinely used throughout the NHS in service planning and delivery, with local and regional health commissioning services being able to access data to meet their own patient need. The GPPS website notes the role of Exeter in developing the survey [5.6], and draws directly on, and summarises Exeter’s research. For example, the survey and Exeter’s research were used to identify that out-of-hours appointments are more commonly reported in those practices where patients report more difficulty in getting appointments during normal working hours, and that the convenience of opening hours was the strongest single predictor of use of out-of-hours care [5.6]; such data has been directly used by Clinical Commissioning Groups (CCGs) to inform improvements in service provision [5.7; 5.8]. Other routine users of the GPPS data include senior (NHS Board level) NHS commissioners and planners, noting the direct link to the research base for GPPS. Using data for service planning, GP practices and CCGs extensively use information derived from the survey to better understand and support their patients and local services [5.2].
CCGs routinely receive comparative data to monitor Outcomes Indicators and for use in the NHS Improvement and Assessment Framework. In 2019, Exeter conducted a survey of 39 English CCGs regarding their use of GPPS data - 35 CCGs (representing ~4 million patient population across 7 regions) responded and were able to provide detailed, specific examples of routine use of GPPS within their planning arrangements. Of these, 23 CCGs used their GPPS data for improving extended access schemes, 11 to inform services to better meet local need, 9 to assist in monitoring of practice performance, and 5 used the data to inform consideration of alternative methods of patient consultation [5.8].
4.4 Public use of patient-reported data on primary care
As a publicly available website [5.1] the results of the GPPS surveys are accessible to the public down to the level of individual practices. The NHS website and Public Health England’s widely used ‘General Practice Profiles’ use data from GPPS to report and provide comparative data on patient experiences of GP practices. This enables patients to make informed choices over their preferred general practice - for example, on the basis of patient-reported experience of out-of-hours care or appointment availability.
4.5 Quality monitoring and inspection of primary care practices
The GPPS provides Care Quality Commission (CQC) inspectors with data on how all 6,900 GP practices in England are performing and identifies any changes over time. The published research by the University of Exeter underpins and quality-assures this process. For example, survey data currently serves as one of four national data sources for the CQC GP Insight regulatory model – an initiative that strives to achieve one of their key priorities for their 2016-21 Strategy to deliver an ‘intelligence-driven approach to regulation’ [5.9]. Indicators developed from the GPPS are used in the GP Insight model in the inspection of every GP practice in England. Inspectors use them to guide questioning and include in the ‘evidence tables’ alongside inspection reports (for example around appointment availability, and patients feeling listened to or having sufficient time).
The UK Chief Inspector of General Practice, went further in support of the importance of the GPPS data, stating:
“Around 40% of the data behind the programme (GP Insight programme) is dependent on data derived directly from the GP Patient Survey ... It remains a key element in our inspection regime and I wanted to document my support of the underpinning academic research and resultant publications which have facilitated and supported our internal processes. ”. [5.10]
5. Sources to corroborate the impact
Annual GPPS National Summary Reports, published each July/August (2014-2020); accessed at: https://www.gp-patient.co.uk/surveysandreports
Letter from the National Director of NHS Commissioning at NHS England.
Minutes; National Primary Care Measures and Indicators Work Stream (NHS England, Public Health England) November 2016. Action to obtain advice from Campbell re use of composite GP patient survey indicator; Specific related action to revise metric on patient experience, referencing 2007 research. NHS England GP practice Indicator Review – draft indicator set and new approach for identifying outliers April 2017, subsequently reflected in NHS England National guidance (General Practice Outcome standards; technical annex https://bhcic.co.uk/wp-content/uploads/2016/09/GPOS_Technical_Annex.pdf)
Reference to GP Patient Survey in Hansard archive of House of Commons records; discussion on GP access and data from the survey reporting that 25% of patients are waiting more than a week to see their GP. The Minister for Community and Social Care (Alistair Burt).The results of the last GPPS show that 91.9% of all patients get convenient appointments. Of the 8% who are unable to get an appointment or a convenient appointment, 4.2% indicated that they went to A&E. Recommendation followed to transform GP access – Improved Access scheme. https://hansard.parliament.uk/Commons/2016-02 09/debates/16020965000017/GPAccess?highlight=improved%20access#contribution-16020965000067
Report from Nuffield Trust using data from GP patient survey with regard to GP access. https://www.nuffieldtrust.org.uk/files/2019-01/improving-access-and-continuity-in-general-practice-evidence-review-final-update-01-2019.pdf
NHS GPPS – Frequently Asked Questions: https://gp\-patient.co.uk/faq & NHS GPPS – case studies of use. https://www.gp-patient.co.uk/casestudies
The Barking and Dagenham, Havering and Redbridge (BHR) Clinical Commissioning Groups (CCGs) 2019 Improving access out-of-hours: Improving access out of hours - Evaluation of extended-hours primary care access hubs http://allcatsrgrey.org.uk/wp/download/primary_care/bhr3-report-b1881-rgb-3.pdf
Survey of 39 CCGs across 7 regions, on their use of GPPS data (Excel spreadsheet of survey results: Verbatim text responses and main uses as coded)
Care Quality Commission (2017). GP Insight: NHS GP Practices Frequently Asked Questions. https://www.cqc.org.uk/sites/default/files/201703_gpinsight_faq.pdf
Letter and specific examples of use of GPPS in the routine inspection of General Practices in England, received from UK Chief Inspector of General Practice. (Letter to Prof Ballard, re: GP Patient survey)
Additional contextual information
Grant funding
Grant number | Value of grant |
---|---|
RP-PG-0608-10050 | £929,000 |
0000 | £396,674 |
0000 | £623,000 |
0000 | £100,000 |