Skip to main

Impact case study database

The impact case study database allows you to browse and search for impact case studies submitted to the REF 2021. Use the search and filters below to find the impact case studies you are looking for.
Waiting for server

ICS-8 Enhancing the health care experience for patients, families and nurses using eight key performance indicators

1. Summary of the impact

This programme of research, led by Ulster University, impacts on the health care experience for patients, families and nurses (beneficiaries) through the implementation of eight key performance indicators (KPIs), thereby addressing a deficit in how the quality of nursing care is measured. Implementing the KPIs generates data that nursing teams can use to critically evaluate the patient experience and make improvements to person-centred practice. The reach has extended across a range of clinical specialities, spanning the United Kingdom, Europe and Australia.

Impact can be evidenced in three areas:

I1 enhancement of the health care experience for patients and families;

I2 enhancement of the health care experience for nurses;

I3 influencing strategy and policy.

2. Underpinning research

The underpinning research comprises: (i) our original study that led to development of the key performance indicators (KPIs) for nursing and midwifery and accompanying measurement tools; and (ii) a series of implementation research studies that have evaluated the use of these KPIs in a range of clinical settings across the UK, Europe and Australia.

The original research to develop KPIs

The aim of the original research (2012) was to gain consensus from a range of key stakeholders on KPIs that were appropriate and relevant for nursing and midwifery practice. This study adopted a nominal group technique that resulted in identification of eight top-ranked indicators and measurement tools considered core for nursing and midwifery practice ( R1). The KPIs are presented below.

KPI 1: Consistent delivery of nursing care against identified need

KPI 2: Patient confidence in the knowledge and skills of the nurse

KPI 3: Patient sense of safety whilst under the care of the nurse

KPI 4: Patient involvement in decisions made about his/her nursing care

KPI 5: Time spent by nurses with the patient

KPI 6: Respect from the nurse for patient’s preference and choice

KPI 7: Nurse’s support for patients to care for themselves where appropriate

KPI 8: Nurse’s understanding of what is important to the patient and their family

Uniqueness of the KPIs to nursing and midwifery

The eight indicators identified were considered novel in that they: (i) did not conform to the majority of other nursing metrics generally reported in the international literature; (ii) were strategically aligned to work on improving patient experience; and (iii) measured person-centred practice. Person-centredness is an approach to practice that is underpinned by the importance of therapeutic relationships among professionals, patients and others significant to them in their lives. At the heart of the KPIs is the unique contribution of nursing and ultimately its impact on patient outcomes.

The measurement tools

The measurement tools accompanying the KPIs comprised: a patient survey; a tool to observe time spent with patients; patient and family stories; and a review of the patient record undertaken in conjunction with nurse interviews. A feasibility study was undertaken to test the eight KPIs and measurement tools as an approach to evidencing the patient experience involving nine practice settings in three participating organisations across the United Kingdom and Republic of Ireland (2015). The findings revealed that the data generated by the KPIs did accurately evidence the patient experience and that it also provided feedback to nurses and midwives that informed the development of person-centred practice (R2).

Implementation research studies

The eight KPIs and measurement tools have been tested in a series of international implementation research studies in a range of clinical settings led by Ulster University. The Paediatric International Nursing Study (PINS) involved acute paediatric inpatients from 12 organisations across Europe and Australia (2015). The aim was to explore factors that influenced the successful implementation of the eight KPIs, and how the evidence generated impacted on person-centred practice across a range of services provided to sick children. This study confirmed that using the KPIs generated data that engaged nursing teams in the critical evaluation of the patient experience to make changes in practice, for example, more effective communication with children and their families, thus enhancing care experience (R3). The evolution of the research led to a further study to develop and test the feasibility of an app (iMPAKT) on a mobile device, in collaboration with School of Computing (2016). This technology enabled the gathering of different data sources into one platform, using the measurement tools. The study demonstrated that the prototype iMPAKT app made information more accessible, captured it in real-time and enabled it to be used more easily to improve the experience of care (R4). The app was developed further for use with community nursing teams across Scotland and Northern Ireland (2018), highlighting its transferability into a community context (R5). Drawing on outcomes from previous studies, further research was undertaken to explore the impact of implementing the eight KPIs to support the development of person-centred care across ambulatory chemotherapy units in Northern Ireland (2019). The findings demonstrated that a collaborative approach to implementing the KPIs was directly correlated with enhancement of the health care experience for patients and nurses, for example in relation to shared decision making (R6).

3. References to the research

Outputs can be provided by Ulster University on request.

The following outputs have been blind peer reviewed by internationally-based editorial boards.

R1 = McCance TV, Telford L, Wilson J, MacLeod O & Dowd A (2012) Identifying key performance indicators for nursing and midwifery care using a consensus approach. Journal of Clinical Nursing, 21(7 & 8): 1145-1154.

R2 = McCance T, Hastings J & Dowler H. (2015) Evaluating the use of key performance indicators to evidence the patient experience. Journal of Clinical Nursing, 24(21-22): 3084-3094.

R3 = McCance T, Wilson V & Kornman K (2016) Paediatric International Nursing Study: using person-centred key performance indicators to benchmark children’s services. Journal of Clinical Nursing, 25(13-14): DOI: 10.1111/jocn.13232.

R4 = McCance T, Lynch B, Boomer C, Brown D, Nugent C, Ennis A, Garcia-Constantino M, Cleland I, Edgar D, Radbron E & Wilson V. (2020) Implementing and Measuring Person-centredness using an APP for Knowledge Transfer: the iMPAKT App. International Journal of Quality in Healthcare, doi: 10.1093/intqhc/mzaa018.

R5 = McCance T, Dickson C, Daly L, Boomer C, Brown D, Lynch B, MacArthur J, Mountain K & McCormack B (2020) Implementing person-centred key performance indicators to strengthen leadership in community nursing: a feasibility study. Journal of Nursing Management, doi: 10.1111/jonm.13107.

R6 = McCance T, Lynch B, Nevin L & (2020) Co-producing and Implementing Person-centred Key Performance Indicators in Cancer Nursing (CIP-CAN). Commissioned report.

Peer reviewed funding associated with the underpinning research is summarised below.

Title Funder Researcher Dates Amount
Co-producing and Implementing Person-centred Key Performance Indicators in CAncer Nursing (CIP-CAN) Macmillan Cancer and Public Health Agency, Northern Ireland CI: T McCance 2018-2019 GBP74,423 GBP28,119
Strengthening Leadership in Community Nursing Teams (SLICC) Burdett Trust for Nursing Joint CIs: T McCance and B McCormack 2018-2019 GBP40,485
iMPAKT: Implementing and Measuring Person-centredness using an App for Knowledge Transfer HSC R&D Division, Public Health Agency, Northern Ireland CI: T McCance 2016-2019 GBP72,391
Key Performance Indicators for Nursing & Midwifery Department of Health, Social Services and Public Safety, Northern Ireland CI: T McCance 2009-2010 GBP48,300
Total Funding GBP263,718

4. Details of the impact

The purpose of this programme of work was to evaluate the impact of implementing the KPIs on the experience of care for patients, families and staff across different clinical contexts. Impacts I1 and I2 are interconnected within the implementation studies and are described simultaneously.

Enhancement of the health care experience for patients and families and for nurses

Through the Paediatric International Nursing Study (PINS), the eight KPIs were implemented across a range of sick children’s services (2013-2015) (R3). It involved 6 organisations in Australia (Princess Margaret Hospital for Children, Perth; Royal North Shore Hospital, NSW; Hornsby Hospital, NSW; Nepean Blue Mountains Local Health District, NSW; Women's and Children's Hospital, Adelaide; and the Sydney Children’s Hospitals Network) and 6 in Europe (Hans Christian Andersen Children's Hospital, Odense, Denmark; Temple Street Children's University Hospital, Dublin; Great Ormond Street Hospital, London, England; East Kent, England; Royal Belfast Hospital for Sick Children, Northern Ireland; and South Eastern Trust, Northern Ireland). Participating sites confirmed the value of the data captured by the KPIs as a means of highlighting “ the positive and excellent care that is provided and is recognised and appreciated by families”, whilst at the same time being able to act on “ immediate negative feedback that identifies people or interdisciplinary services and requires attention i.e. is unsafe or disrespectful(I1, C1). It was this that drove practice change by providing meaningful information, permitting nurses to pinpoint directly what specific areas of practice needed improved (I1, C1, C2). Improvements focused on areas such as nursing documentation, bedside handovers (I1, C3) and supporting information for families (I1, C2). Similarly, there was evidence that using the KPIs embraced the value of nursing, impacting on staff morale and providing opportunities to celebrate good practice (I2, C1, C2).

As a result of PINS (R3), there was organisational roll out by the Women’s and Children’s Health Network in South Australia across 4 divisions, covering 8 nursing and midwifery teams (I1, C1). The KPI data collected from 2013-2020 had significant reach involving approximately 1,285 patients and families completing surveys or stories, resulting in approximately 80 action cycles (I1, C1). Examples include: allocation guidelines of patients to staff to improve consistency in care, and development of patient profiles for complex care so families don’t have to repeat their stories (I1, C1). Staff engagement in this process “ raise[d] awareness of the impact of care, what person-centred care looks like and what works and doesn’t work(I2, C1).

Development of the iMPAKT App (Implementing and Measuring Person-centredness using an APP for Knowledge Transfer) offered a technological solution for collecting the KPI data and was evaluated as an effective method to measure the patient experience in real time (R4). Nurses (n=22) participated from a range of clinical contexts (5 sites in South Eastern Health & Social Care Trust, Northern Ireland and 6 sites in Illawarra Shoalhaven Local Health District, Australia). Illawarra Shoalhaven Local Health District has continued to use the iMPAKT app, engaging a total of 177 nursing and midwifery staff across a range of clinical settings. The number of beneficiaries totalled 677 patients and carers (C5). The impact on nursing practice included: an improved team focus approach to care delivery; changes to staffing allocations and rostering to enhance consistent care; development of person-centred nursing documentation; and improvement in visibility of nurses for patients (I1, C5). The positive impact on staff of using the KPI data to improve practice was also emphasised, with the introduction of regular staff celebrations. One unit also developed an employee of the month programme. Other staff benefits included: provision of clinical supervision and reflection; and increased opportunities to de-brief following challenging situations (I2, C5).

The KPIs have also been implemented by NHS Lothian, Scotland, as part of an organisational care assurance programme between 2017 and 2019. Across 3 hospital sites, 31 acute wards used the KPIs, with 7,940 patients completing the survey and 253 patients providing stories (C6). Clinical teams were considered the main beneficiaries, with the reporting of improved staff morale (I2, C6). Patient story feedback was also key in improving practice, such as changing ways of working and addressing long standing environment issues (I1, C6). Use of the KPIs within NHS Lothian led to further collaboration with South Eastern Health & Social Care Trust in Northern Ireland to test the implementation of the KPIs within a community nursing context (R5).

The significance of the original research (R1, R2) resulted in the implementation of the KPIs within chemotherapy units (n=5), across the entire region of Northern Ireland from 2018 to 2019 (R6). A key impact was nurses’ ability to truly involve patients in decisions about their care and the things that really mattered to them in their daily lives (I1, C7). This influenced a significant regional change in the electronic documentation system used in chemotherapy environments. There was no area in the documentation where nursing staff could capture what was currently important to the patient and their family, leading to significant inconsistency between what the nurse said was important to the patient and a review of the patient documentation. Consequently, the restrictive dropdown menu to record what was important to the patient was expanded with a free text option to ensure what is important to the patient was accurately captured and evidenced as an ongoing dialogue (I1, C8). The impact was also significant for chemotherapy nurses and influenced their ability to clearly articulate their role and enhance their contribution to the patient experience (I2, C7). Nursing teams felt disempowered to reduce waiting times for patients, but the KPI data encouraged them to think differently about this issue and as a result developed person-centred interventions. It was “ small simple changes that can play a big part of a patient’s experience and journey”, with examples including: offering complimentary therapies to reduce stress and anxiety and to facilitate relaxation; working with the catering department to install a fridge in the unit to provide patients with fresh cool bottled water; and developing a post-discharge pack to provide a range of information to enhance experience of patients after they have finished their chemotherapy treatment. These interventions led to an enhanced patient experience (I1, C7).

Influencing strategy and policy

Finally, there is evidence of the programme of work influencing strategy and policy. NHS Lothian, Scotland, illustrated this through the impact of the work on the Board’s Patient Experience Strategy (I3, C6). Similarly, within Illawarra Shoalhaven, Australia, the KPIs aligned with the Nursing and Midwifery Workforce Plan and “ provided a link between policy and practice(I3, C5). Within Northern Ireland the KPIs are being used to inform policy for nursing through the Northern Ireland Cancer Strategy and the Nursing and Midwifery Task Group (I3, C9). Further implementation of the KPIs on an international stage is evidenced by the establishment of a Chief Nursing Officers’ Collaborative between Australia and Northern Ireland to further develop the iMPAKT app for large scale use (I3, C10).

5. Sources to corroborate the impact

C1: Testimonial from Advanced Nurse Consultant, Women and Children’s Health Network, South Australia.

C2: Testimonial from former Senior Nurse Manager, South Eastern Health and Social Care

Trust, Northern Ireland.

C3: Audit reports for implementing bedside handovers from two areas within Women and Children’s Health Network, South Australia.

C4: Poster presentation and oral presentation from Women and Children’s Health Network, South Australia.

C5: Testimonial from Executive Director of Nursing & Midwifery, Illawarra Shoalhaven Local Health District, New South Wales, Australia.

C6: Testimonial from Chief Nurse Research & Development, NHS Lothian, Scotland.

C7: Two programme participant video testimonials from the project Co-producing and Implementing Person-centred Key Performance Indicators in Cancer Nursing (CIP-CAN)

(from (i) Nurse Champion and Peer Facilitator (person with lived experience) in the

Belfast Health & Social Care Trust, NI, and (ii) Nurse Champion in the Southern

Health & Social Care Trust, NI).

C8: Testimonial from Macmillan Service Improvement Manager, Northern Health & Social Care Trust, Northern Ireland.

C9: Testimonial from Chief Nursing Officer, Department of Health, Northern Ireland.

C10: Case for funding for CNO Collaborative and associated email from Chief Nurse,

DOH (NI).

Additional contextual information

Grant funding

Grant number Value of grant
6507179; 6507195 £74,423
CIP-CAN £28,119
n/a £40,485
RES/5198/15 £72,391
n/a £48,300