Impact case study database
A: Design and use of a rapid assessment tool that improves detection of delirium benefits patients and health services
1. Summary of the impact
Underpinning Research: To address the highly significant problem of under-diagnosis of delirium in the acute hospital setting, the Edinburgh Neuroscience team designed and validated a new clinical test for delirium detection in routine practice called the 4 ‘A’s Test (4AT).
Significance and Reach of Impact: The 4AT is recommended in 11 national and 14 international guidelines/care pathways, and is now the most-used clinical tool for delirium assessment in 80% of NHS (117/146) acute hospital trusts in the UK).
12 clinical audits all showed that adoption of the 4AT resulted in marked increases in delirium detection rates, and elicited a specific pathway to treat newly identified patients with delirium. This has led to patient benefits, including reductions in falls, behavioural issues and length of hospital stay.
Improved detection of delirium using the 4AT results in a conservatively estimated cost saving of GBP100 per patient. With 731,000 acute, emergency admissions for patients aged 75 and over per year in England alone, and 4AT used in 77% of NHS England acute hospital trusts, this can translate to NHS savings of GBP56,000,000 every year.
2. Underpinning research
The Challenge: Delirium is significantly under-diagnosed in the acute hospital setting
Delirium is an acute-onset, severe neuropsychiatric syndrome that affects 1 in 7 hospital patients. It is triggered by underlying medical illness, surgery or drugs, and results in disorientation and frightening delusions and hallucinations. Delirium is independently associated with adverse outcomes such as an increased length of stay, falls, future dementia, acceleration of existing neurodegeneration, long-term institutionalisation and mortality. Delirium detection is also crucial because it may be the only indicator of significant acute underlying illness, and of complications such as pneumonia and falls. However, with usual care, only around 1/3 of cases are formally detected.
To understand why delirium is under-diagnosed, MacLullich conducted a multicentre survey amongst 784 UK trainee doctors (the largest study of this type). Results showed that only 21% of the trainees had good knowledge of how to diagnose delirium; and only 8% reported using specific screening tools for delirium [3.1].
Development of a rapid screening tool suitable for use in routine, non-specialist care
Based on this survey, MacLullich concluded that there was a need for delirium screening tools that are brief and do not rely on formal, special training to better delirium detection in routine, non-specialist care. The 2010 National Institute for Health and Care Excellence (NICE) guidelines also emphasised the need for a delirium screening tool suitable for routine use, and recommended the short Confusion Assessment Method (CAM; first published in 1990). However, the CAM requires formal training, takes 5-10 minutes to complete, and can be challenging to implement in routine care.
Thus, to address this urgent requirement, MacLullich developed ‘the 4 ‘A’s Test’ (4AT) assessment tool in 2011 in collaboration with NHS staff. The 4AT comprises four items: a) Alertness, b) Abbreviated Mental Test 4, c) Attention (Months Backwards Test) and d) Acute Change or Fluctuating Course. It has a score range of 0-12, with a score of 4 or more denoting possible delirium. Importantly, the 4AT was specifically designed to meet the needs of routine, non-specialist care that no existing tool fulfilled:
brevity (less than 2 minutes)
no special training required
simple to score
does not require physical responses
allows for assessment of ‘untestable’ patients (who cannot undergo cognitive testing or clinical interview due to severe drowsiness or agitation)
incorporates general cognitive screening to avoid the need for separate tools for delirium and other causes of cognitive impairment.
The 4AT was first published in 2011 on a bespoke website written and hosted by MacLullich (www.the4AT.com\) [5.1].
The 4AT is highly specific and sensitive, and suitable for use in routine care
(i) Accuracy:
The first 4AT validation study, a consecutive patient study in 234 hospitalised older people, demonstrated that the 4AT had 90% sensitivity and 84% specificity compared with a reference standard delirium assessment [3.2].
A subsequent National Institute for Health Research-funded, Standards for Reporting Diagnostic Accuracy compliant validation study (n=785) in patients aged over 70 years compared the accuracy of the 4AT in delirium diagnoses with the CAM, finding that the 4AT and CAM showed similar specificity (94% vs. 100%, respectively), but the 4AT showed a significantly higher sensitivity than the CAM (76% vs. 40%) [3.3, 3.4]. Importantly, a positive 4AT score accurately predicted two key negative outcomes associated with delirium: longer length of stay in hospital and increased mortality [3.4].
EN researchers also conducted a systematic review and meta-analysis of 17 4AT validation studies (3702 observations), and found a pooled sensitivity of 88% and a pooled specificity of 88% [3.5]. In terms of numbers of observations, the 4AT has now displaced the CAM as having the most published formal diagnostic test accuracy data of any delirium tool in the literature.
(ii) Suitability for use in routine care:
The EN team conducted a survey of 100 doctors and nurses worldwide who had experience of using the 4AT, and found that 95% agreed that routine use of 4AT in their unit was feasible, with many specifically referring to its ease (39%) and speed (29%) of use [3.4].
A health economic analysis embedded in the same study showed that while a diagnosis of delirium was associated with higher costs than for patients with no delirium, a wrongly missed diagnosis was estimated to cost GBP2,067 more per patient than a true positive. Given that the 4AT was significantly more sensitive than the CAM, using the 4AT as the assessment tool was shown to incur lower healthcare costs overall, thus supporting the use of the 4AT in the NHS [3.4].
3. References to the research
[3.1] Davis D & MacLullich A. Understanding barriers to delirium care: a multicentre survey of knowledge and attitudes amongst UK junior doctors. Age and Ageing 2009; 38:559-563
[3.2] Bellelli G, Morandi A, Davis DH, Mazzola P, Turco R, Gentile S, Ryan T, Cash H, Guerini F, Torpilliesi T, Del Santo F, Trabucchi M, Annoni G, MacLullich AM. Validation of the 4AT, a new instrument for rapid delirium screening: a study in 234 hospitalised older people. Age Ageing. 2014 Jul;43(4):496-502. doi: 10.1093/ageing/afu021
[3.3] Shenkin SD, Fox C, Godfrey M, Siddiqi N, Goodacre S, Young J, Anand A, Gray A, Hanley J, MacRaild A, Steven J, Black PL, Tieges Z, Boyd J, Stephen J, Weir CJ, MacLullich AMJ. Delirium detection in older acute medical inpatients: a multicentre prospective comparative diagnostic test accuracy study of the 4AT and the confusion assessment method. BMC Medicine 2019; 17:138. doi: 10.1186/s12916-019-1367-9
[3.4] MacLullich A, Shenkin S, Goodacre S, Godfrey M, Hanley J, Stiobhairt A, Lavender E, Boyd J, Stephen J, Weir C, MacRaild A, Steven J, Black P, Diernberger K, Hall P, Tieges Z, Fox C, Anand A, Young J, Siddiqi N, Gray A. The 4 ‘A’s Test for detecting delirium in acute medical patients: a diagnostic accuracy study. Health Technol Assess 2019; 23(40). doi: 10.3310/hta23400
[3.5] Tieges Z, MacLullich AMJ, Anand A, Brookes C, Cassarino M, O’Connor M, Ryan D, Saller T, Arora RC, Chang Y, Agarwal K, Taffet G, Quinn T, Shenkin SD, Galvin R. Diagnostic accuracy of the 4AT for delirium detection in older adults: systematic review and meta-analysis. Age Ageing 2020; afaa24. doi: 10.1093/ageing/afaa224
4. Details of the impact
Impact on policy for identification and treatment of delirium
The 4AT has a substantial presence in guidelines and care pathways, both national (11) and international (14). The 4AT is the assessment tool of choice in 18 guidelines, including the 2020 Royal College of Physicians guidelines for standardising the assessment of acute illness severity, NICE-accredited Scottish Intercollegiate Guidelines Network delirium guidelines 2019, New Zealand Delirium Capability Guidelines 2018 and Norwegian guidelines for multidisciplinary treatment of hip fractures 2018. The 4AT is also included amongst the recommended tools in a further 7 guidelines including the European Society of Anaesthesiology guideline for postoperative delirium 2017 and the Australian Delirium Clinical Care Standard 2016. These all cite the original paper [3.2] or the 4AT website [5.1], and emphasise the importance of screening for delirium as a first step in a patient management and treatment plan. All 25 guidelines are listed in [5.2].
Notably, in July 2020, NICE issued an Exceptional Surveillance Report stating that, because the EN-led study demonstrated similar specificity and higher sensitivity of the 4AT compared with the CAM [3.4], their recommendation on delirium assessment would be revised [5.3].
Impact on Covid-19 policy
Because delirium is very common in Covid-19, in March 2020, the British Geriatrics Society, European Delirium Association and Royal College of Psychiatrists authored ‘Coronavirus: Managing delirium in confirmed and suspected cases’. The 4AT is the recommended assessment tool [5.4].
Impact on clinical practice
The 4AT is now the primary tool for delirium assessment in the NHS. In July 2020, replies to a UK-wide Freedom of Information request from 154 NHS acute hospital trusts showed that, of the 146 trusts who had a formal screening process in clinical practice outside the intensive care unit, 80% (117/146) use the 4AT tool to screen for delirium. This equates to 77% of NHS England trusts, 100% of trusts in both Scotland and Wales, and 80% of trusts in Northern Ireland. In most of those hospital trusts, 4AT is the only tool used (77/117), with the remaining trusts using 4AT alongside other tools. Thus, despite the (soon to be reconsidered) 2010 NICE recommendations to use the CAM, the 4AT is now used more in clinical practice than the CAM. The latter is only used in 45% (65/146) of hospital trusts, and the majority of those (39/65) use the CAM alongside other delirium assessment tools [5.5].
Use of the 4AT is particularly common in management of patients with hip fracture. Given that delirium is the most common complication of surgery and anaesthesia in older people, the National Hip Fracture Database has, since 2017, mandated use of the 4AT as part of the Best Practice Tariff for patients undergoing surgery after hip fracture. Their Annual Report 2018 found that delirium assessment with the 4AT was carried out in 90% of all eligible patients (95.3% in England) from 65,958 admissions. The 4AT identified 24.9% (14,781) patients as having ’possible delirium’ [5.6], showing appropriate levels of clinical detection.
International uptake of the 4AT is growing rapidly. An indirect indicator of its reach is the 4AT website, launched in June 2011. Accesses to the site have increased rapidly; from 6,995 users and 20,381 page views in 2014 to 77,629 users and 156,684 views in 2020. The top 5 user countries are the UK, Australia, USA, Ireland and Canada [5.1]. Evidence of international clinical uptake is exemplified by the Australian and New Zealand Hip Fracture Registry Annual Report 2020, which shows that the 4AT is the most common tool used for delirium assessment in hip fracture patients - used by 43% (50/117) hospitals; the CAM by 33% (39/117) [5.7]. The 4AT has been translated into 17 languages with affirmative validation studies published for the French, German, Iranian, Norwegian and Thai versions [all validation studies listed in 5.8]. In addition, Arabic, Chinese, Danish, Dutch, Finnish, Icelandic, Italian, Korean, Polish, Russian, Spanish and Turkish versions have been created by clinicians, after seeking permission from MacLullich [5.1].
Impact on health and welfare
Detection of delirium using 4AT in the acute clinical setting is the critical first step in unlocking a delirium management pathway, for example the TIME bundle (Think Investigate Manage Engage; detailed in [5.2r]). 12 published clinical audits, undertaken between October 2013 and 2020 from diverse clinical settings, have all demonstrated that adoption of the 4AT significantly increases rates of delirium screening [all audits listed in 5.9]. For example, a 2018 quality improvement study in York Teaching Hospital acute medical unit introduced the 4AT to ensure that patients with delirium were being identified. Assessment of delirium increased from 0% to 64%, with a subsequent shift increase in diagnoses [5.9a]. 7 of these 12 audits specified that a diagnosis of delirium using the 4AT triggered the use of a delirium management pathway, the most common being TIME.
This increasing attention to delirium screening is leading to increasing rates of delirium detection in the UK. Data from Public Health Scotland show increased rates of formal coding of delirium in hospital administrative records, reflecting increased clinical detection rates, between 2013 and 2020. For example, in the 85-89 age group, delirium was coded in 2.8% of cases in 2012-2013, compared with 10.5% cases in 2019-2020, with steady increments in between. Similarly, Public Health England data shows that in the 85+ age group, delirium was coded in 2.1% in 2012-2013, rising to 8.3% in 2019-2020. The parallel increase in UK 4AT use (as evidenced by the Freedom Of Information report and website accesses as above) suggests that 4AT implementation is likely to have been a factor in these improvements [5.10].
Triggering a formal delirium management pathway results in clear clinical benefits for patients. The TIME pathway, for example, includes explicit instructions to clinically assess and treat potential underlying causes of delirium, such as sepsis or drug intoxication, as well as delirium symptoms (such as unmanageable agitation), thereby reducing associated adverse outcomes.
These improved patient outcomes are exemplified in a 2016 report ‘Improving Older Peoples’ Acute Care’, where rapid identification of delirium was a focus to improve care. This revealed that when 4AT assessment on admission was introduced in an acute ward in NHS Glasgow, a 50% reduction in falls was seen over a year compared with the baseline median [5.9b].
Another example of better patient outcomes following the rapid detection of delirium using 4AT is seen in a 2019 Australian study, where the tool was introduced to improve recognition and management of delirium in high-risk wards. Over a 3-month period, 88% of patients were screened, resulting in delirium diagnosis increasing from 8.2% to 19.4%. This was accompanied by a 48% reduction in behavioural response calls, reduced severity of falls, and length of stay was reduced by 0.8 days [5.9c].
Importantly, increased rates of delirium detection, enabled by adoption of the 4AT, are improving older people’s experience of care as a whole; several case studies published in a 2015 Health Improvement Scotland impact report describe not only better management of delirium, but also increased attention to other elements of the TIME pathway, including pain management and nutrition [5.9j]. Similarly, the 4AT was incorporated into a falls care bundle in NHS Glasgow, allowing staff to target patients at greater risk of falls [5.9j], and a project undertaken by NHS Highland demonstrated that quicker diagnosis of delirium, achieved following implementation of the 4AT, led to more timely treatment and referral to the older adult mental health team where necessary [5.9j].
The transformative impact of the 4AT on the health and welfare of delirium patients is summarised by comments made by a prominent Professor of Old Age Psychiatry and a Trustee of Alzheimer’s Research UK: “ The 4’A’s Test is already considered the gold standard in practice for the detection of delirium, and rightly so […] The consequence of missed delirium – both for individual patients and the NHS – are serious and expensive. Anything that increases delirium awareness and improves detection will have clear benefits” [5.11].
Economic Impact
Health economic modelling quantified the savings associated with the use of the 4AT and the consequent reductions in missed diagnoses. This suggests a conservatively estimated saving of approximately GBP100 for each patient aged over 70 admitted to an emergency acute medical or geriatric department, assessed for delirium using the 4AT rather than clinical judgement alone [5.12]. With 731,000 emergency admissions of 72 hours or more for patients over the age of 75 per year in NHS England alone (2018/19), there is ample opportunity to make this saving [5.12]. With 4AT currently used in 77% of NHS England acute hospital trusts [5.5], this would translate to NHS savings of GBP56,000,000 every year.
5. Sources to corroborate the impact
[5.1] The 4AT website and access figures
[5.2] All guidelines and care pathways recommending use of 4AT to screen for delirium.
[5.3] NICE guideline CG103: 2020 exceptional surveillance of delirium: prevention, diagnosis and management
[5.4] Coronavirus: Managing delirium in confirmed and suspected cases; p.2
[5.5 Formal Freedom of Information report showing use of 4AT in clinical practice in the UK
[5.6] National Hip Fracture Database Annual Report September 2018; pages 22 & 23
[5.7] Australian and New Zealand Hip Fracture Registry Annual Report 2020; page 81
[5.8] Validation studies performed on translated versions of the 4AT
[5.9] Audits and annual reports showing impact of 4AT on health and welfare
[5.10] Delirium diagnoses in patient discharge summaries
[5.11] Expert commentary on the importance of the 4AT for delirium patients and the NHS
[5.12] Formal Health Economic Report of the Potential NHS Cost Savings from Improved Detection of Delirium using the 4AT vs Usual Care
Additional contextual information
Grant funding
Grant number | Value of grant |
---|---|
HTA 11/143/01 | £637,354 |
IS3-T06/03 | £5,000 |