Impact case study database
Developing more effective and meaningful auditory alarms for a global medical device safety standard
1. Summary of the impact
Medical device auditory alarms are typically poor in design, and are usually hard to learn and discriminate between, difficult to locate, and are usually shrill and aversive, leading to ‘alarm fatigue’. Working closely with international standards organisations, Prof Judy Edworthy has led a project which has contributed to the update of a global medical device safety standard (IEC 60601-1-8) where cognitive science and psychoacoustic theory have driven the development of both new alarms and appropriate guidance on alarms for non-experts. Current beneficiaries are key stakeholders (standards committee members and their networks) from five continents who have served on the committees and other relevant bodies, developers, and early-moving medical device manufacturers.
2. Underpinning research
Auditory alarms research lies at the interface of auditory perception, auditory cognition and human factors research. Typically, clinical audible alarms are very poor, and because no established set of agreed metrics have historically been available to allow a person to evaluate the effectiveness and safety of an auditory alarm, there has been reluctance to change them even when they are known to be problematic. In 2015 relevant IEC and ISO committees identified high-priority issues that needed to be addressed in the next amendment to the international clinical safety standard IEC 60601-1-8, ‘Medical electrical equipment - Part 1-8: General requirements for basic safety and essential performance - Collateral Standard: General requirements, tests and guidance for alarm systems in medical electrical equipment and medical electrical systems’ which included updating the audible alarms specified within. Prof Edworthy was invited to direct this process and used it as a way not only to develop better, safer, alarms but also to show how the new alarms were developed and tested, and to provide a model as to how to approach assessing the effectiveness of audible alarms, which until now has had no commercially- or scientifically-agreed protocol.
The underpinning principle of almost all findings is that the more a sound resembles a complex, harmonically rich and meaningful sound (and the less like a harmonically poor, aversive and meaningless beep) the better it will perform as an audible alarm. Thus ‘auditory icons’ (typically complex, real-world or metaphorical sounds) are better candidates for alarm sounds than the beeps, bells buzzers and melodies typically used. A series of published studies was carried out showing the development and testing of the sounds eventually adopted in the update. Most of this research demonstrates clearly that auditory icons, or auditory-icon-style alarm sounds, perform better as alarm sounds than almost all other sounds. Aside from the earlier development studies, the auditory icons tested in the studied listed were the auditory icons now incorporated into the standard, giving them provenance in the public domain. These studies also provide a set of measurements and methods which can be used for others for developing and evaluating alarms in the future.
Learnability: The ease with which sounds can be learned is strongly influenced by the degree to which the sound and its meaning are related. Edworthy et al1 demonstrate this with respect to several sets of potential alarm sounds which were designed to have varying sound-meaning relationships. The sounds with the best learnability were auditory icons. For groups of sounds, learnability can also be influenced by the acoustic variability of the set of sounds being learned [3.2 & 3.3].This is important because many alarm sounds are typically used in one environment. McDougall et al [3.3] provides a fully factorial exploration of these two key variables (sound-meaning relationship and acoustic variability), showing that both factors influence learnability. Here, auditory icons are demonstrated to be good alarms because they have very clear meanings and are acoustically diverse.
Localizability: Using known principles from auditory perception, Edworthy et al [3.1 & 3.4] demonstrate how a person’s ability to locate a sound source is related to the harmonic structure of the sound and that more harmonically complex alarm signals such as auditory icons are easier to localize than harmonically poor tonal ‘beeps’.
Acoustic masking: Prof Edworthy has worked on an NIH-funded project with Dr Matthew Bolton (SUNY at Buffalo, US) and Dr Andrew Boyd, University of Illinois at Chicago, US, from 2016-2019. This project has developed a formal methods approach (a computer science approach) to predicting masking of audible alarms, confirmed through testing of human participants [3.5]. The results demonstrate that there is high potential for masking of one alarm by another within the previous (until 2020) IEC 60601-1-8 alarms, largely brought about by their similarity to one another. Masking is less likely if the alarms have greater variability, for example, if auditory icons are used.
Audibility: Prof Edworthy has carried out a series of studies on audibility of the best-performing sounds from the earlier studies [3.1 & 3.4] in typical Intensive Care Unit noise along with colleagues at the University of Miami, FL, US [3.6], showing that the harmonically rich and complex auditory icons are audible at very low signal-to-noise ratios. This means that they can be presented at lower loudness levels that traditional alarms.
Simulation: Bennett et al [3.6], as well as other studies, also demonstrated that auditory icons produce faster and more accurate responses from anaesthetist participants than traditional alarms, and that the use of auditory icons is less stressful and fatiguing than the tonal alarms typically used in clinical care.
All of the studies demonstrate that auditory icons (in this case the specific icons subsequently incorporated into the standard) are very good candidates as alarm signals because they are easy to learn, easy to localize, are resistant to masking, and are less fatiguing to respond to than other, more traditional, alarm signals. Because they have been published in peer-reviewed journals, the methods used to develop and test are also robust and in the public domain for all relevant stakeholders. Such attention to the provenance and science behind a set of alarms is unique.
3. References to the research
Edworthy, J., Reid, S., McDougall, S., Edworthy, J., Hall, S., Bennett, D., Khan, J, & Pye, E. (2017). The Recognizability and Localizability of Auditory Alarms: Setting Global Medical Device Standards. Human Factors, 59(7), 1108-1127 https://doi.org/10.1177%2F0018720817712004; Altmetric 59 (top 5% of all research); 20 citations (Google scholar); Field citation ratio of 7.93( = 7.93 times higher than average)
Edworthy, J., Hellier, E., Titchener, K, Naweed, A. & Roels, R (2011) Heterogeneity in alarm sets makes them easier to learn International Journal of Industrial Ergonomics, 41, 136-146
McDougall, S. J. P., Reed Edworthy, J., Sinimeri, D., Goodliffe, J., Bradley, D., & Foster, J (2019) Searching for meaning in sound: Learning and interpreting alarm sounds in visual environments. To appear in Journal of Experimental Psychology: Applied
Edworthy, J, Reid, S, Peel, K, Lock, S, Williams, J, Newbury, C, Foster, J & Farrington, M (2018) The impact of workload on the ability to localize audible alarms. Applied Ergonomics, 72, 88-93
Bolton, M. L., Zheng, X., Li, M., Edworthy, J. R., & Boyd, A. D. (2019). An experimental validation of masking in IEC 60601-1-8: 2006-compliant alarm sounds. Human Factors, 0018720819862911.
Bennett, C L, Dudaryk, R, Crenshaw, N, Edworthy, J, McNeer, R R (2019) Recommendation of new medical alarms based on audibility, identifiability, and preference in a randomized simulation based study. Critical Care Medicine,47(8), 1050-1057
4. Details of the impact
Updating the standard:
Three connected committees (AAMI 60601-1-8, IEC 60601-1-8 and a joint alarms working group), each comprising members across five continents, were tasked in 2015 with updating the standard AAMI/IEC 60601-1-8. The committees co-opted Prof Edworthy to their membership, and recognized that all alarm-related activity needed to be transparent, documented, and backed up by research to be found in the public domain, as well as requiring regular updates and presentations to the committee by Prof Edworthy, in order to win approval from all quarters. Prof Edworthy recommended quite early on in this process that auditory icons (those developed and tested in the research papers) would be the best alarms to adopt for the update of the standard. Convincing stakeholders that a change to sounds which will ultimately dramatically change the soundscape of all clinical areas, even with overwhelming evidence, required considerable time and attention.
The impacts claimed are policy change, as well as changes to understanding and process. Some parts of this project are immediately accessible to a broad range of beneficiaries, for example downloadable alarm sounds are available to all without further development from stakeholders. Other parts of the project, such as the guidance and the table discussed below, are useful for medical device companies, and known experts in this field who have now been provided with tools which provide greater understanding of the design and evaluation process.
The impacts:
- Publication of the standard
Updating a worldwide standard takes years of work, the pace of which is dictated by the various agreements, procedures, votes, and interactions between the relevant committees and authorities. The final, voting draft of the standard was voted on in July and August 2019 and resulted in favourable votes from 17/18 countries and 19/19 countries respectively across the two key alarms committees. The updated standard was then published in July 2020. The updated standard differed from the previous version in two important ways in addition to having much improved alarms: the updated alarm sounds are downloadable, and the standard contains extensive guidance and help for researchers, designers and manufacturers in terms of developing and evaluating alarms.
2. Updating and improving the audible alarms in IEC 60601-1-8
The updated alarm signals are available to download from the standard body’s website for immediate implementation by manufacturers and other stakeholders such as medical human factors and medical engineering specialists [5.1]. These are of direct benefit to manufacturers as they can now, for the first time, incorporate the alarms directly as .wav files into their equipment. In the longer term, the alarms will benefit patients, clinical staff, and guests, as the alarms are less aversive and can be played more quietly; clinicians find them more memorable and localizable, and less fatiguing, than the old alarms (all evident from the research, section 2). Prof Edworthy wrote the majority of the narrative, tables and figures in the standard relating to the acoustic and temporal specifications of the new alarms [5.2]. The website Prevention.com (upwards of 5M monthly unique users per month (October, 2020)) selected the development of the new alarms as one of its ten health breakthrough awards for 2019 [5.3].
2. Improving the audible alarm guidance in IEC 60601-1-8
For the first time, the standard contains information allowing medical device manufacturers to develop their own alarm signals, as well as their own alarm and risk categories. Annex H (pp.100-105) in the standard) is mostly written by Prof Edworthy.b A table in Annex H contains all relevant metrics concerning the alarms incorporated into the new standard, as follows:
A table (H1) delineating learnability, localizability and other performance metrics for the alarms specified, generated from the published papers, for manufacturers to compare with their own designs (as they must demonstrate comparable performance if they wish to implement their own alarms)
Detailed advice describing how in-house designs might be developed and tested for learnability, localizability, detectability, and performance in simulation, based on simplified versions of some of the procedures used in the relevant published papers
Advice as to how new or different categories of risk might be generated in a meaningful and empirical way (based on an AAMI-funded study involving Prof Edworthy, directed by Dr Melanie Wright of Trinity Health, Boise, ID, US).
The beneficiaries of this improved guidance are medical device manufacturers, human factors and auditory specialists, sound designers, testing houses, and other writers of clinical safety guidance documents and policy (see 3 and 4 below). Dave Osborn, the chair of the three interconnected committees, says ‘ This standard affects ALL medical equipment worldwide that is either patient connected or controls the flow of energy into or out a patient in all environments …Professor Edworthy has driven all of this work, from the project’s initiation to the recruitment of other key scientists in the area in order to carry out both the formative and the summative testing of the candidate alarm signals in increasingly realistic settings, such as simulated, task sharing environments. Professor Edworthy has also written significant parts of the standard’ [5.4]. In his letter he also summarises the likely long-term beneficiaries of the update to the standard, including patients.
3. Working with industry (policy change, tools, understanding, process)
Prof Edworthy has worked extensively with two medical instrument companies, Masimo Inc (based in Irvine, CA, USA) and Kestra Inc (based in Seattle, WA, USA) and has less formal connections with other companies. Masimo makes non-invasive sensors which are currently used by over 100 million patients each year e Prof Edworthy has worked with a UK sound designer (Henry Daw, London) also working with Masimo, from 2018 onwards to develop alarms in improving audible alarms which comply with the standard. They have developed a new version of the ‘general’ alarm sound specified in the standard to make it distinctive to Masimo. Nicholas Barker, VP of Design at Masimo, says ‘ Professor Edworthy’s expertise has allowed our company to prepare for the publishing of the update of IEC 60601-1-8 during the year 2020…The insights and advice Professor Edworthy have provided allow Masimo to enhance its leading role in the industry and to meet its high standards for consumer satisfaction and patient safety [5.5].
[text removed for publication] [5.6].
- Working with key designers and research teams
A key benefit of the new standard is that it allows manufacturers to develop new categories of risk and new types of alarms, and provides them with benchmarking data with which to compare their own designs, if they wish to do more than download the sounds already provided (see 1 above). Prof Edworthy has worked with various design and research groups to expedite this process. For example, a project is underway within the Human Factors group at Lisbon University, who say ‘ In our project funded by the Portuguese Science and Technology Foundation, we are designing clinical alarms with the goal of creating a library of sounds that will be made available and that ultimately would benefit manufacturers who would be able to choose them for their devices. For this we are using the IEC 60601-1-8 as design tool and its alarms as best-case scenario baseline.’[5.7].
5. Sources to corroborate the impact
- Alarm website
https://isotc.iso.org/livelink/livelink?func=ll&objId=20885884&objAction=browse&viewType=1
- Global medical device safety standard (IEC 60601-1-8)
Annex G (pp 43-47) including both the sounds described and their acoustic details as set out in Tables G. 1 and G. 2, Figures G.1 and G.2, and Table G.4. Prof Edworthy wrote most of the text of Annex H (pp. 48-53),and collated and provided the data for Table H.1 (p.49). Annex H provides guidance on testing methods that developers and testers might use in developing and testing their own alarms. This guidance is based on the methods used in the published papers detailing the development of the alarm sounds in the standard. Of the 80 references listed, 16 are co-authored by Prof Edworthy and of the 16 references cited from 2017 onwards, 10 are Prof Edworthy’s.
- Prevention.com link to health technology awards
https://www.prevention.com/health/g30198374/health-breakthrough-awards/
Letter from the Chair of the IEC 60601-1-8 committee
Letter from Nicholas Barker, VP Design, Masimo Inc
Letter from Laura Gustavson, Vice President, Kestra
Email from Joana Vieira, Lisbon University
Additional contextual information
Grant funding
| Grant number | Value of grant |
|---|---|
| 18HS024679 | £502,021 |
| n/a | £7,230 |
| n/a | £24,590 |