Impact case study database

4.1 Planned, Causal, Internal Clinical Impact

In our research/impact model, we recruited 28 child clients aged 6-14 to two clinical research projects, treated in 1 hour therapy sessions in a 10-12 week block. Most of the clinical impact arising during the census period occurred within the Ultraphonix project (2015-2016), and is corroborated by our underpinning research output [5], so we focus here on that Case Series.

15 children recruited for Ultraphonix were suitable for and completed our protocol of motor-based therapy with ultrasound visual biofeedback [5]. Overall, they improved speech intelligibility (rated holistically by their parents, see below). Intelligibility improved because our research interventions identified, targeted and successfully resolved errors in key consonants or vowels that had been persistent and unresolved, to a clinically significant degree for 10 of these children, as quantified by effect-size analysis [5].

Audible improvements in speech sounds above stable baseline (BL) can be shown as “percent treated targets correct (% TTC)”. Figure 1 charts the mean improvement of all the 15 children. Accuracy increased from below 5% TTC at baseline to 60% TTC in maintenance assessments (3 months post-intervention), as measured in clinic using probes, which, because they comprised words that had not been used in treatment, show both generalisation and retention [5-6].

Figure 1. Mean increase in Treated Targets Correct in Ultraphonix Project (+/- 1 s.d.)

Generally [5], our intervention protocol (a) was successful, (b) in a wide variety of targeted speech sounds, (c) was relatively rapid, and (d) speech continued to improve after treatment ended (Figure 1). Wide individual differences in rates and levels of improvement were found, suggesting children with comorbidities might benefit from higher dosage [5].

As noted, holistic improvements to the children’s intelligibility were reported by parents, post-intervention. These were quantified via the “Intelligibility in Context Scale” (ICS) [5]. Children improved from being “sometimes understood” to “usually understood” [5] outwith clinic. This improved intelligibility is permanent, and improves the quality of life of the children [s1].

ULTRAX (2011-2014) and Ultraphonix (2015-2016) plus other treatment by our PhD students and clinical research staff were the first using UTI in Europe. In a systematic review of all 29 published intervention-based research studies worldwide 1985-2017 [s2], ULTRAX [6] was one of 19 studies with positive outputs for all its participants. Like Ultraphonix, but unlike most previous studies [6, 8], [s8], ULTRAX targeted a diverse range of speech sounds (vs. the focus on consonant /r/ in North American research). ULTRAX had 8 child participants, the second highest enrolment to 2017. However, only three ULTRAX children were treated during the impact census period, so we do not highlight the project as a whole here, beyond noting aspects that apply to all the participants. Moreover, Ultraphonix had more participants, some with comorbidities, and a more diverse set of targets. Also, we noted above that materials from both projects fed back, cyclically, into our underpinning (and other) research [6], and that research then generated more impact (see below).

Longer term, increased accuracy and intelligibility can bring long-lasting and deep social, personal and educational benefits, augmenting the evidence of %TTC and ICS in [4, 5]. For example, client A (treated in ULTRAX before Aug 2013) sent an unsolicited email in 2018 illustrating the long-lasting nature of the impact (i.e. after August 2013):

“*the … speech therapy at Queen Margaret University helped me more than nine years of normal speech therapy through the schools and the doctors. Having this experience helped because it made me understand the movements of my tongue through looking at the scans. This experience gave me so much more confidence and really helped me get from bottom of almost all of my classes in primary school and really struggling. It helped me moved up in my classes and not feel so shy.*”(sic) [s1]

4.2 Early Stage External Clinical Impact

[s3]: A paediatric NHS Speech and Language Therapy service piloted a specialist clinical provision 2015-2016, adapting our clinical protocols [4, 5] for 12 children with varied disorders. Beneficial impacts were (and see also **[s10]**):

1. Confirming diagnosis of articulatory difficulty in terms of tongue placement

2. Supporting children and young people to understand what they needed to change in terms of their tongue placement

3. Helping families to understand their children’s articulatory difficulties and treatment

4. Being a useful motivator for some children

[s4] An external EPSRC research project (ULTRAX-2020) has been extending our underpinning research. It incorporated clinical intervention which we cite as impact. It shows “ that by using ultrasound to provide visual feedback of the tongue in real-time, children can learn to produce speech sounds which have previously been impossible for them”. Due to COVID, planned collaborations with clinicians were cancelled or severely curtailed in 2020.

[s5] ULTRAX-2020 adopted clinical protocols and norms from [4-5], disseminating an open access Clinicians' Resource Manual in 2018. This free-to-use 87 page clinical manual from SLTs and clinical researchers on the ULTRAX-2020 project presents our underpinning research in a format tailored for practical clinical use, including our wordlists and protocols. [s6] notes for those clients “whose errors persist despite treatment”, our protocols succeed, and do so for a wider variety of targets than other clinical research groups have addressed.

4.3 Engineering, Educational and Open Science Resources

[s7] Our ULTRAX and Ultraphonix data (annotated and labelled) was adopted as the initial content for “UltraSuite” (2018), an open curated repository for machine learning engineers. Our contribution comprised over 13 hours of speech from 76 children, of which the clinical speech component (from before and during treatment) is the largest articulatory corpus worldwide.

Our underpinning research [3] was used for a website [s8] that is used for phonetic courses, self-learning, and continuing professional development for SLTs [s9]. For example, [s10] notes that [s8] is used by clinicians to “develop the child’s and parents’ knowledge of how particular sounds

are made”, enhancing therapeutic intervention and providing motivation. [s8] is itself the subject of an impact case study from the University of Glasgow, which notes that 10% of over 250,000 users worldwide are SLTs (along with students, the public, second language learning community), and that 99% of users rate it useful.

1. International impact on professional services and professional education: voice analysis in Forensic Phonetics

In evaluating the probability that two or more voice samples are from the same speaker, forensic phoneticians use a multistranded suite of acoustic and perceptual analysis tools. Where acoustic quality of audio recordings used as evidence in court is inadequate for reliable acoustic analysis, perceptual tools (e.g. VPA) become especially important. VPA training based on our research [1] delivered to three key groups of forensic phoneticians as a pathway to impact (JP French Associates (JPFA) + University of York (UoY); Pontifícia Universidade Católica de São Paulo (PUC-SP) + Brazilian Federal Police; and the Netherlands Forensics Institute) generated impact in several ways.

JP French Associates, one of the world’s leading forensic speech laboratories, first adopted our VPA framework in 2010 and continued to do so through the census period, creating a streamlined version specifically for forensic purposes [s1]; this version is now (2020) used by at least two other UK forensic experts/firms. Peter French (founder of JPFA) writes [s2]:

“…. in 2009 we decided that the VPA was the most phonetically-grounded and detailed tool available for voice quality analysis and description……. On the basis of her (Beck’s) training, it became clear that our adopting the VPA would enhance the reliability, replicability and transparency of the tests that we apply when comparing recorded voice samples…… I would estimate the number of cases in which we have used it to be in excess of 1,500. It now [i.e. during the census period] forms a significant element of the forensic speaker comparison procedure and has been critical in some cases in arriving at a positive or negative conclusion. The use of the modified VPA is mentioned in all official JPFA forensic speaker comparison reports, and it has been frequently mentioned in oral evidence delivered from the witness box in trials also.”

A recent report on rigorous testing of VPA included the statement “ We further conclude that the perceptual assessment of voice quality using the VPA scheme is an essential tool in fields such as forensic phonetics” [s3].

VPA is now a core part of the curriculum for UoY’s undergraduate and postgraduate forensic phonetics programmes. VPA is also included in UoY’s CPD course in Forensic Speech and Audio Analysis. Professional groups from six continents, including police, detectives, government employees, acousticians, forensic scientists in other disciplines and software engineers, have participated in the six iterations of this course, influencing forensic casework in the UK and internationally.

In Brazil a Portuguese translation of VPA (“BP-VPAS”) is routinely used by Federal Police in forensic reports (~150 reports per year). Federal Police training in forensic science includes a module on VPA facilitated by PUC-SP and police from regional states confirm its value as an investigative tool [s4]. In December 2020 Brazilian government recognition of the importance of this work was demonstrated by the announcement of CAPES funding (amount currently confidential) for a strategic partnership to develop undergraduate and postgraduate programmes in forensic science which will include VPA as a core module (partner institutions = Federal Police, PUC-SP, University of Campinas).

The utilization of VPA in forensic phonetics in the UK, Europe and Brazil, drawing on our underpinning research, is supporting evidence-based decisions in criminal cases, thus contributing to the delivery of justice.

2. Commercial impact of VPA and VPA-based acoustic analysis in creative industries

As part of a planned approach to generating impact our research on acoustic classification of voices based on VPA criteria [3] has led to public and private sector investment, which has enabled the creation of a new commercial enterprise in the voice-over industry (worth $4.4bn in 2015). Nowadays, voice-over jobs are frequently negotiated through online market places where voice-over job adverts are posted and voice-over artists audition by submitting voice samples. This creates a bottleneck for the task of shortlisting samples (often several hundred), which is a time-consuming, tedious and potentially ineffective method of voice selection. To develop a solution for this problem based on our research and associated models, we co-founded the limited company ‘The Voice Distillery Ltd’ in March 2020 (company number 656265). This is led by a multidisciplinary team of directors including a speech scientist (Schaeffler), a speech and language therapist (Beck) and two external directors. One is a software engineer and one an experienced voice-over artist and voice coach. The company raised private investment of £35,000 in May 2020 [s4] and has received a ‘SMART Scotland award’ of £67,000 in February 2020 [s5]. SMART is a Scottish business grant scheme aimed at SME’s for their early stage R&D. The ongoing feasibility study develops an AI based system to automatically evaluate and rank voice auditions with respect to client brief and target population preferences, testing the reliability of such a system in reference to a VPA-based representation level. Voice Distillery (currently, in late 2020) has four employees and six freelancers associated with the company [s4]. Despite the Covid19 pandemic, the company has reached all milestones of the SMART feasibility study so far and had recorded, annotated and processed over 3000 voice samples, submitted as auditions by voiceover artists, by the end of 2020, signalling how it has begun, building on our research, to meet the needs of this sector for objective and efficient voice classification. A business development consultant reporting on the company has made specific reference to our innovative application of technology, supported by our phonetic research, to solve industry problems [s6].

3. International impact on clinical guidelines and health policy

Much of our research is designed to generate clinical impact, and our findings have directly informed international clinical guidelines for voice assessment and therapy during Covid19. Our original research [4] gave specific recommendations about which clinical acoustic voice parameters can be reliably derived from smartphone recordings. Our findings for fundamental frequency (f0), shimmer and jitter were then integrated in published clinical guidelines, written by a consortium of 11 voice and swallowing disorder experts from five countries. These guidelines [s7], published in several languages, constitute consensus recommendations from the consortium, and are meant as an international clinical guide for speech-language therapists during the Covid19 pandemic. One of the recommendations provided in the text addresses the use of mobile phones and states that “ Clinicians can use voice samples recorded through a cell phone to obtain F0”, explicitly quoting our work.