Impact case study database
Combined Motor and Language Therapy (MaLT) system for stroke and brain injury rehabilitation.
1. Summary of the impact
A combined Motor and Language Therapy (MaLT) system is providing opportunities for stroke and brain injury patients to self-manage and augment their rehabilitation through game play. The novel platform was developed through a partnership between biomedical engineers and clinical language scientists at the University of Reading, in collaboration with NHS physiotherapists, speech and language therapists (SLTs) and patients. In 2016, the research team started a collaboration with medical device manufacturer Evolv to incorporate MaLT into the company’s CE-marked commercial rehabilitation product, Evolvrehab, which has been used in over 300 hospitals, clinics and care homes in 20 countries around the world. The research team have co-developed two dual-therapy games (each with three therapy modes) with the company to date, building on Evolvrehab’s existing suite of physical therapy modules. In December 2020, the company formally incorporated software for the new games into the Evolvrehab platform and these have been made available, via early release, to patients and therapists at St George’s Hospital London and through Hobbs, which provides rehabilitation to clients across the South East of England.
2. Underpinning research
Motivation: One person has a stroke every five minutes in the UK, equating to 100,000 people per year (Stroke Association, 2020). Whilst improved acute medical care has increased survival rates following stroke or other acquired brain injury, it has led to a growing prevalence of people with long-term rehabilitation needs. Within the UK alone over 1.2 million people live with the long-term consequences of stroke and 120,000 with the effects of traumatic brain injury (TBI). Within this population, language impairments and motor impairments are common. Up to 77% of acute stage stroke survivors and up to 45% of chronic TBI survivors can experience difficulties in using upper limbs and approximately one third of people with stroke and TBI suffer from language impairments, including aphasia. Such impairments have long-term implications for functional independence, social contact, self-esteem and mental health. Motor and language impairments are frequently co-morbid in acquired neurological conditions, including stroke, because of the proximal nature of aspects of the motor and language networks within the brain. By contrast, physiotherapy and speech and language therapy almost universally occur in isolation.
From concept to development platform: In 2012, working with therapists at the Royal Berkshire Hospital (RBH) and Headway (the brain injury association), Professor Rachel McCrindle established a prescriptive rehabilitation and assessment system, which incorporated Microsoft Kinect’s motion-sensing gaming technology [1]. McCrindle’s platform overcame two major drawbacks to using off-the-shelf motion-sensing games for motor rehabilitation: that consumer games are often too complex for patients with severe cognitive impairments [2]; and that consumer games do not provide usable metrics on patient recovery.
The new platform was dynamically adaptable to the abilities of a patient: if a patient was struggling with a task then the next challenge would be simplified or, conversely, would be made progressively harder if the underlying algorithm judged the task to be too easy for that patient. Therapists could also set up the system for each patient, tailoring it to their condition, abilities and interests.
By involving health professionals from the outset, therapists have defined the metrics they need the system to collect in order to track patient progress within a session, across sessions, and in comparison to identified ‘norms’. These data can be analysed by the therapist (both in or after the therapy session, or remotely if sessions were conducted at home) to improve personalised care. Interestingly, early tests showed that the new platform provided data that was unavailable through traditional methods. For example, the progression of a motion and how individuals compensate for restricted joint movements has been usefully recorded in cases where patients have had very little rotation in their elbows.
Subsequently, in collaboration with Dr Holly Robson and Dr Lotte Meteyard from the University of Reading School of Psychology and Clinical Language Sciences, McCrindle integrated three language therapy exercises into the platform [3], each structured into multiple levels of increasing difficulty:
single word comprehension e.g. ‘find the dog’,
initial phoneme identification e.g. ‘find the one that begins with d’,
rhyme identification e.g. ‘find the one that rhymes with log’.
Figure 1: Screenshots of prototype MaLT games. Single word comprehension (top left), Initial phoneme (top right), Rhyming word (bottom left), Speaking task (bottom right).
Patients respond to these language therapy items by moving the arm and hand in response to items on the screen (see Figure 1). This differs from other types of computer-based language therapy that do not require gross motor movements. There are over 1,000 therapy items with each therapy trial selected by algorithm. This leads to unique gameplay every time and the capacity to develop adaptive therapy exercises to further stimulate the one-to-one therapy experience. Importantly, the database structure means that additional items can be rapidly added to MaLT and specific items can be selected by the therapist on a patient-by-patient basis, thus conforming to the core rehabilitation principle of patient-centred care.
Public Patient Involvement: The design of the games was informed throughout the development of MaLT through discussions with a Public and Patient Involvement (PPI) group [2, 3]. Pilot trials with stroke survivors (n=3, for the duration of six to eight weeks) demonstrated that motion sensor technology can be successfully combined with a language therapy task to target both upper limb and linguistic impairment in patients following brain injury. Importantly these initial studies showed that the combined therapy approach was viable for high dosage rehabilitation [3].
Summary: By 2016, MaLT was a fully working research and demonstration system. The MaLT approach is novel in (i) offering rehabilitation that targets both upper limb movement and language comprehension simultaneously; (ii) having a speech and language therapy focussed database that can be tailored to patient interest; (iii) using specific algorithms to generate the increasingly challenging, yet unique, game play elements presented to the patient; and (iv) being adaptable for patients whose first language is not English.
3. References to the research
The underpinning research to develop MaLT was supported through the NIHR Brain Injury Healthcare Technology Co-operative’s Innovation Small Funding Competition 2014-15 (grant number RE004803). This resulted in original research articles in international peer-reviewed journals including Methods of Information in Medicine and Visible Language alongside peer reviewed original articles in books, as is common in the field of computer sciences ( Lecture Notes in Computer Science, Pervasive Health).
- Simmons, S., McCrindle, R., Sperrin, M. and Smith, A. (2013). ‘ Prescription software for recovery and rehabilitation using Microsoft Kinect’. Pervasive Health.
DOI: https://doi.org/10.4108/icst.pervasivehealth.2013.252249
- McCrindle R., Simmons S., Case R., Sperrin M., Smith A., Lock C., (2014). ‘Encouraging Brain Injury Rehabilitation through Ludic Engagement’. In: Stephanidis C., Antona M. (eds) Universal Access in Human-Computer Interaction. Aging and Assistive Environments. UAHCI 2014. Lecture Notes in Computer Science, 8515, 310-320.
DOI: https://doi.org/10.1007/978-3-319-07446-7_30
- Wairagkar, M. , McCrindle, R. , Robson, H. , Meteyard, L., Sperrin, M., Smith, A. and Pugh, M. (2017). ‘MaLT - combined motor and language therapy tool for brain injury patients using Kinect’. Methods of Information in Medicine, 56 (2) 127-137.
DOI: https://doi.org/10.3414/ME16-02-0015
- Moys, J.-L., Martinez-Freile, C., McCrindle, R., Meteyard, L. , Robson, H., Kendrick, L. and Wairagkar, M. (2018). ‘ Exploring illustration styles for materials used in visual resources for people with aphasia’. Visible Language. 52 (3), 96-113. Centaur ID: 79738.
4. Details of the impact
The National Clinical Guideline for Stroke Rehabilitation (2016) states that people with stroke should accumulate at least 45 minutes of each appropriate therapy every day, at a frequency that enables them to meet their rehabilitation goals, and for as long as they are willing and capable of participating and showing measurable benefit from treatment [E1]. MaLT provides an opportunity to increase the ability of stroke and traumatic brain injury patients to self-manage their rehabilitation whilst addressing the requirement for high dosage by combining motor and language therapies through a single platform. Additionally, therapy delivered via games-based technology is more entertaining than traditional rehabilitation exercises, which also increases engagement and subsequent patient benefit.
Partnership with digital rehabilitation solutions provider
In February 2016, the research team initiated a collaboration with Evolv (a spin-off from the company Virtualware), a CE-certified medical device manufacturer specialising in developing hardware and software solutions for the digital health sphere. Their platform for rehabilitation, EvolvRehab (formerly VirtualRehab), has been used in over 300 hospitals, rehabilitation clinics, day centres and residential care homes in 20 countries (including the UK, Spain, Italy, Germany, Hong Kong, Korea, Australia, Russia, USA, Mexico and Argentina) and by thousands of patients. EvolvRehab, like MaLT is an exergame platform, based on Microsoft Kinect technology [E2]. Evolv’s CEO said:
“The concept behind MaLT matched perfectly with our goal of integrating different types of therapeutic approaches into our solution to help treat a variety of impairments caused by neurological conditions like stroke. Aphasia is one of the most common cognitive impairments resulting from stroke and requires high doses of specialised treatment, often only found in hospitals or rehabilitation centres, and which can often be boring for patients.
“The fact that the researchers from Reading had already developed a game-like treatment that combined cognitive with motor activity tasks helped integrate it seamlessly into our existing exergame concept for neuromotor rehabilitation.” [E3]
A license agreement was initiated between the University of Reading and Evolv, providing the company with non-exclusive access to MaLT’s software code and SLT database, and the academic team at Reading has worked with the Evolv developers to align and integrate MaLT’s code into their commercial platform. Evolv has invested approximately GBP 50,000 and three months of developer time into this development.
Early versions of MaLT games that were developed with Evolv have been showcased by the company at major international tradeshows in 2017, 2018, and at RehabWeek in 2019. Images from these events help to depict the progression of MaLT within the Evolv platform [E4].
Distribution of MaLT as part of EvolvRehab platform
The formal integration of MaLT into the EvolvRehab platform was completed in December 2020 with two new combined therapy game packages ( Santa’s Workshop and Fiesta Pinata) each with three rehabilitation modes (spoken word, initial phoneme, rhyme matching) and nine levels of difficulty added to their exergame catalogue (see Figure 2).
The new MaLT games have been made available, via an early release, to therapists and patients in the South of England through St Georges Hospital, London and Hobbs Rehab, which provides residential inpatient rehabilitation as well as outpatient community services across Berkshire, Dorset, Hampshire, Somerset, and Surrey. As Evolv releases scheduled software updates in Q1 2021, MaLT will be available to all EvolvRehab’s existing English language customers. Evolv has also made certain MaLT tasks compatible with the six other languages available to users of EvolvRehab (Spanish, Italian, Chinese, Korean, Romanian and Slovakian), which will distribute MaLT to a wider and global clinical population [E3].
Figure 2: Screenshots of the two MaLT games, Santa’s Workshop (top left) and Fiesta Pinata (top right) with example dashboard providing data to the therapist (bottom left) and user progress summary (bottom right).
MaLT as a co-development platform
As well as the commercial integration and distribution of MaLT as part of EvolvRehab, MaLT is an ongoing research and development platform. To increase portability for speech and language therapists, a tablet version, SpeLT (Speech and Language Therapy), has been developed. To extend MaLT to the diverse communities of Reading, the Reading team are co-developing versions of MaLT and SpeLT in Nepalese, Polish and Urdu with the acute stroke unit in the RBH to extend the use of MaLT to stroke patients whose first language is not English. By working with stroke patients, clinicians, and a tech company, McCrindle et al. are ensuring that new technologies are co-developed to best effect. Aside from strengthening the Reading team’s collaborations with EvolvRehab, the RBH, and the PPI panel, other benefits have included seed corn funding from the NIHR Brain Injury Healthcare Technology Cooperative for a second project, which addresses the communication and socialising difficulties that stroke patients encounter in daily living.
Raising Awareness
MaLT has been widely demonstrated, including to Penny Mordent MP on her Ministerial visit to the University of Reading in 2017, and aviator Polly Vacher on her visit to the Biomedical Engineering Department in 2018; to colleagues in the RBH; to the MaLT stroke PPI panel and to members of the public/trade at high profile external events including the Royal Berkshire NHS Foundation Trust Members Open Day in 2014, 2017 and 2018.
Summary: MaLT is a unique computer game rehabilitation system that combines (aspects) of physiotherapy and speech and language therapy which has been integrated into a CE-mark product that is used by therapists and their clients around the world, enabling stroke and traumatic brain injury patients to better self-manage their rehabilitation. MaLT brings together essential motor and language therapies through a single platform which is more engaging than traditional methods and tailored to the individual.
5. Sources to corroborate the impact
Extracted page from National Clinical Guideline for Stroke, Fifth Edition, 2016
Consumer product information about Evolv Rehab, from the Evolv website
Testimonial from Evolv Chief Executive
Social media reports from Evolv, showcasing MaLT at international trade shows between November 2017 and June 2019