Skip to main

Impact case study database

The impact case study database allows you to browse and search for impact case studies submitted to the REF 2021. Use the search and filters below to find the impact case studies you are looking for.

Search and filter

Filter by

  • University of Northumbria at Newcastle
   None selected
  • 4 - Psychology, Psychiatry and Neuroscience
   None selected
   None selected
   None selected
   None selected
   None selected
   None selected
Waiting for server
Download currently selected sections for currently selected case studies (spreadsheet) (generating)
Download currently selected case study PDFs (zip) (generating)
Download tags for the currently selected case studies (spreadsheet) (generating)
Currently displaying text from case study section
Showing impact case studies 1 to 5 of 5
Submitting institution
University of Northumbria at Newcastle
Unit of assessment
4 - Psychology, Psychiatry and Neuroscience
Summary impact type
Societal
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Early identification of intellectual disabilities is crucial to ensure individuals receive appropriate support, maximising life chances, and minimising negative outcomes. However, many people experience delayed or missed diagnosis. Research from Northumbria University – through the refinement and validation of two screening tools to diagnose intellectual disabilities – has enabled more timely diagnosis of an intellectual disability. Importantly, unlike previously used diagnostic tools, these screening tools can be administered by non-professionals within hard-to-reach populations. They have been recommended and used by key healthcare organisations in the UK (Royal College of Nursing, Royal College of Psychiatry, and Public Health England) and internationally, most notably in Australia. This has facilitated the screening of over 7,500 otherwise unassessed prisoners, young offenders, and victims of crime in the UK and Australia and allowed those identified with an intellectual disability to be referred for extra assessments and support.

2. Underpinning research

People with an intellectual disability (or learning disability) face many challenges, including health inequalities and increased risk of homelessness and imprisonment. Early identification of intellectual disability is crucial to ensure individuals receive appropriate support, maximising life chances and minimising negative outcomes, yet missed and delayed identification of intellectual disability is common. In order to make this process more efficient, the Learning Disability Screening Questionnaire (LDSQ) and Child and Adolescent Intellectual Disability Screening Questionnaire (CAIDS-Q) screening tools were developed by Professor Karen McKenzie when working in the NHS. An ongoing problem, however, was the need for further studies to refine and, crucially, to validate the effectiveness and benefits of these diagnostic tools. After joining Northumbria University in 2014, McKenzie led new quantitative research to enhance and demonstrate the full potential of the LDSQ and CAIDS-Q screening questionnaires, to both extend their versatility and reach [ R1- R6].

Diagnosis of intellectual disability is time-consuming and requires assessment by trained professional psychologists, e.g., using the gold standard Wechsler Adult Intelligence Scales (WAIS) for adults and the Wechsler Intelligence Scales for Children (WISC) for children. The LDSQ and CAIDS-Q questionnaires are used as screening tools to help identify people likely to have an intellectual disability. They are composed of short questions (e.g., an individual’s ability to tell the time or read) and allow a percentage score to be calculated from the answers. These screening tools provide an accurate, quick, easy to use identification of people who are likely to have an intellectual disability so that they can be referred for further assessment. Importantly, the tools can be administered by anyone. Research by McKenzie and colleagues at Northumbria re-assessed the LDSQ in 2015 against the updated 4th edition, WAIS-IV (the most recent version of WAIS), to confirm it was still suitable and valid for use [ R1]. The research team also used a multi-group item response theory approach (a widely used model that analyses data from different groups) to test the LDSQ for gender bias. In a sample of 211 males and 132 females, results showed that gender did not influence the outcome of the screening tool [ R2].

McKenzie’s research has refined and improved the screening tools to extend their versatility and reach. Whereas previously the CAIDS-Q had only determined if a child was ‘likely’ or ’not likely’ to have an intellectual disability, McKenzie developed a method to convert a CAIDS-Q score to an IQ score, making it more useful for clinicians and even providing an alternative IQ estimation method in circumstances where full intellectual assessments are not feasible [ R3]. In 2019, McKenzie and Murray validated the CAIDS-Q for use in NHS paediatric services (often the first point of contact for a child who may have an intellectual disability) by comparing it against commonly used assessments within clinical practice: WISC-IV and the Adaptive Behaviour Assessment System 2nd/3rd edition. This study illustrated the strong psychometric properties and high accuracy of the CAIDS-Q in this setting as it correctly identified all children that were already known, or subsequently found, to have an intellectual disability (54/181) through gold standard assessment [ R4].

McKenzie and Murray explored the potential benefits of using the screening tools with hard-to-reach populations, particularly within homeless services [ R5]. This qualitative study used semi-structured interviews (n=24) with staff and service users to identify support needs of homeless people with an intellectual disability, and the role that the LDSQ could play in such a setting. The LDSQ was considered to have many benefits: it could identify potentially highly vulnerable people, enable the provision of tailored support, and facilitate access to specialist services [ R5]. Such benefits have also been identified in other services including healthcare (specifically intellectual disability and midwifery), education, and social work [ R6].

3. References to the research

R1. Karen McKenzie, Sharples, P.*, and Murray, A.L.** ( 2015) ‘Validating the Learning Disability Screening Questionnaire against the WAIS IV’ Intellectual and Developmental Disabilities 53 (4): 301-307 https://doi.org/10.1352/1934-9556-53.4.301

R2. Murray, A.L.**, Booth, T.**, and Karen McKenzie ( 2015) ‘An analysis of differential item functioning by gender in the Learning Disability Screening Questionnaire (LDSQ)’ Research in Developmental Disabilities 39: 76-82 https://doi.org/10.1016/j.ridd.2014.12.006

R3. Karen McKenzie and Murray, A.L.** ( 2015) ‘Evaluating the use of the Child and Adolescent Intellectual Disability Screening Questionnaire (CAIDS-Q) to estimate IQ in children with low intellectual ability’ Research in Developmental Disabilities 37: 31-6 https://doi.org/10.1016/j.ridd.2014.11.001

R4. Karen McKenzie, George Murray +, Murray, A.L.**, Delahunty, L.***, Hutton, L., Murray, K.****, and O’Hare, A.** ( 2019) ‘Child and Adolescent Intellectual Disability Screening Questionnaire to identify children with intellectual disability’ Developmental Medicine and Child Neurology 61 (4): 444-450 https://doi.org/10.1111/dmcn.13998

R5. Karen McKenzie, George Murray, Wilson, H.*****, and Delahunty, L.*** ( 2019) ‘Homelessness – ‘It will crumble men’: The views of staff and service users about facilitating the identification and support of people with an intellectual disability in homeless services’ Health and Social Care in the Community 27 (4): e514-e521 https://doi.org/10.1111/hsc.12750

R6. Karen McKenzie, George Murray, Murray, A.L.*, and Rachel Martin ( 2019) ‘The impact of an adult intellectual disability screening questionnaire on service providers and users’ Journal of Applied Research in Intellectual Disabilities 33 (3): 457-464 https://doi.org/10.1111/jar.12687

*University of Glasgow; **University of Edinburgh; ***NHS Lothian and University of Edinburgh dual role; **University of Edinburgh; ****Edinburgh Napier; *****NHS Borders

+Additional Consultant roles: August 2015 – July 2016, November 2018 – October 2020

4. Details of the impact

Northumbria’s research extended the validity, reliability, versatility, and positive impact of the LDSQ and CAIDS-Q screening tools, led to policy and practice change across a wide range of services in the UK and internationally, and increased use of the CAIDS-Q and LDSQ. This has had a direct impact on individuals living with intellectual disabilities as diagnosis has enabled them to access support and improve wellbeing and life chances.

4.1 Improved support for people living with intellectual disabilities and their carers

The LDSQ and CAIDS-Q have helped identify vulnerable people who were not previously known to have an intellectual disability, enabling them to receive appropriate support and adjustments. Recent research indicates that over 60% of staff and service users across a variety of settings felt that the CAIDS-Q or LDSQ had helped identify the person’s support needs more quickly, resulted in increased support, and improved the person’s wellbeing and life chances [ E1, p5].

The CAIDS-Q is now available online on the UK-based Learning Disability Matters for Families WebApp [ E2] (a toolkit developed by the NHS North East & Cumbria Learning Disability Network) and provides resources to families and professionals in relation to intellectual disability. Dr Karen Horridge, Consultant Paediatrician (Disability), notes: ‘The inclusion of the CAIDS-Q has allowed…immediate feedback on whether the child/young person is likely to have a learning disability or not. The person is then signposted to sources of support, if required. There is no equivalent evidence-based measure available elsewhere’ [ E3]. Over 500 people used the CAIDS-Q in just the first few weeks after its launch in 2020 [ E3]. Dr Horridge highlights that:

‘The CAIDS-Q helps to ensure that more children receive a timely diagnosis, and that those children who are most likely to have a learning disability received further assessment. As well as being freely available to those accessing the WebApp, the CAIDS-Q is used as part of routine practice by many paediatricians and other professionals’ [ E3].

4.2 Incorporation of the LDSQ and CAIDS-Q into policy and practice

The LDSQ and CAIDS-Q screening tools have been recommended by a range of high-profile bodies in the UK, including the Royal College of Nursing, HM Inspectorate of Probation and HM Inspectorate of Prisons, and the Royal College of Psychiatry [ E1, p4-5]. The screening tools are now embedded in service pathways across the UK. Health Education England [ E4, p7] and at least 130 NHS health boards and council areas in the UK use the tools to support referrals. Internationally, the tools are used in New Zealand, Norway, the USA, and Australia (e.g., the Australian Senate Community Affairs Reference Committee and the Cultural and Indigenous Research Centre Australia report using the CAIDS-Q to facilitate referral to clinical services for assessment) [ E1, p6].

The screening tools have been found to be of particular use in the criminal justice system. Since 2015, at least 22 prisons across the UK have been using the LDSQ to identify prisoners with a possible intellectual disability [ E5], including HMP Brinsford [ E1, p5] and HMP Aylesbury [ E6, p16]. The LDSQ is also used by the police and probation services in Thames Valley, Gloucestershire, and Birmingham [ E5]. Jenny Talbot from the Prison Reform Trust states:

‘The LDSQ is quick, accurate and easy to use (and does not need the person using it to have a professional training), which means that it has been integrated into settings where it would otherwise be difficult to screen for learning disability […identifying learning disabilities at an early stage of the criminal justice system] enables those who might have a learning disability, and who are therefore at increased vulnerability and risk, to be identified in a timely way and receive additional support and assessment as required, for example, identifying the need for an Appropriate Adult, provision of tailored interventions, and accessible information and resources’ [ E5].

Further to this, the LDSQ is routinely used in the service pathways of NHS Trusts and other organisations that provide health and social care to criminal justice services, and those who have been victims of crime, in Yorkshire and Humber, Ashton, Leigh and Wigan Community Healthcare NHS Trust, Cumbria Partnership NHS Foundation Trust, Birmingham Community Healthcare, Wigan Council, Surrey Autism Partnership Board, Manchester University NHS Foundation Trust, and Oxleas NHS Foundation Trust [ E5].

Criminal justice services using the LDSQ report several benefits of this tool over others, including quick, reliable identification of people with an intellectual disability, and more detailed – fewer inappropriate – referrals [ E7, p22]. The scale at which individuals are being assessed and referred using the LDSQ is exemplified by data from Birmingham NHS Healthcare’s Community Forensic Team, who screened 6,683 people using the LDSQ between 2015-2018, of whom 2,963 individuals were referred for further support [ E7, p18].

The LDSQ screening tool is also used to support victims of crime – most notably within Saint Mary’s Sexual Assault Referral Centre (SARC), a service in Manchester, UK, for victims of rape or sexual assault. The SARC routinely uses the LDSQ to assess victims for likely intellectual disability and consequently ensure they receive appropriate support to meet their needs. This is a particularly important tool for this service as pilot studies using the LDSQ revealed that on average 1 in 12 individuals using the service have an intellectual disability and require specialist support [ E8, p2]. A press release from SARC in 2020 highlights that the data gained from the LDSQ is making the organisation ‘start to think about how services should be tailored for people with learning difficulties. For instance, the LDSQ score can be routinely shared with the police who can arrange additional support…such as representatives who can advocate on their behalf’ [ E8, p2] .

This impact is not limited to the UK. The SARC’s work with the LDSQ is being replicated by counterpart organisations in Australia [ E8, p3]. Further, youth justice programmes in the state of New South Wales (NSW) in Australia are using the CAIDS-Q screening tool to assess all those who participate in Youth on Track (YoT) – an early intervention programme aimed at reducing offending rates among young people, funded by the Department of Communities and Justice – totalling around 350 individuals a year [ E9]. Mandy Loundar, manager at Youth Justice NSW, states that the benefits of using the CAIDS-Q are that it is ‘accessible and easy to use with the young people with complex needs by YoT caseworkers’ [ E9]. Loundar is also clear that CAIDS-Q leads to more constructive relationships with service users as it reminds caseworkers that young people ‘may have a disability instead of their behaviour being simply seen as “bad” behaviour’ [ E9]. The CAIDS-Q is also being used in other areas of Australia, such as Victoria [ E9]. The tool has been recommended by the Victorian Auditor-General’s Office [ E1, p4-5] and was incorporated into Victoria’s Youth Justice Strategic Plan 2020-2030 [ E10, p38].

5. Sources to corroborate the impact

Ref. Source of corroboration Link to claimed impact
E1 McKenzie, K. et al. ( 2020) ‘The use of an impact questionnaire as a framework to evaluate the impact of research on policy and practice: screening questionnaires for intellectual disability’ Research Evaluation 1-13 https://doi.org/10.1093/reseval/rvaa019 Summary of the range of impacts of the LDSQ and CAIDS-Q on policy and practice in the UK and internationally
E2 CAIDS-Q WebApp Screenshots – 2a) first page and 2b) results page Confirms inclusion of the CAIDS-Q on the Learning Disability Matters website
E3 Testimonial - Dr Karen Horridge, Consultant Paediatrician (Disability) Confirms the availability and use of the CAIDS-Q at Learning Disability Matters and the benefits of its use by professionals and families
E4 Health Education England (2018). Providing community forensic services for people with Learning Disabilities and/or Autistic Spectrum Conditions Workforce Competency Framework. Confirms recommended use of the LDSQ as a tool to help diagnosis, as part of competency framework for community forensic services, including homeless services. p12 [NB ‘quotient’ used instead of ‘questionnaire’]
E5 Testimonial - Jenny Talbot, Prison Reform Trust Confirms the use of LDSQ in UK prisons, police and probation services
E6 Quality Network for Prison Mental Health Services Annual Report Cycle 4, 2018-2019 Reports on the use of the LDSQ in HMP Aylesbury
E7 The Role of a LD Practitioner in Birmingham Liaison & Diversion Team (April 2019) Presentation Slides Confirms the use of the LDSQ as part of the Liaison and Diversion pathway and the associated benefits
E8 Saint Mary’s Sexual Assault Referral Centre (SARC) press release (accessed 08.10.20) Confirms use of the LDSQ at SARC, associated benefits for sexual assault victims and work with Australia
E9 Testimonial - Mandy Loundar, Manager, Strategic Projects, Youth Justice New South Wales, Australia Confirms use of CAIDS-Q in Youth on Track programme and numbers screened
E10 Youth Justice Strategic Plan 2020-2030 Confirms embedding of CAIDS-Q into strategy
Submitting institution
University of Northumbria at Newcastle
Unit of assessment
4 - Psychology, Psychiatry and Neuroscience
Summary impact type
Health
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Currently approximately 700,000 people have a diagnosis of Autism Spectrum Disorder (ASD) in the UK. However, the complex and interrelated nature of ASD symptoms, compounded by issues associated with assessment practices and protocols, inhibits timely completion of formal diagnostic processes. Research undertaken at Northumbria University has facilitated the first systematic, national exploration of factors influencing ASD assessment waiting times, at individual, service, and policy level, to improve the ASD diagnostic pathway. This research informed strategies that ultimately led to an average reduction in waiting times for ASD assessment in Scotland by 4.3 weeks. The research has directly influenced Scottish government, NHS, and local authority policy and practice through the development of new systems and processes for clinicians working in ASD services. Further, Northumbria’s research is recommended within the Scottish Intercollegiate Guideline Network (SIGN) ASD guidelines, which influence clinical practice across Scotland.

2. Underpinning research

Autism Spectrum Disorder (ASD) is a lifelong developmental disorder. Symptoms are multidimensional and include social and communication impairments, restricted repetitive activities, and associated features including attention deficits and a range of medical and psychological comorbidities [ R1]. Despite ASD being a developmental condition, longstanding failure to carry out timely assessments has meant that many diagnoses do not occur until adulthood [ R2]. Delayed ASD diagnosis can have serious implications for children and adults, including stress, dissatisfaction with the diagnostic process, and the inability to access appropriate support and resources [ R2]. Research undertaken by Professor Karen McKenzie, since joining Northumbria in 2014, has helped to identify a range of factors that inhibit the timely assessment of those with ASD. The findings have guided ongoing work in this area, and the design of interventions that reduce waiting times and thus help individuals (including families and carers) more readily access the support and resources that are available [ R1- R6].

McKenzie’s work has sought to address the lack of available evidence, both nationally and internationally, relating to the underlying causes of delays in ASD diagnostic processes, and which models for assessment provide the most optimal results [ R1, R2]. In collaboration with colleagues from the University of Edinburgh and NHS Fife, McKenzie produced an empirical assessment of diagnostic tools such as the Autism-Spectrum Quotient (AQ), a subjective self-assessment questionnaire designed to measure the expression of Autism-Spectrum traits in an individual [ R1]. The study indicated how it can be difficult to capture the multidimensionality of ASD in a single measure, which can lead to difficulties when attempting to estimate an association between a specific symptom measured by the AQ and a resultant diagnosis [ R1].

McKenzie made similar, vital contributions to this field through her work with the Autism Achieve Alliance (AAA), a multidisciplinary partnership of clinicians and clinical researchers investigating child and adult ASD services. This collaboration represents the first, systematic national exploration (in Scotland) of factors influencing ASD assessment to improve the ASD diagnostic pathways. McKenzie was lead for the adult aspect of the programme, with specific responsibility for coordinating the research in respect of adult services. McKenzie was also responsible for working collaboratively with (5 of 11) participating adult-specific services, to identify and address specific areas for improvement. McKenzie’s research was relevant to both adult and child services and played an important role in shaping methodologies and analysis across the entirety of the AAA project [ R2- R6].

McKenzie led an analysis of 150 case notes (80 child and 70 adult cases) from 16 diagnosing services across Scotland, which demonstrated that, within children's services, increasing the amount of relevant information available pre-assessment is likely to lower the total duration of the assessment process by reducing the number of contacts required [ R2]. The study also showed that having a high risk of ASD as an adult appears to result in being seen more quickly following referral, but also increases the number of contacts needed and total assessment duration [ R2]. The results suggested that comprehensive information about the individual that is directly relevant to the diagnosis of ASD should be routinely sought prior to, or at the point of, referral. These findings were supplemented by research that highlighted a significant difference in the mean age of referral and diagnosis for girls compared to boys, supporting evidence of delayed recognition of ASD in younger girls prior to assessment [ R3].

A further study drew on focus group interviews with 95 clinicians from 8 child and 8 adult services on the duration and quality of the diagnostic pathway and highlighted the need to develop efficient assessment and communication processes [ R4]. These findings supported conclusions offered in McKenzie’s earlier study, which had stressed the importance of information collected by non-specialist services (such as general psychiatric services) in relation to the diagnosis of ASD to speed up diagnostic processes [ R2, R4]. Again, these recommendations were bolstered by additional research in which McKenzie recommended that more training in the use of current diagnostic interviews, combined with a systematic use of evidence-based standardised assessment screening tools and protocols, would improve the efficiency and effectiveness of ASD care pathways [ R5, R6].

Based on the preliminary findings, McKenzie and the AAA team proposed a strategy to reduce waiting times for ASD diagnosis. This approach suggested the use of practice development interventions for clinical staff to share best practice and increase their skills and confidence in effective diagnostic pathways (through mentorship, peer group forums, workshops to drive/sustain change, and validating the roles of practitioners [ R6]). McKenzie played a central role in evaluating the implementation of a pilot scheme in 11 adult ASD diagnostic services in Scotland (across 6 NHS board regions), using multi-level statistical analysis to determine if, and to what extent, it effectively reduced waiting times for ASD diagnostic assessment [ R6]. The results indicated how this approach can successfully reduce the wait time for ASD diagnostic assessment in adult services (from 21.3 weeks down to 17 weeks on average), raising the potential for significant improvements to be made at a national level [ R6].

3. References to the research

R1. Murray, A. L.1,2,3, Karen McKenzie, Kuenssberg, R.4, and Booth, T.3 ( 2015) ‘Do the Autism Spectrum Quotient (AQ) and Autism Spectrum Quotient Short Form (AQ-S) Primarily Reflect General ASD Traits or Specific ASD traits? A Bi-Factor Analysis’ Assessment 24 (4): 444-457 https://doi.org/10.1177/1073191115611230

R2. Karen McKenzie, Forsyth, K.2,5, O’Hare, A.3, McClure, I.3,5, Rutherford, M.2,5, Murray, A.L., and Irvine, L.5 ( 2015) ‘Factors influencing waiting times for diagnosis of Autism Spectrum Disorder in children and adults’ Research in Developmental Disabilities 45-46: 300-306 https://doi.org/10.1016/j.ridd.2015.07.033

R3. Rutherford, M., Karen McKenzie, Johnson, T.2, Catchpole, C.2, O’Hare, A., McClure, I., Forsyth, K., McCartney, D.2,5, and Murray, A.L. ( 2016) ‘Gender ratio in a clinical population sample, age of diagnosis and duration of assessment in children and adults with Autism Spectrum Disorder’ Autism 20 (5): 628-634 https://doi.org/10.1177/1362361315617879

R4. Rutherford, M., Karen McKenzie, Forsyth, K., McCartney, D., O’Hare, A., and McClure, I. ( 2016) ‘Why are they waiting? Exploring professional perspectives and developing solutions to delayed diagnosis of Autism Spectrum Disorder in adults and children’ Research in Autism Spectrum Disorders 31: 53-65 https://doi.org/10.1016/j.rasd.2016.06.004

R5. Rutherford, M., **Karen McKenzie , McClure, I., Forsyth, K., O’Hare, A., McCartney, D., and Finlayson, I.2,5 ( 2016) ’A national study to investigate the clinical use of standardised instruments in autism spectrum disorder assessment of children and adults in Scotland’ Research in Autism Spectrum Disorders 29-30: 93-100 https://doi.org/10.1016/j.rasd.2016.05.003

R6. Rutherford, M., Forsyth, K., Karen McKenzie, McClure, I., Murray, A.L., McCartney, D., Irvine, L., and O’Hare, A. ( 2018) ‘Implementation of a practice development model to reduce the wait for Autism Spectrum diagnosis in adults’ Journal of Autism and Developmental Disorders 48: 2677–2691 https://doi.org/10.1007/s10803-018-3501-5

1University of Cambridge 2Queen Margaret University 3University of Edinburgh 4NHS Fife

5NHS Lothian

4. Details of the impact

McKenzie’s research on the factors affecting the diagnosis of ASD has led to impact in relation to 1) policy, strategy, and planning, 2) practice, and 3) outcomes regarding waiting times.

4.1 Informed national policy, strategy, and planning

As a result of McKenzie’s research expertise, she was part of the 2016 Scottish Intercollegiate Guidelines Network (SIGN) guidance development group, to update the existing national clinical guidelines and checklist for the ‘Assessment, diagnosis and interventions for ASD’ [ E1]. SIGN guidelines facilitate the implementation of effective measures to improve the standards of healthcare in ASD services across Scotland. The resulting SIGN 145 guidance document, published in 2016, now incorporates guidance that applies to adults and older people as there was a ‘need for a guideline which reflects the whole age range’ [ E1, p1]. It now recommends, in line with the research, the need to include ‘all relevant information regarding any concerns, the individual’s current situation and details of any professionals involved’ and the development of protocols to ‘ gather as much relevant information preassessment as possible, in order to reduce assessment duration’ [ E1, p44].

McKenzie’s research also informed the development of NHS Education for Scotland Autism Training Framework: Optimising Outcomes Training Plan [ E2], which sets the standards for all autism-related clinical training in Scotland’s health services – as part of professional training or as CPD – and provides a framework to enable service leads and individual practitioners to establish the training needs of their staff. The plan recognises that acquiring more information for a person prior to their diagnostic assessment can be key to reducing the wait for diagnosis, as highlighted in the research [ R2], and highlights the need for training clinical staff to facilitate better identification, screening, assessment, and diagnosis of ASD, particularly within adult services [ E2, p33].

4.2 Improved understanding of, and practices involved in, ASD referral and assessment

McKenzie’s research (and subsequent clinical recommendations listed above) informed the action plans of health and social care partnerships (HSCPs) in Scotland, including: The Autism Strategy in Fife (2014-2024) [ E3], the East Lothian Strategy (2015) [ E4], and the Perth and Kinross Autism Strategy (2016-2021) [ E5]; areas serving a combined population of approximately 632,000. Collaboration between these HSCPs and the AAA: ‘worked to upskill the workforce and improve processes to reduce waits and evidence best practice’ for adult services in Fife [ E3, p9], led to quicker referral to diagnosis times in East Lothian child services [ E4, p6], and in Perth and Kinross ‘all diagnostic teams have been made aware of…up to date research on reducing waiting times’ [ E5, p8].

The AAA Autism Knowledge Hub was created to enable Scottish autism diagnostic services and clinicians (e.g. from Lothian, Fife, Ayrshire and Arran, Borders, Highland, and Glasgow) to share best practice and collate documentation, protocols, action plans and recommendations from the research, and subsequent practice, in one place. The Scottish Government identified the Knowledge Hub as a key resource to ‘facilitate improvement for autism assessment and diagnosis for all age groups and genders’ [ E6, p10].

4.3 Reduced wait for ASD assessment

The research clarified the causes and extent of delays in ASD diagnosis in Scotland and developed a strategy to improve the ASD diagnostic pathway. Once implemented, these strategies introduced more efficient working and communication into services, improving the effectiveness of care pathways in both child and adult services [ E7]. Mentoring and support were also made available to clinicians involved in diagnosing ASD [ E8]. Results from the implementation of this strategy in 11 adult diagnostic services across Scotland concluded that waiting times reduced by an average of 29.9 days (4.3 weeks) [ E8; E9, p16].

Andy Fleming, Autism Spectrum Disorder Nurse Practitioner, NHS Lanarkshire (population of 650,000), acknowledged the benefits of the research and intervention in this area of Scotland, noting an average reduction in wait times of 36 days. He stated that this strategy led Lanarkshire’s ASD diagnostic pathway to become ‘ more focused, effective, and efficient for both staff and patients’ by: 1) introducing a clear assessment pathway focused on information gathering and reporting back to patients (to adhere to good practice guidance specified by SIGN and NICE), 2) timely provision of information to potential referrers to ensure the appropriateness of referrals, 3) introducing systems to pre-empt, and ultimately prevent, non-attendance at appointments (e.g. sending reminders), and 4) working to improve information sharing between team members [ E10]. Fleming identified key benefits of improved post-diagnostic support and the opportunity to share good practice with, and learn from, the experiences of other services throughout Scotland:

The timely diagnosis of autism and reduced waiting times for assessment and diagnosis…allow those who are diagnosed to receive additional support and resources that they may require and they can be signposted to organisations and support networks that may be of benefit to them, which will help to improve the outcomes for them. The sooner that diagnosis takes place the sooner that such support can be arranged. This might include specialist input from staff to provide interventions to help with social interaction, managing anxiety, and behaviours that challenge, such as aggression. As long waiting times can lead to dissatisfaction with and loss of confidence in services, reduced waits can counter this’ [ E10, p2].

5. Sources to corroborate the impact

Ref. Source of corroboration Link to claimed impact
E1 SIGN 145: Assessment, diagnosis and interventions for autism spectrum disorders - A national clinical guideline (June 2016) Research cited in SIGN 145 and influenced the wider development of the guidelines
E2 The NHS Education for Scotland Autism Training Framework: Optimising Outcomes Training Plan for ASD (May 2015 Update) Research cited and highlighted required training needs for Scotland
E3 Autism Strategy in Fife, 2014-2024 - Published 15 Jan 2017 by Fife Council Confirms collaboration to upskill workforce and reduce waits
E4 East Lothian Autism Strategy 2015 Confirms quicker referral to diagnosis times
E5 Perth and Kinross Autism Strategy, 2016-2021 Confirms diagnostic teams made aware of waiting times research
E6 The Scottish Strategy for Autism Outcomes and Priorities 2018-2021 Refers to benefits of the research and Knowledge Hub for sharing good practice
E7 Rutherford, M. et al. (2016) ‘Why are they waiting? Exploring professional perspectives and developing solutions to delayed diagnosis of Autism Spectrum Disorder in adults and children’ Research in Autism Spectrum Disorders 31: 53-65 Confirms strategy implementation and improvements to care pathways. Also R4
E8 Rutherford, M. et al. (2018) ‘Implementation of a practice development model to reduce the wait for Autism Spectrum diagnosis in adults’ Journal of Autism and Developmental Disorders 48: 2677–2691 Evaluation of new strategy – demonstrates success in reducing waiting times for ASD diagnosis. Also R6
E9 Scottish Parliament - SPICe Briefing Autism Spectrum Disorder, 2017 Scottish Government briefing on the research and associated significant reduction in waiting times
E10 Testimonial - Andy Fleming, Autism Spectrum Disorder Nurse Practitioner, NHS Lanarkshire Example of benefits of the research to a large clinical service and to those being assessed for ASD
Submitting institution
University of Northumbria at Newcastle
Unit of assessment
4 - Psychology, Psychiatry and Neuroscience
Summary impact type
Societal
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Cybercrime costs the UK economy approximately GBP27,000,000,000 every year, with many cyberattacks attributed to human error. Research on human-centred cybersecurity at Northumbria University established new theoretical frameworks and developed practical approaches for reducing vulnerability to cyberattacks. With regards to policy, Northumbria’s work shaped UK Government campaigns led by the National Cyber Security Centre (including the national Cyber Aware campaign [text removed for publication]), as well as the Government’s approach to securing the ‘Internet of Things’, influencing new funding streams worth GBP32,000,000. Cybersecurity was also improved through novel training and practice. Using Northumbria’s research, ThinkCyber redesigned their main cybersecurity awareness training product and developed a new product, generating economic impact. Research was also used by one of the largest private hospitals in Europe to improve behaviours of more than 5,000 members of staff. The Northumbria team also developed and delivered new training for older adults, a group particularly vulnerable to cybercrime.

2. Underpinning research

Human behaviour, including non-compliance with company policy, is a key area of cybersecurity vulnerability for many organisations. Northumbria University specialises in human-centred cybersecurity research and is known for its innovative work in changing cybersecurity attitudes and behaviours. In acknowledgement of this work, Northumbria has been recognised as an Academic Centre of Excellence in Cybersecurity by the National Cyber Security Centre (NCSC). NCSC is a branch of the Government Communications Headquarters, responsible for improving the security of the UK’s online activity through technological improvements and advice to citizens and organisations.

Northumbria research that underpins the impact includes the EPSRC project CHAISE (Choice Architecture for Information Security Decisions under Uncertainty, 2013-2016). This was one of the four inaugural projects funded as part of the UK’s Research Institute in Sociotechnical Cybersecurity (RISCS). The Northumbria team provided the behavioural science input and generated new knowledge about how to generate, design, and assess behavioural ‘nudges’ (through changing of interface menus, colour coding of icons, and flagging of warning elements in phishing emails) that can reduce cybersecurity vulnerability in the workplace [ R1]. The project also used new gamified methods to detect workplace vulnerabilities and reveal the tensions between corporate approaches to cybersecurity compliance and the attitudes and behaviours of employees [ R2].

Additionally, the researchers used protection-motivation theory (PMT), which seeks to understand the cognitive processes of threat appraisal and coping appraisal (‘Am I vulnerable to threat?’, ‘Do I know what to do?’, ‘Do I believe this will be effective?’) [ R3, R4]. PMT can predict security intentions and adaptive or maladaptive behaviours implemented as a response to the threat [ R3]. Northumbria research demonstrated that a simple message about protective actions was more influential than a ‘fear appeal’ in improving security behaviour [ R3, R4].

Most online activities require a secure digital identity to gain access to goods, services, and information (e.g. email or online banking). Passwords, PINS, and biometrics are among the most commonly used means to authenticate ‘digital identity’ but these are not always usable or acceptable to the public. This kind of ‘digital identity’ was the focus of the EPSRC funded IMPRINTS project (Identity Management: Public Responses to Identity Technologies and Services, 2011-2014). The Northumbria team was the first to use an inclusive, citizen-led, and value sensitive approach to identify factors affecting consumer acceptance of digital identity and authentication mechanisms. This included an examination of those elements that are widely disliked and impede take up, and elements that are liked and so facilitate use. The researchers assessed six diverse citizen groups: young people, older adults, refugees, minority ethnic women, people with disabilities, and mental health service users. Trust and acceptance factors were found to differ widely across these communities and across technologies. The research underlined key factors and motivators likely to encourage engagement with new identity technologies, while highlighting that a critical issue is to get the basics of governance right in order to ensure that diverse communities adopt new systems [ R5]. This project was selected as a ‘Big Idea for the Future’ by RCUK and shortlisted for the Novay Digital Identity Award in 2013 .

The IMPRINTS initiative highlighted some of the specific cybersecurity challenges faced by older adults, a demographic that is increasingly a target for cyber-attacks. Picking up this work, the EPSRC funded cSALSA project (Cybersecurity Across the Lifespan, ongoing since 2017) focused on mitigating age-related vulnerabilities. The Northumbria team had demonstrated the kinds of difficulties older adults and other marginalised communities experienced when trying to authenticate their identity using passwords, PINS, biometrics, and other novel methods [ R5]. In this new project, the team documented the specific security challenges experienced by older adults and showed the kinds of communication strategies that would be most effective and influential in changing older adult behaviour. The important role of radio, commercial support, and community resources were highlighted, as well as the drawbacks of over-reliance on social information sources [ R6]. This work has influenced policy and campaign strategies in UK Government.

3. References to the research

  1. Nicholson*, J., Lynne Coventry, and Pam Briggs ( 2017) ‘Can we fight social engineering attacks by social means? Assessing social salience as a means to improve phish detection’ Proceedings of the Thirteenth Symposium on Usable Privacy and Security (SOUPS 2017) https://www.usenix.org/system/files/conference/soups2017/soups2017-nicholson.pdf Papers submitted undergo three rounds of peer review with c.20% acceptance.

  2. Nicholson*, J., Lynne Coventry, and Pam Briggs ( 2018) ‘Introducing the cybersurvival task: assessing and addressing staff beliefs about effective cyber protection’ Proceedings of the Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018) https://www.usenix.org/system/files/conference/soups2018/soups2018-nicholson.pdf Papers submitted undergo three rounds of peer review with c.20% acceptance.

  3. Van Bavel**, R., Rodríguez-Priego**, N., Vila**, J., and Pam Briggs ( 2019) ‘Using protection motivation theory in the design of nudges to improve online security behaviour’ International Journal of Human-Computer Studies 123: 29-39 https://doi.org/10.1016/j.ijhcs.2018.11.003

  4. Blythe*, J. M. and Lynne Coventry ( 2018) ‘Costly but effective: Comparing the factors that influence employee anti-malware behaviours’ Computers in Human Behavior 87: 87-97 https://doi.org/10.1016/j.chb.2018.05.023

  5. Pam Briggs and Lisa Thomas ( 2015) ‘An inclusive, value sensitive design perspective on future identity technologies’ ACM Transactions on Computer-Human Interaction 22(5): 1-28 https://doi.org/10.1145/2778972

  6. Nicholson*, J., Lynne Coventry, and Pam Briggs ( 2019) ‘If it's important it will be a headline: Cybersecurity information seeking in older adults’ Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems: 1-11 https://doi.org/10.1145/3290605.3300579

* Internal Northumbria co-authors: J. Nicholson from UoA11 (Computer Science and Informatics), J. M. Blythe (Northumbria PhD student, supervised by Lynne Coventry)

** External co-authors: R. van Bavel and N. Rodríguez-Priego (Joint Research Centre, European Commission), J. Vila (University of Valencia, Spain)

4. Details of the impact

This research has influenced UK and European organisations, enabling them to improve approaches to cybersecurity through innovative training initiatives, policies, and practice.

4.1 Reducing vulnerability to cyberattacks through improved training

Following a presentation of Northumbria research to the national Research Institute in the Science of Cybersecurity (RISCS) in 2018, Professor Pam Briggs was approached by digital training company ThinkCyber to share research findings, particularly around how PMT can be used to design prompts for appropriate behaviour [ R1- R3]. This work influenced the company’s approach to training: ‘little and often, easy to access, simple and actionable, context driven/real-time’ [ E1, p2]. The company blogged the influence of this work on its thinking [ E1, p5] and incorporated Northumbria’s findings into its digital cyber awareness training product Redflags [ E1, p2]. Redflags won ‘Best Professional Training or Certification Programme’ at the 2020 SC Awards Europe (cybersecurity magazine) [ E1, p7] and the company has since been added to the CyberTech100 list of the best global cybersecurity companies . ThinkCyber also asked Briggs to collaborate in developing other new products, submitting a joint InnovateUK grant ‘Reimagining Cyber Security Awareness Training’, which was awarded in November 2018. This enabled Northumbria’s team to work with ThinkCyber’s clients Camden Council, Deloitte, and Axelos, running workshops to identify problems and suggest design solutions, integrating the ‘value-sensitive design’ approach [ R5] from earlier research [ E1, p2]. Tim Ward, CEO of ThinkCyber, confirmed that ‘Two particular ideas from this [work with Northumbria] – Security Bulletins and Real-time tips have evolved into key parts of our product. [text removed for publication] [ E1, p2].

[text removed for publication]

The team has also worked directly with older people in the North East of England, through peer-to-peer training to improve e-safety. This work was undertaken in collaboration with the University of the Third Age and the Old Low Light Heritage Centre. Beginning in 2017, the team ran workshops focused on six themes and trained 242 older adults with the result that ‘participants improved their online behaviours, in particular with regards to creating better passwords and being more vigilant online’ [ E3]. Following these, as part of a ‘CyberGuardians’ project commissioned via EPSRC’s Not-Equal Network Plus, Northumbria trained 14 cyber security ambassadors aged in their 50s, 60s, and 70s on how to interact with technology securely. As of September 2020, these ambassadors had cascaded key harm prevention information into their communities, helping approximately 820 people with become more cyber secure [ E3, p3].

4.2 Impact on public policy and practice

Northumbria’s team has made a substantial contribution to the thinking of UK and European bodies working on the human elements of cybersecurity, as few psychologists have been involved in this field from the outset. Paul Waller, Head of Research, NCSC Capability, noted: ‘Professors Briggs’ and Coventry’s sustained public insistence on relating sociotechnical security problems back to psychological theory has percolated the community and our thinking’ [ E4, p1]. The policy and practice community has benefitted through three types of policy impact: impact on new funding streams, impact on major campaigns to improve cyber security activity by the public, and impact on policy guidance.

Funding Streams: In early 2014, Briggs and Coventry were asked by the then UK Government Behavioural Insights Team to deliver a UK Government Office for Science report on ‘Using behavioural insights to improve the public’s use of cyber security best practice’ [ E5]. The report, published on 8 May 2014, highlighted the lack of reliable behavioural data on internet users’ understanding and use of cybersecurity measures. The report led to an EPSRC ‘Human Dimensions of Cybersecurity Workshop’ on 22 September 2014 (attended by Briggs) and a subsequent GBP5,000,000 EPSRC call on ‘Human Dimensions of Cybersecurity’, launched in 2016. Also in 2014, Coventry was asked to contribute a behavioural science/security perspective to the Blackett Review on the ‘Internet of Things [IoT]: Making the most of the second digital revolution’, which addressed issues of trust and public acceptance [ E6, p38]. The research agenda outlined in that report led directly to the establishment of the GBP14,000,000 National Centre of Excellence for IoT Systems Cybersecurity (PETRAS), with the GBP13,800,000 PETRAS 2 announced in 2019. PETRAS consists of 16 institutions (including Northumbria University) and ensures that the IoT technologies are safely and securely applied in consumer and business contexts [ E6, p41] .

Public campaigns: ‘Cyber Aware’ is the UK Government’s national campaign on cyber security and recent iterations have been influenced by Northumbria research. ‘Cyber Aware’ is led by NCSC and delivered in partnership with the Cabinet Office, Home Office, and the Department for Digital, Culture, Media & Sport (DCMS). The campaign has been running for six years, but was relaunched in April 2020 in the light of increasing online activity during the COVID-19 pandemic and received a major marketing push in December 2020 headed up by Penny Mordaunt, the Paymaster General, and Lindy Cameron, Chief Executive of the NCSC. This new approach to the Cyber Aware campaign focused on ideas of positive framing, coping strategies, and realistic action – making cyber security approachable for everyone – that come from the work of Briggs and Coventry [ E4, p3]. Other NCSC campaigns influenced by this work include ‘ You Shape Security’ and ‘ People are the Strongest Link’ [ E4, p3].

[text removed for publication]

Policy guidance: Policy has been influenced in Europe, with Briggs’ research using PMT used to assess the effectiveness of a range of cybersecurity nudges [R3]. This generated three European Commission technical reports for the Joint Research Centre, which provide evidence-based support to European policymaking. These reports covered effective privacy and security ‘nudges’ in warning messages and banners, with Briggs’ contributions explicitly acknowledged [E7, p4, p54 and p100 of the compiled document]. The digital identity research also fed into an EU policy guidance document ‘Cybersecurity in the European Digital Single Market’ which stated ‘the importance of sub-topics such as digital identity and the notion of "trust" emerged as central cross-cutting themes which aided the development of the opinion’ [E8, p53]. Briggs is cited as a contributor [E8, p84].

In the UK, the DCMS used the Blackett IoT report to create the Government guidance ‘Secure by Design: Improving the cyber security of consumer Internet of Things’ [ E9]. Coventry was invited onto the panel to create this document and the report referenced cSALSA [ R6] as the source of findings that helps government ‘to design more effective cyber security advice and educational materials that are tailored for different audiences’ [ E9, p27]. The digital identity research [ R5] led to an invitation for Briggs to join the All Party Parliamentary Group on Digital Identity. The findings from the IMPRINTS project then informed the House of Commons Science and Technology Committee Report on ‘Current and Future Uses of Biometric Data and Technologies’, where the challenge in establishing trust – especially around biometrics – was noted from IMPRINTS [ E10, p29, p43].

5. Sources to corroborate the impact

Ref. Source of corroboration Link to claimed impact
E1 Compilation of ThinkCyber materials: testimonial - Tim Ward (CEO and Co-founder of the ThinkCyber), information about RedFlags product, blog about using protection-motivation theory, and SC 2020 award Corroborates Northumbria’s input into award-winning cybersecurity awareness training, economic impact, and deployment of research in marketing (blog)
E2 [text removed for publication] [text removed for publication]
E3 Compilation of a testimonial from Mike Martin (former Northumbria Regional Committee Member of the University of the Third Age (U3A)), and U3A CyberGuardians project report Corroborates improved cybersecurity awareness among 820 older people in the North East of England
E4 Two testimonials from the National Cyber Security Centre (NCSC): a) Paul Waller (Head of Research, NCSC Capability), b) Ceri Goncalves Jones (former NCSC employee) Corroborates Northumbria’s input into UK cybersecurity policy development
E5 UK Government Office for Science report ‘Using behavioural insights to improve the public’s use of cyber security best practice’ (2014) Report produced by Northumbria’s psychology department (Coventry, Briggs, Blythe, Tran), showing impact on policy and how behaviour change theories can be used to improve cybersecurity behaviours
E6 UK Government Office for Science report ‘The Internet of Things: Making the most of the Second Digital Revolution’ (2014) and screenshot of PETRAS website Corroborates impact on cybersecurity policy in the UK and the role of Northumbria research in PETRAS
E7 Compilation of three reports from the European Commission, Joint Research Centre (all from 2016): ‘The effect of warning messages on secure behaviour online’ (p1); ‘Nudging online security behaviour with warning messages’ (p51); ‘Testing the effect of the cookie banners on behaviour’ (p97) The three reports corroborate that Northumbria’s researchers acted as external consultants in forming EC’s thinking on cybersecurity
E8 European Commission, High Level Group of Scientific Advisors report ‘Cybersecurity in the European Digital Single Market’ (2017) The report draws on Northumbria’s IMPRINTS research project findings. Corroborates impact on EU cybersecurity policy
E9 UK Government, Department for Digital, Culture, Media and Sport ‘Secure by Design: Improving the cyber security of consumer Internet of Things’ (2018) Corroborates that Northumbria’s researchers contributed to creation of policy guidelines for secure consumer behaviour in the UK
E10 House of Commons, Science and Technology Select Committee report ‘Current and future uses of biometric data and technologies’ (2015) Corroborates impact on cybersecurity policy in the UK
Submitting institution
University of Northumbria at Newcastle
Unit of assessment
4 - Psychology, Psychiatry and Neuroscience
Summary impact type
Health
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Insomnia is a significant, costly condition, known to contribute to the development and/or worsening of other conditions, including depression. Cognitive Behavioural Therapy (CBT) is traditionally considered the first line of treatment for insomnia. However, success is often hampered by a lack of qualified providers and high levels of non-adherence and attrition. Over the last decade, Professor Jason Ellis at Northumbria University has led research to improve the understanding of insomnia, culminating in the development of an alternative treatment – a brief, ‘one-shot’, CBT-I intervention. Development of the intervention has informed EU and USA sleep guidelines and improved clinical understanding and skills for treating sleep issues in the UK and beyond. Ellis has also used his research to improve public understanding of, and ability to self-manage, insomnia, through the development of publicly accessible self-help tools, including a book entitled The One-Week Insomnia Cure, the content of which was serialised in the Daily Mail in 2017. Feedback on the book and serialisation demonstrates a change in understanding, which has helped readers to manage their sleep issues, leading to improved sleep outcomes among the general population.

2. Underpinning research

Insomnia is a condition that affects approximately 16% of the population of the UK and Europe. The research institute RAND Europe estimates that this costs the UK economy GBP40,000,000,000 per year in reduced productivity and absences from work. Without treatment, insomnia is unrelenting and persistent and can lead to the development or worsening of other health problems, including depression [ R1- R3].

Since 2009, Professor Jason Ellis from Northumbria University has led research (funded in-part by the ESRC) exploring the clinical manifestations of, and potential new ways to treat, insomnia. Within this research, Ellis has worked to improve understanding of acute insomnia, a condition that traditionally receives relatively little attention in comparison to persistent/chronic insomnia [ R1]. Ellis’ research provided the first accepted working definition of acute insomnia for research and clinical practice – insomnia which persists for a period of between 2 weeks and 3 months, resulting in distress and reduced quality of life [ R1]. Recognising an absence of published estimates of the prevalence/incidence of acute insomnia and data regarding transition to chronic insomnia or remission, Ellis led an epidemiological study to ascertain such data in the UK and USA [ R2]. This research, published in 2012, determined that the prevalence of acute insomnia was 7.9% in the UK and 9.5% in the USA, and that the annual incidence rate is between 33-36% in the UK. These findings aid the understanding of the pathogenesis of insomnia and improved comprehension of when and how treatment should be initiated [ R2]. In 2014, Ellis published a mixed-model inception study exploring the link between acute insomnia and depression, which identified a clear link between poorer sleep experienced because of acute insomnia and a first-ever episode of depression [ R3].

The traditional first-line treatment for insomnia is Cognitive Behavioural Therapy for Insomnia (CBT-I), which requires 8-10 hours of face-to-face contact with a healthcare professional, over a period of 6 to 8 weeks. The success of CBT-I is known to be hampered by a lack of qualified providers and high levels of non-adherence and attrition. Using this knowledge, Ellis developed a new form of treatment – a brief, ‘one-shot’, CBT-I intervention comprising a one-hour session (covering sleep education and sleep restriction) aimed at managing dysfunctional behaviours during the acute phase of insomnia [ R4]. An accompanying self-help pamphlet was also provided, explaining the principles of stimulus control, cognitive control, and the use of imagery distraction techniques – conveyed as the three D’s: detect, detach, distract - to manage dysfunctional sleep-related thinking [ R4]. A randomised controlled trial exploring the efficacy of this variant of CBT-I conducted by Ellis and colleagues concluded that full CBT-I was not necessary – rather, this brief version with fewer components would be suitable for circumventing the development of insomnia disorder [ R4]. Results from the study of this new intervention demonstrated it was effective and efficacious, resulting in a 73% remission status at three months post-treatment [ R4]. Following this, Ellis and colleagues performed additional research to test the efficacy of brief CBT-I in different contexts. In 2016, they conducted a pilot study to examine the extent to which delivery approach – group- vs. individual-therapy - impacted the success of brief CBT-I [ R5]. The results revealed that the brief CBT-I intervention could be delivered through either method, with only a slight reduction in efficacy (of 1%) identified in the group setting [ R5]. In 2018, Ellis and colleagues tested the success of the intervention with a vulnerable population, namely male prisoners, again finding a significant remission rate at one-month post-treatment, as well as substantial reductions in symptoms of anxiety and depression [ R6].

3. References to the research

R1. Jason Ellis, Gehrman, P.**, Espie, C.A.**, Riemann D.**, and Perlis, M.** ( 2012) ‘Acute Insomnia: Current Conceptualizations and Future Directions’ Sleep Medicine Reviews 16: 5-14 DOI: 10.1016/j.smrv.2011.02.002

R2. Jason Ellis, Perlis, M.L., Neale, L.*, Espie, C.A., and Bastien, C.H.** ( 2012) ‘The Natural History of Insomnia: Focus on Prevalence and Incidence of Acute Insomnia’ Journal of Psychiatric Research 46: 1278-1285 DOI: 10.1016/j.jpsychires.2012.07.001

R3. Jason Ellis, Perlis, M.L., Gardani, M.**, Bastien, C.H., and Espie, C.A. ( 2014) ‘The Natural History of Insomnia: Acute insomnia and first onset depression’ Sleep 37(1): 97-106 DOI: doi.org/10.5665/sleep.3316

R4. Jason Ellis, Cushing, T.*, and Germain, A.** ( 2015) ‘Treating Acute Insomnia: A Randomised Control Trial of a ‘Single-shot’ of Cognitive Behavioral Therapy for Insomnia’ Sleep 38(6): 971–978 DOI:  10.5665/sleep.4752

R5. Boullin, P.*, Ellwood, C.*, and Jason Ellis ( 2016) ‘Group versus individual treatment for acute insomnia: A pilot investigation of a ‘one-shot’ treatment strategy’ Brain Sciences 7: 1-9 DOI: 10.3390/brainsci7010001

R6. Randall, C.*, Nowakowski, S.**, and Jason Ellis ( 2018) ‘Managing Acute Insomnia in Prison: Evaluation of a ‘one-shot’ Cognitive Behavioral Therapy for Insomnia (CBT-I) intervention’ Behavioral Sleep Medicine 17(6): 827-836 DOI: 10.1080/15402002.2018.1518227

*Internal collaborators: L. Neale – research fellow; T. Cushing, P. Boullin, C. Ellwood – students

**External collaborators: P. Gehrman, M. Perlis – University of Pennsylvania; C. Espie (2012), M. Gardani – University of Glasgow; D. Riemann – Feriburg University Medical Centre; C. Bastien – Université Laval, Canada; C. Espie (2014) – University of Oxford; A. Germain – University of Pittsburgh; S. Nowakowski – University of Texas

4. Details of the impact

Ellis’ research has led to two main areas of impact, regarding: 1) improved clinical understanding, guidance, and skills for treating sleep issues and insomnia and 2) public understanding of sleep issues and ability to self-manage insomnia.

4.1 Improving clinical understanding of sleep issues and ways to treat insomnia for patient benefit

Ellis’ work has improved clinical understanding of insomnia and improved skills to effectively treat this condition. Since 2013, Ellis has delivered training on his brief, ‘one-shot’, CBT-I approach to over 580 healthcare professionals in the UK at locations including: NHS Aintree, the Edinburgh Sleep Service, Surrey & Borders NHS Foundation Trust, and NHS Scotland (Psychological Interventions team). Training has also been delivered by Ellis internationally to around 250 professionals from a wide range of countries, including the USA (n=>150), Holland (n=35), and Japan (n=5). Participation in this training, alongside engagement with Ellis’ broader research on insomnia, has been recognised to have a substantial effect on clinicians. Dr Renata Riha, a consultant in Sleep and Respiratory Medicine at Royal Infirmary of Edinburgh, states that it has ‘ helped me to deal more effectively with patient problems due to the speed of the intervention, on a day-to-day clinical basis’ [ E1]. In addition : ‘Professor Ellis’ research is seen as ground-breaking in the field of sleep, particularly his one-shot intervention… [his] work is valued very highly and seen as inspirational… I personally know individuals who have been inspired to change career after engaging with his work’ [ E1].

Similarly, Dr Richard Twaites, Consultant Clinical Psychologist at Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust, invited Ellis to run a two-day sleep training course tailored to the First Step workforce (a service providing telephone-based therapy within Cumbria, Northumberland, Tyne and Wear, and Lancashire). Approximately 40 mental health practitioners were trained in how to deliver brief CBT-I practices, which have subsequently been rolled out in NHS clinical practice [ E2]. Twaites notes:

Staff were extremely receptive to this training and it has become clear since that this training has drastically boosted their understanding of and interest in brief CBT-I as a way to treat sleep issues. This training also provided [40+ members of] staff with the skills to be able to deliver CBT-I, using this approach, to patients who contact our service with sleep disorders’ [ E2].

Therapy services across the UK, including those specialising in treating sleep issues, have incorporated Ellis’ brief CBT-I intervention into their practice and are now using it to treat patients, including the Edinburgh Sleep Service [ E1], Berkshire Healthcare NHS Foundation Trust [ E3], and First Steps [ E2]. Accounts from clinicians using the brief CBT-I intervention clearly indicate that it is having a positive impact on patients sleep. Riha from the Edinburgh Sleep Clinic (which treats 2,000 patients a year) states that the intervention has improved practice across the department since it was incorporated into care protocols, by ‘ chang[ing] the way that we approach patient referrals for behavioural sleep issues [as] we are able to send the leaflet [Ellis’ aforementioned pamphlet] as an immediate intervention to the patient and their general practitioner allowing them to ‘get started’ and understand the process’ [ E1]. Riha notes that patients ‘ appear to find this approach very easy to follow and empowering, as it gives them a renewed sense of ownership over their future’ [ E1].

This impact is not restricted to the UK. Brief CBT-I has been incorporated into clinical practice in other countries, including the USA, Australia, and Iceland. One example demonstrating how far this intervention has been adopted is its use in the Republic of Korea. Dr Chan-Young Kwon, Specialist in Neuropsychiatry of Korean Medicine, highlights that he has been using the brief CBT-I intervention with his patients since opening his clinic in May 2020, with positive results:

‘Incorporating Prof. Ellis’ one-shot CBT-I approach into my practice has given my patients more choice over their treatment and I believe it has improved the sense of self-efficacy among patients and their engagement with the treatment process…I often meet patients who feel that keeping a diary (to organise the previous day), as recommended by Prof. Ellis’s pamphlet, has improved their sleep. From this I believe this practice can help the sleep onset, by reducing excessive ruminant thoughts’ [ E4].

Ellis’ own evaluative work demonstrates the effective and efficacious nature of the brief CBT-I intervention, including a 73% remission status at three months post-treatment [ E5]. The patient benefit of brief CBT-I has also been confirmed by an independent evaluative study conducted in 2018 by the University of Reading [ E6], which applied the intervention in adolescent depression and identified ‘ positive improvements in sleep but also an indication of improvements in depressive symptoms’ [ E3]. Dr Faith Orchard, a psychologist who worked on this evaluation, confirmed that since the completion of this work in 2018, Berkshire Healthcare NHS Foundation Trust have continued to offer the intervention to young people referred to the Child and Adolescent Mental Health Service for difficulties with anxiety and depression. In addition, it has been adapted into a workshop series that was tested for feasibility and acceptability with three local schools. This involved a total of 41 secondary school students, 96% of whom reported that they found the sessions ‘useful’, particularly because of their ‘calm’ and ‘non-judgmental’ nature [ E3].

Ellis’ brief CBT-I intervention has also been recognised in guidance on the management of sleep problems, including the European Guidelines for the Assessment and Management of Insomnia [ E7] – a reference document for all clinicians involved in the stepped-care management of insomnia – and a response to managing sleep in the context of COVID-19 published by the Society for Behavioural Sleep Medicine, which acknowledges that the three D’s (detect, detach, distract) can assist patients with managing new-onset (acute) insomnia [ E8, p3].

4.2 Improving public understanding, and supporting self-management, of insomnia

In addition to the development of an intervention for use by professionals Ellis has also published a self-help book targeted at members of the public entitled The One-Week Insomnia Cure (2017), which has sold more than 10,000 copies around the world. The book presents techniques informed by Ellis’ research to address sleep issues including CBT-I, and, as the title suggests, includes a 7-day course that readers can follow to improve their sleep. The content of this book has been approved by clinicians working in the field; Surrey Hospital Insomnia Unit, for example, recommend the book as an appropriate form of self-help for individuals living with insomnia [ E9, p4].

The book has received substantial media interest and following its publication Ellis was invited to appear on television shows to discuss his research, including This Morning [ E10] (around 1,000,000 viewers a day). The Daily Mail (readership: 1,511,357 – Audit Bureau of Circulations) published a serialisation based on the book in February 2017 – comprised of 6 articles which have since been shared online over 2,250 times [ E10]. Louise Atkinson, the freelance journalist involved in producing this serialisation, acknowledges it was ‘ very well received by the editors and readers alike […particularly because] it offered a ‘new’ solution (sleep restriction) with advice and guidance from a respected expert’ [ E11]. Indeed, Atkinson notes that the Daily Mail were so impressed with the series that they ‘invested in a TV advert, which attracted new readers and helped boost newspaper circulation for that period – the ultimate measure of success in newspaper land!’ [ E11].

Through the publication of the book, and subsequent media engagement, Ellis has worked to make his research accessible to the public, to support individuals to identify and self-manage their sleep issues in an effective way. Case studies denoting people’s experience of following Ellis’ approach, published within the Daily Mail serialisation, highlight the benefit on their sleep:

‘I’ve seen a dramatic improvement in my sleep quality and I’m delighted […the solutions] were simple, but effective… I’m thrilled I’ve been able to sleep properly for the first time in nearly two years’ [ E10, p2-3].

‘Now I am getting at least three more hours of sleep every night. I can’t believe the vast improvement and I no longer dread going to sleep’ [ E10, p19].

58% of reviewers on Amazon gave the book 5 stars, with several stating it ‘worked’ and ‘really help[ed]’ [ E12]. One individual stated ‘by the 3rd night I was sleeping more deeply… I am falling asleep as soon as I go to bed now and sleeping deeply, waking only rarely now… I’ve broken the addiction to my phone and checking the bedroom clock… I am confident things will go on improving’ [ E12, p1].

As part of the Daily Mail serialisation, readers were provided with a 32-page sleep diary to use alongside the articles to develop a personalised plan to address their specific sleep issue. Readers were encouraged to send completed diaries to Ellis for analysis. Of the 347 diaries returned, 213 contained sufficient data for analysis [ E13]. Results revealed a statistically significant improvement in sleep, with regards to reductions in time to sleep onset (p=.001), waking after sleep onset (p=.002), and number of awakenings (p=.002) [ E13].

5. Sources to corroborate the impact

Ref. Source of corroboration Link to claimed impact
E1 Testimonial - Dr Renata Riha, Consultant in Sleep and Respiratory Medicine, Royal Infirmary of Edinburgh Demonstrates use of Prof. Ellis’ brief CBT-I within the Edinburgh Sleep Clinic
E2 Testimonial - First Steps Service: Dr Richard Thwaites - Cumbria, Northumberland, Tyne and Wear NHS Foundation Trust and Dave Sanford - Lancashire and South Cumbria NHS Foundation Trust Confirms use of Prof. Ellis’ brief CBT-I training and use of the intervention within IAPT services
E3 Testimonial - Dr Faith Orchard, Lecturer in Psychology, University of Sussex Confirms use of brief CBT-I in the Anxiety and Depression in Young People Research Clinic and Berkshire Healthcare NHS Foundation Trust
E4 Testimonial - Dr Chan-Young Kwon, Specialist in Neuropsychiatry of Korean Medicine, Republic of Korea Demonstrates influence of Prof. Ellis’ work on brief CBT-I on clinical understanding and practice in Korea
E5 Jason Ellis, Cushing, T., and Germain, A. (2015) ‘Treating Acute Insomnia: A Randomised Control Trial of a ‘Single-shot’ of Cognitive Behavioral Therapy for Insomnia’ Sleep 38(6), pp. 971–978 Demonstrates the brief CBT-I intervention is effective at addressing sleep issues
E6 Orchard, F. et al. (2019) ‘Adapting Brief CBT-I for Depressed Adolescents: A Case Illustration of the Sleeping Better Program’ Cognitive and Behavioral Practice 27(3), pp. 336-346 Demonstrates success of brief CBT-I when used with young adults through an evaluation conducted at the Anxiety and Depression in Young People Research Clinic
E7 Crew, E. C. et al. (2020) The Society of Behavioral Sleep Medicine (SBSM) COVID-19 Task Force: Objectives and Summary Recommendations for Managing Sleep during a Pandemic Behavioral Sleep Medicine, pp.1-3 Ellis’ work on brief CBT-I is recommended as a way to manage insomnia during COVID – cites three D’s from R4
E8 Riemann, D. et al. (2017) ‘European guidelines for the diagnosis and treatment of insomnia’ Journal of sleep research 26(6), pp. 675-700 Demonstrates EU recognition of brief CBT-I, such as Ellis’ work, for use with individuals with acute insomnia
E9 Website – Royal Surry Insomnia Clinic Demonstrates that Surrey Hospital Insomnia Unit recommends Prof. Ellis’ book to patients as a form of self-help
E10 Media coverage - a) Daily Mail Serialisation published February 2017 (p.1-85) b) This Morning, ITV.com – How to banish your insomnia in one week (p.86-91) Demonstrates the existence of Daily Mail serialisation of Prof. Ellis’ book The One-Week Insomnia Cure Confirms Prof. Ellis’ appearance on This Morning to discuss his research
E11 Testimonial - Louise Atkinson, Freelance Journalist Confirms that Prof. Ellis’ book was serialised in the Daily Mail and the views of publishers and readers
E12 Amazon reviews for The One Week Insomnia Cure (accessed 25/02/2021) Showcases customer feedback on the book, demonstrating benefit to sleep
E13 Audit summary – Daily Mail sleep diary analysis. Raw data available on request Demonstrates statistically significant difference (improvement) in sleep quality between first and final entry
Submitting institution
University of Northumbria at Newcastle
Unit of assessment
4 - Psychology, Psychiatry and Neuroscience
Summary impact type
Technological
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

The market for brain health supplements is estimated to be worth up to EUR1,000,000,000 in the European Union and at similar levels in the USA. Research is essential to support the development and promotion of these products. The Brain, Performance and Nutrition Research Centre (BPNRC) at Northumbria University conducts trials evaluating the effects dietary supplements and their component parts have on brain function and behaviour. The leading international manufacturers of nutritional and dietary ingredients and supplements use Northumbria’s research to learn about the nature and extent of the effects of their products, to develop new products and ingredients, and to support marketing and sales. Specific examples include: [text removed for publication] (iii) establishing the efficacy and effective dose of International Flavors and Fragrances Inc’s (IFF) popular Neuravena® product; and (iv) Bayer HealthCare’s use of research to support claims, and sales growth, for its Supradyn TM product.

2. Underpinning research

Northumbria University’s Brain, Performance and Nutrition Research Centre (BPNRC) conducts research evaluating the effects of dietary components on brain function and behaviour. These include micro- and macronutrients, bioactive phytochemicals, whole foods (e.g., nuts), and dietary supplements (e.g., herbal extracts). All studies, including those cited here, are conducted to the rigorous standards of human clinical trials, including pre-registration, the adoption of randomised, placebo-controlled, double blind-designs, and the use of sensitive study outcomes. Parameters such as cognitive function (e.g., attention, working memory, episodic memory, and executive function) and mood are assessed using Northumbria University’s Computerised Mental Performance Assessment System (COMPASS), a flexible software framework designed by Northumbria to deliver customised cognitive assessments [ R1].

[Text removed for publication]

[Text removed for publication]. Similar results were found for Neuravena®, a product containing Avena sativa extract and made by IFF Health, a manufacturer of functional natural health and nutrition ingredients. Repeated assessment using COMPASS across a single testing visit following acute doses (800mg and 1600 mg) of Neuravena® revealed broad cognitive benefits of the 800mg dose. Benefits included improved working memory, episodic memory, executive function, and speed of performance across all tasks, evident from one-hour post-dose in healthy adults (n=42, repeated measures). As such, this study established that the optimal dose of Avena sativa is lower than previously thought, at 800mg or less [ R4].

Since 2008, Northumbria has collaborated with Bayer Consumer Care AG (Basel), an international enterprise that manufactures several leading global brands of multi-vitamin/mineral (MVM) supplements. The original research demonstrated that Bayer’s BeroccaTM and SupradynTM MVM supplements were associated with improved cognitive function and mood in healthy adults. A subsequent series of studies carried out during 2012-2018 expanded this research base by assessing updated formulations and novel applications for these products. The aim of this body of research was to explore whether the metabolic effects of water-soluble vitamins underpinned changes in physiological or psychological function during physically or cognitively-demanding activities. In one study, research assessed the impact of acute doses of Berocca BoostTM, an MVM supplement also containing guarana, an extract with stimulant properties derived from the seeds of the Amazonian Paullinia cupana plant. In the study, the consumption of Berocca BoostTM prior to a bout of moderate-intensity exercise resulted in performance benefits in terms of lower perceived exertion during exercise in active males (n=40, repeated measures) [ R5]. Furthermore, performance on working memory (accuracy) and episodic memory (speed) tasks – assessed before and after exercise, using the assessment tool COMPASS – was also significantly improved [ R5]. Another study combined multiple methods: metabolic (measuring O2 consumption and CO2 production), cognitive (using COMPASS), and cerebral blood flow (NIRS). The study focused on the metabolic effects of single doses and also chronic (over a period of 56 days) supplementation with SupradynTM in healthy females (n=120) [ R6]. This research found that (a) acute supplementation modulated metabolic parameters, increasing energy expenditure and cerebral blood flow during cognitive task performance following a single dose, and that (b) these metabolic consequences were sustained during chronic supplementation [ R6].

3. References to the research

R1. David Kennedy, Fiona Dodd, Bernadette Robertson, Okello, E.J.**, Jonathon Reay, Scholey, A. B.**, and Crystal Haskell ( 2011) Monoterpenoid extract of sage (Salvia lavandulaefolia) with cholinesterase inhibiting properties improves cognitive performance and mood in healthy adults. Journal of Psychopharmacology 25(8): 1088-1100 https://doi.org/10.1177/0269881110385594

R2. David Kennedy, Pace, S.*, Crystal Haskell, Okello, E.J.**, Milne, A.*, and Andrew Scholey ( 2006) ‘Effects of cholinesterase inhibiting sage (Salvia officinalis) on mood, anxiety and performance on a psychological stressor battery’ Neuropsychopharmacology 31(4): 845-52 https://doi.org/10.1038/sj.npp.1300907

R3. Emma Wightman, Philippa Jackson, Julie Khan, Joanne Forster, Heiner, F.**, Feistel, B.**, Suarez, C.G.**, Pischel, I.**, and David Kennedy ( 2018) ‘The Acute and Chronic Cognitive and Cerebral Blood Flow Effects of a Sideritis scardica (Greek Mountain Tea) Extract: A Double Blind, Randomized, Placebo Controlled, Parallel Groups Study in Healthy Humans’ Nutrients 10(8): 955 https://doi.org/10.3390/nu10080955

R4. David Kennedy, Philippa Jackson, Joanne Forster, Julie Khan, Grothe, T.**, Perrinjaquet-Moccetti, T.**, and Crystal Haskell-Ramsay ( 2017) ‘Acute effects of a wild green oat (Avena sativa) extract on cognitive function in middle-aged adults: a double-blind, placebo controlled, within-subjects trial’ Nutritional Neuroscience 20(2): 135-151 https://doi.org/10.1080/1028415X.2015.1101304

R5. Rachel Veasey, Crystal Haskell-Ramsay, David Kennedy, Wishart, K.**, Maggini, S.**, Fuchs, C.J.*, and Stevenson, E.J.* ( 2015) ‘The Effects of Supplementation with a Vitamin and Mineral Complex with Guaraná Prior to Fasted Exercise on Affect, Exertion, Cognitive Performance, and Substrate Metabolism: A Randomized Controlled Trial’ Nutrients 7: 5272 https://doi.org/10.3390/nu7085272

R6. David Kennedy, Stevenson E.J.**, Philippa Jackson, Sarah Dunn, Wishart, K.**, Bieri, G.**, Barella, L.**, Alexandra Carne, Fiona Dodd, Bernadette Robertson, Joanne Forster, and Crystal Haskell-Ramsay ( 2016) ‘Multivitamins and minerals modulate whole-body energy metabolism and cerebral blood-flow during cognitive task performance: a double-blind, randomised, placebo-controlled trial’ Nutrition and Metabolism 13:11

https://doi.org/10.1186/s12986-016-0071-4

*Internal collaborators: C. Fuchs and E. Stevenson – Department of Sport, Exercise and Rehabilitation (UoA24); S. Pace – employed at Northumbria until 02/2005; A. Milne – technical support within BPNRC since 2013

**External collaborators: E. J. Okello – Newcastle University; A. B. Scholey – Swinburne University, Australia; F. Heiner, B. Feistel and C. G. Suarez – Finzelberg GmbH & Co.; I. Pischel – consultant; T. Grothe and T. Perrinjaquet-Moccetti – Frutarom, Switzerland; K. Wishart, S. Maggini, G. Bieri and L. Barella – Bayer HealthCare, Switzerland

4. Details of the impact

BPNRC’s research on the effects of dietary components has had impact in three distinct areas, in relation to (1) bringing to market new dietary supplements, including [text removed for publication], (2) improving dose recommendations for Neuravena, and (3) supporting the international marketing initiatives of one of Europe’s biggest-selling supplements.

4.1 Bringing new supplements to market

[Text removed for publication]

4.2 Improving dose recommendations for Neuravena® ( Avena Sativa)

IFF Health – a division of International Flavours and Fragrances Inc. – is a leading global natural health and nutrition ingredient manufacturer. IFF Inc. had sales of USD5,100,000,000 in 2019. Northumbria research was used to inform a new lower dose of IFF’s Avena sativa extract product, Neuravena. The research provided new insights by demonstrating that 800mg was sufficient to have a positive effect [ R4]. IFF Health states that ‘this dose is lower than what was previously used (1250mg), and current dose recommendations are based on the study results’ [ E8]. This discovery is also important for consumers as ‘a dose of 800mg is more economically viable as well as user friendly (one capsule instead of two, for example)’ [ E8]. This further benefitted the company, as it supports ‘promoting the ingredient to new markets and new users’ [ E8]. In addition, using COMPASS in this study ‘enabled us to check for many attributes of the extract at the same time, and is regarded by many in the market as an advanced method for checking effects on cognition’ [ E8]. The results revealed improved speed of performance across all the cognitive tasks and these data were used to develop a new concept showing how to use Neuravena in products targeting consumers involved with video games [ E8, E9]. Finally, the research supports the company to meet regulatory requirements around the world:

‘the study is integral for establishing the efficacy and safety of the extract with regards regulatory processes. Especially in the USA and EU, where we mainly operate and have many customers, the sale of nutraceutical products is governed by strict rules and standards, so the research is very important so that we can meet those requirements’ [ E8].

4.3 Supporting international marketing initiatives

Bayer Consumer Care AG (Basel) has used Northumbria’s research [ R1, R2] to promote its products and recognises the commercial value of the research findings. Bayer states that ‘these studies help differentiate the products in the marketplace via the development of more distinctive claims’ and ‘play a substantial role in their commercial success’ [ E10]. For example, SupradynTM which is marketed in Europe, Latin America, Africa, and Asia Pacific, and is worth more than EUR200,000,000 in annual sales – ‘continues to gain further market share since 2 years, proving its stronger equity and execution vs. its competitors in the market; the brand grew globally +7%, vs. the category growth of 4%’ [ E10]. In order to achieve this, Northumbria research was used to produce new marketing materials that have been used ‘to support claims about product effectiveness (i.e., works from the first intake, clinically/scientifically tested/proven) [ E10, E11]. The research has also been used ‘to train the Bayer medical staff in the company’s affiliates around the world, to train the sales forces (targeting pharmacies) and consequently the pharmacy staff’ [ E10]. Additionally, the publications are used as ‘references in documents submitted to health authorities for claims substantiation’ [ E10]. In summary Bayer stated the collaboration with Northumbria University ‘is crucial for the continued success of our products’ [ E10].

5. Sources to corroborate the impact

Ref. Corroborating Source Link to Impact
E1 [text removed for publication] [text removed for publication]
E2 [text removed for publication] [text removed for publication]
E3 [text removed for publication] [text removed for publication]
E4 [text removed for publication] [text removed for publication]
E5 [text removed for publication] [text removed for publication]
E6 [text removed for publication] [text removed for publication]
E7 [text removed for publication] [text removed for publication]
E8 TESTIMONIAL - Dr. Itay Shafat, Product Manager Cognitive Line and Sports Nutrition, IFF Health Underwrites all changes/benefits at IFF Health
E9 PRODUCT BROCHURE - IFF Health Neuravena for e-gamers Cites Kennedy et al. 2017 (R6) study findings
E10 TESTIMONIAL - Silvia Maggini, Global Medical Category Head, Nutritionals and Digestive Health, Bayer Consumer Care AG Underwrites all changes/benefits at Bayer
E11 (source: YouTube) Cites Kennedy et al. 2016 (R2) and uses the phrase ‘works from the first time’ which was derived from the research findings
Showing impact case studies 1 to 5 of 5

Filter by higher education institution

UK regions
Select one or more of the following higher education institutions and then click Apply selected filters when you have finished.
No higher education institutions found.
Institutions

Filter by unit of assessment

Main panels
Select one or more of the following units of assessment and then click Apply selected filters when you have finished.
No unit of assessments found.
Units of assessment

Filter by continued case study

Select one or more of the following states and then click Apply selected filters when you have finished.

Filter by summary impact type

Select one or more of the following summary impact types and then click Apply selected filters when you have finished.

Filter by impact UK location

UK Countries
Select one or more of the following UK locations and then click Apply selected filters when you have finished.
No UK locations found.
Impact UK locations

Filter by impact global location

Continents
Select one or more of the following global locations and then click Apply selected filters when you have finished.
No global locations found.
Impact global locations

Filter by underpinning research subject

Subject areas
Select one or more of the following underpinning research subjects and then click Apply selected filters when you have finished.
No subjects found.
Underpinning research subjects