Impact case study database

These impacts arise from (1) specific research outputs reframed in accessible format for teachers and students of English Language A level through workshops, classroom materials and an online course, and (2) from the body of research of individuals and teams informing design of the online course and expert consultancy. The engagement activities were led by case study author Sam Hellmuth, working with a wide range of colleagues across the Department.

English Language was introduced as an A level subject in 2000 and expanded rapidly; in 2020, 15,053 students sat English Language A level in the UK [ Joint Council for Qualifications Provisional GCE A Level Results June 2020]. A 2006 HEA English Subject Centre report [5.1] highlighted a specific need for i) accessible professional development opportunities for teachers who deliver English Language A level, who typically have limited training in linguistics, and ii) resources to support transition into Higher Education of students of English Language A level.

To respond to this need, in a series of York English Language Toolkit Continuing Professional Development (CPD) workshops, we have pioneered an approach which fosters knowledge exchange between researchers and teachers, in two ways. Firstly, each York research case study is presented by its author(s), with an opportunity for discussion, so teachers interact with experts in each linguistic sub-field. Secondly, we propose classroom materials on A Level curriculum topics linked to the research, which teachers can freely adapt. Providing our own idea of how to translate the research for the classroom creates common ground for mutual exchange.

Eighteen research case studies [including each of 3.1-3.6] with accompanying open access classroom materials, developed for a series of CPD workshops held between 2014 and 2020, are available through a multimedia website (launched 2019) [ englishlanguagetoolkit.york.ac.uk]. The first on-campus one day workshops were regional (e.g. 13 teachers in 2014) but later recruited nationally (29 in 2015 and 35 in 2019) with delegates from as far afield as South Wales. The 2020 workshop ran online and attracted 582 pre-registrations including 546 UK teachers of whom 410 report teaching the AQA specification. Classroom materials and pre-workshop talks were posted in advance and the live webinar was attended by 318 delegates reporting affiliation to a UK school/college [5.2]. 2020 workshop resources have been accessed 3,400 times (ranging from 60-150 per worksheet) and YouTube views of 2020 workshop talks total 937 (to 31/12/20) [5.2]. Evaluation comprises immediate post-workshop surveys and email feedback [5.3], two teacher interviews [5.4] and two surveys run 6 months or more after a workshop [5.5].

Teachers new to the subject who have attended the workshops report increased confidence: “The range of resources on the York website are excellent and have given me the confidence to start teaching English language in September” [5.3]; “I’ve felt more confident about utilising these ideas in the classroom” [5.5]. More experienced teachers also value a chance to update their knowledge: “They get me excited, for a start, and it means rather than delivering the same old PowerPoint I say, ‘Oh great, let’s add something in.’ So [students are] getting a broader remit from me as a teacher” [5.4].

Teachers report using the materials to enhance their teaching practice, with effective uptake by students: “[D]uring Covid-lockdown, I was able to share the website and resources easily with my students via Google Classroom. Part of their wider reading homework was to study the materials [on 3.2] and some used the information really effectively in their essays” [5.3]. A Sixth Form College teacher reports: “I’ve already forwarded some of the online resources to students who are looking to extend their wider reading over the summer. [Name] is hoping to study Law and I know that she’s been excited to look at some of the courtroom data in relation to Paul [Drew]’s talk [3.1] and [Name] is looking at accent bias so the discussions from Dom [Watt] [3.4] will ... add an extra dimension to what she knows already” [5.3]. Another teacher noted: “The up to date research on courtroom questioning [3.1] has become an integral part of my teaching of occupational lang, and has helped more than one student with their NEA [Non-exam assessment] investigations” [5.5]. Another teacher reports that a student was inspired by the ‘be like’ case study [3.6] to collect data on inter-generational family use of ‘be like’ for their NEA coursework: “this was just spot on for a language investigation in terms of scope, data size and findings” [5.3].

Comments from teachers who attended several workshops show sustained impact on practice, with evidence of forward knowledge transfer to other teachers: “This year I've presented all the workshop materials and ideas ... for training three other English Language A level teachers; with a y12 class I've used the material presented by Paul Kerswill on 'Jafaican' [3.5] as part of their study of dialect changes and public perception of/attitudes to these” [5.5].

In a survey six months after the 2020 webinar (43 respondents, 13.5% of attendees): 77% report using materials in teaching and 67% report the materials or webinar changed how they teach; 70% report positive outcomes for students as a result [5.5]. Free text comments give concrete examples of: enhanced knowledge, understanding and confidence of teachers and/or of students; use of the materials in non-exam coursework; adjustments to teachers’ approach to the A Level course; contributions to enhanced remote learning during Covid lockdown; and inspiration for students to apply to study linguistics at university [5.5]. One teacher writes: “We have updated our A Level English Language resources on child language development and discourse analysis with handouts and tasks from the talks. This has benefited the students enormously, as they have been able to engage with university level content and research” [5.5].

Building on our experience working with teachers, in 2019 we added a student-facing resource. The FutureLearn Massive Open Online Course (MOOC) Introduction to Sociolinguistics: Accents, Attitudes and Identity presents methods and findings of research in language variation and change [3.3-3.5] in a step-by-step format which assumes no prior knowledge of linguistics. It is an accessible self-study tool for A level English Language students, useful also for teachers and the general public. The September 2019 run had 1,137 active learners (defined as a learner who has completed at least one course step), of which 304 (19.6%) were ‘fully participating learners’ (FPL) who completed >90% of the course. The 2020-21 run has 3,766 active learners and 1,113 FPL (21.9%) to date [to 31/12/20], and in the end of course survey (665 responses to 31/12/20) 96% of respondents said they had acquired new knowledge/skills [5.6]. Comments at course completion show that learners who are A level students apply what they have learned to their studies and display renewed confidence to pursue further study: “It was so interesting to learn more in depth about something I am so passionate about and am definitely going to study at university. ... There are several aspects from the course I can take and put into my own English Language A-Level and expand my answers further”; “I am hoping to study Linguistics with Spanish at [Institution], and this is has actually given me a real insight into some of the things I will be looking at. … I have also been able to make links with a couple of the subject areas I am currently studying (mainly my English language) so that has been a great help” [5.6].

After recommending the Sept 2019 MOOC to a student, a Sixth Form College teacher reports: “One of the reasons she wanted to do [the MOOC] was to get confidence for university debating and discussion, as she’s off to do speech and language therapy. Across the year, her confidence was exponentially higher. She went from a grade C to the strongest piece of NEA [coursework] in the year, which would have been at A* level. Her mock exam was an A. It was really, really lovely. I think it gave her the confidence - she started doing the wider reading - it was a really good leaping off point for her. And she did start to get that bit of evaluation; that really clicked for her quite early on, which was a real benefit” [5.4]. This college incorporated the MOOC directly into their curriculum: “With the blended learning that we’re doing, it’s going to be written into the scheme of work that they should be working through that” [5.4].

AQA 7702 English Language is the market-leading specification for AS and A level English Language in the UK [5.1]. The current specification was revised in 2015 to meet government requirements and renew subject coverage. Since 2015 we have tailored our CPD materials to the adjusted structure and content of the new AQA specification. Fit to the specification is demonstrated in the Toolkit website ‘index’ page [ englishlanguagetoolkit.york.ac.uk/index], which was prepared by a teacher consultant and is organised by components of the AQA assessment.

A senior examiner and lead moderator for AQA English Language A/AS level writes: “One of the aims of the 2015 AQA specification was to refresh the content of this course and bring it up to date. AQA wanted to offer links to more recent research and encourage students and teachers to keep abreast of developments in contemporary linguistics and language study, and the York English Language Toolkit has been a key resource in helping teachers of the AQA specification achieve this goal” [5.7]. Comments from teachers confirm that the materials provide ‘stretch’ in ways that closely fit the specification: “I really appreciate the graphs as these are often used by AQA to support questions and also help students to visualise the information. It also shows them ways that they can present their own work in an investigation” [5.3]; “to get into band five, which is the top of our bands, [the students] need to be constantly evaluative, they need to be testing out ideas, and they need to be brave enough to challenge published writers in Paper Two. The ability to do that with real data [from the case studies] is useful for both them and us” [5.4].

Successful translation of research content into students’ assessed work is corroborated by the AQA Senior Examiner: “It is clear that teachers and pupils have used the toolkit very effectively. Material from Toolkit sessions has appeared in various forms of assessed student work, including exams and NEA investigations. Debates about ‘Jafaican’ (MLE) have been prominent [3.5] in this work, while many students have chosen to look at representations of online identity and research accent and social attitudes [3.3]. The use of such case studies, derived from the toolkit, allowed students to fully explore and engage with the specification. Many of the themes and approaches adopted by students drawing on the case studies are those that allow them to access the higher bands of the mark scheme” [5.7]. They conclude: “The York English Language Toolkit has been an important resource in helping teachers develop their students’ understanding in the course, with up to date research and resources for tackling the revised AS and A Level English Language specification. It has clearly proven popular with teachers and students alike, and has been used by students in assessed work of the highest standards” [5.7].

Cambridge International 9093 is an international AS/A Level English Language qualification. It was taken in 2018-19 by 27,920 students in 1,164 schools in 104 countries, and in 2019-20 by 27,111 students in 1,036 schools in 103 countries [5.8]. In 2016, Cambridge International commissioned York linguistics researchers to review the content and scope of the AS/A level specifications for English Language (9093). The report [5.9] (authors: Hellmuth, Keren-Portnoy, Kerswill, Sells) was informed by our experience of working with UK teachers, but drew primarily on the body of research by Keren-Portnoy on early child language development [e.g. 3.2] and by Kerswill on the influence of class and ethnicity on language variation and change [e.g. 3.5].

Inspection of the new specification published in 2018 shows that our recommended changes to scope and relevance were taken up [5.10]. The Development Manager in the Qualifications Development team at Cambridge Assessment International Education confirms: “[The York] report played a significant role in shaping our understanding of linguistics and the skill of data and language analysis. As a result of the report, we have made a number of changes to the syllabus to better reflect the progression for students entering university in order to undertake a course in English or Linguistics. These changes include: inclusion of linguistic elements within the subject content for Paper 1 and Paper 2 ...; addition of many linguistic terms such as phonology and morphology for Paper 1 and Paper 2; addition of a new Section A in Paper 3 on Language change ....; amendment to the Child Language Acquisition section from it being very broad and covering years 0-18 to now concentrating on a shorter time period of years 0-8 ...; change in title to Section A within Paper 4 - it is now called English in the World rather than English as a Global Language” [5.8]. The Development Manager concludes: “We believe that the revised syllabus will provide greater progression for students wishing to study English or Linguistics at undergraduate level at University, as well as introducing students to new and relevant skills such as data analysis, and exciting and engaging topics such as Language change” [5.8].

Forensic Speech Science (FSS) is the analysis of speech and audio for evidential and investigative applications. Since August 2013, York research has had considerable impact on the field. Specifically, we have (1) guided national-level Government policy and (2) contributed to high-profile national and international casework and informed professional practice in the field.

Background: In the UK alone, there are an estimated 600 cases per year involving FSS analysis, and J P French Associates (JPFA) has consulted on more of these than any other expert or firm (around 120-200 cases and consultations per year) [5.1a]. Law enforcement and security services also regularly use speech and audio analysis for investigative purposes. Outside the UK, FSS casework is often undertaken by Government forensic laboratories. However, as many countries do not have dedicated labs, JPFA regularly conduct casework internationally, especially in high profile cases [5.1a]. FSS methods have developed considerably in recent years, most notably through the adoption of automatic speaker recognition (ASR) software and new frameworks for expressing conclusions. Forensic science, as a whole, is under increasing pressure to improve standards, in light of high-profile miscarriages of justice based on unscientific and untested forensic evidence. The UK Government Forensic Science Regulator is responsible for improving the quality of forensic evidence, through a process of accreditation, in line with international standards (ISO/IEC17025). Accreditation is becoming a crucial factor in courts’ decisions about the admissibility of expert evidence and the weight attached to it.

Impact:

• In 2013, the Forensic Science Regulator appointed French to chair a five-member committee for Forensic Speech and Audio, which included Harrison and Watt. As confirmed by [text removed for publication], York members were recruited on the basis of their long-standing research expertise in the field [5.1b], underpinned by publications such as [3.1]. The committee produced an Appendix to the Regulator’s Code of Practice and Conduct, published in 2016 [5.2a] with a second issue published in 2020 [5.2b]. The Appendix provides specific guidance on record keeping, handling of materials, methods and validation in FSS. This is the Government’s official guidance for the area, and it is incumbent on experts to follow the recommendations. The Regulator’s Code of Practice, of which the Appendix is a part, will be used when assessing forensic laboratories for accreditation purposes [5.2c] and so it will be essential to show compliance with the guidance. As a result of the guidance in the Appendix, [text removed for publication] is now “including statements (in reports) that note equipment can have impact on the quality of … outputs” [5.1b]. York members “continue to provide guidance through the committee as part of the Regulator’s ongoing efforts to update and streamline the Code of Practice for the forensic science sector” [5.1b].

• In 2015, our research informed an official briefing document for MPs on forensic linguistics and phonetics [5.3], produced by the Parliamentary Office of Science and Technology (POST). The document was written in consultation with French and Watt, and makes extensive reference to our research, particularly in speaker comparison [3.1] and Language Analysis in the Asylum Process (LAAP) [3.5]. The document was used by Roger Mullin MP as the basis of a Private Members Bill to introduce standards into the field, which was brought before the House of Commons in 2016 [5.1c, 5.4]. The Knowledge Exchange Lead at POST states that “ French and colleagues provided further insights, crucial to my understanding of the topic, which fed into my drafting of the briefing” and that “expert insights provided by researchers from York, supported by their publications, were key to the development of the POSTnote [briefing] on Forensic Language Analysis” [5.1c].

• A BBC investigation uncovered alleged cheating on English language oral tests used as part of the UK immigration process. The company administering the tests (ETS) used an automatic speaker recognition system along with trained human listeners to analyse whether the same voices of ‘proxy exam sitters’ were recurring in their data. In 2016, French was asked by the Home Office to produce a report [5.5] outlining how well ETS’s methods were able to detect fraudulent tests. French’s opinion was that, on the basis of the information provided to him, the testing process was robust. French’s report, which makes direct reference to an earlier version of work which became [3.2], has informed parliamentary process, having been cited by the Home Office in its response to the House of Commons Home Affairs Committee [5.6a]. It has also been cited in at least five immigration tribunal rulings as the basis for not overturning asylum decisions [5.6b]. Both French and Harrison were called to a meeting of the All Party Parliamentary Group on Tests of English for International Communication in the House of Commons in 2019. Their evidence is referenced extensively in the group’s report [5.6c].

• York is a world-leading centre for research-led FSS training, through which we have influenced the practice of FSS practitioners around the world. York’s Department of Language and Linguistic Science offers the world’s only postgraduate (MSc) degree in FSS. The programme is heavily informed by our research and provides training in all elements of FSS underpinned by [3.1, 3.5, 3.6], as well as speech technology and forensic statistics underpinned by [3.2, 3.3]. As of August 2020, at least 20 of our graduates now work within FSS, police, law enforcement and security, and Government departments and organisations both nationally and internationally [5.7], [text removed for publication] [5.1d]. According to a Forensic Audio and Video Analyst at the Royal Canadian Mounted Police (RCMP), and former student on the MSc, the programme provides “both the theoretical and practical basis for … day-to-day casework” [5.1e].

• In 2016, French, Harrison, Hughes, and Watt developed a Continuing Professional Development course in FSS. Delivery of the course is made possible through a partnership with Nuance Communications, the world’s largest speech technology company, who provide automatic speaker recognition software for teaching and research (used in [3.3]). The course has run six times between 2016 and 2019, with a total of 61 participants, the majority of whom (approximately 70%) were from national and international law enforcement and security services. Course content is heavily underpinned by our research [3.1], with a particular focus on understanding the principles of automatic systems [3.2, 3.3]. The course has changed practice at [text removed for publication]: it is now part of the training programme for recruits who are new to the field as it “equips them with the knowledge and understanding to more effectively carry out casework” as well as reinforcing “current best practices among ... lab staff” [5.1b]. An Audio Examiner from North Wales Police also said that knowledge gained from the course has “taken a lot of time-consuming ‘guesswork’ out of what I do, now that I know I can expect a particular result by applying a particular technique” [5.1f].

• We have continued to inform the work of JPFA. The company consists of four full-time members of staff, all of whom hold postgraduate degrees in FSS from York, as well as French and Harrison. The relationship between York and JPFA is a long-standing one and the appointments of French and Harrison at the University reflect a strategic decision to strengthen this relationship, providing a more direct pathway for our research to have impact on the majority of forensic speech and audio casework nationally and internationally. The JPFA Research Manager states that “York research is essential in guiding (JPFA) casework practice, both in terms of methods and frameworks for evaluating evidence” [5.1a]. The relationship between JPFA and York, and the benefits to JPFA from York research, have also been identified as reasons for JPFA’s high standing within the field by external organisations [5.1b, 5.1d].

• In their roles at JPFA, French and Harrison regularly conduct casework that affects the outcomes of criminal trials. For example, in 2014, York research on accent variation in the North East of England [3.4] was used by French to inform a speaker comparison in the case of the murder of David Wilson in Sunderland. French’s analysis of a telephone call made to police from the victim’s phone helped police identify where the suspect was from [5.8a]; that call was described by DCI Mark Ord as a “key piece of evidence” [5.8b] in convicting Daniel Johnson. Our research has also informed high-profile international cases. For example, in 2018, French re-transcribed audio recordings for the Supreme Court of the Australian Capital Territory in the retrial of David Eastman, using the protocol in [3.6]. The transcriptions weakened the prosecution’s claim that the recordings contained admissions to murder. This expert evidence was delivered by French personally from the witness box and the judge referred to it when instructing the jury in the retrial, which ended in Eastman’s acquittal [5.9].

• At a national level, York research has informed [text removed for publication] [5.1d]. Our research has also had substantial international impact on casework practices in Government laboratories in Europe and North America. York research “significantly contribute(s) to the scientific underpinning of forensic casework” at the Netherlands Forensic Institute (NFI) [5.1g]. Papers such as [3.5] are used by the Netherlands Immigration and Naturalisation Service in LAAP cases and provide “much needed empirical investigation of language analysis” for such cases [5.1h]. According to a Forensic Speech Scientist from the Bundeskriminalamt (BKA; German Federal forensic laboratory), York is “consistently producing research highly relevant for forensic institutes like ours at the BKA” and is “the only institute worldwide that pursues this line of research with this level of intensity and practical relevance” [5.1i], with specific reference made to our work on automatic speaker recognition [3.2, 3.3]. Finally, York research (with specific reference to [3.2]) informs forensic analysis in all cases conducted by the Royal Canadian Mounted Police Audio Video Analysis Unit (approximately 25 cases per year) and means that the RCMP “is more confident in ... internal forensic casework practices and produce(s) more reliable and robust analysis to support investigations across Canada at all levels of law enforcement” [5.1e].

• In 2014, French, Harrison, and Hughes presented a research talk (on work that led to [3.2]) at a workshop run by the European Network of Forensic Science Institutes (ENFSI). ENFSI is a network of public forensic science laboratories across Europe that has the aim of promoting knowledge exchange and developing and implementing international standards. The outcome of this workshop was a set of guidelines for the use of automatic speaker recognition systems in forensic casework [5.10]. The guidelines make extensive reference to York research, including [3.1]. According to an author of the guidelines, York staff “made important comments that led to improvements of the guideline document” which has “become an integral part of casework at the BKA” [5.1i].

As evidenced above, York research has informed, and responded to, changes within the field of FSS and across forensic science more generally. By informing policy and practice, at both national and international levels, our research has had wide-ranging and far-reaching impact that has shaped the field and will continue to do so in the future.