Impact case study database

Impact on accessibility of published research

Professor Marsden’s research has been key to fundamental changes in the working practices of international peer-reviewed journals – persuading them to take part in two Open Science initiatives of her own creation:

• IRIS (Instruments for Research into Second Languages): a collection of instruments, materials, stimuli and data coding and analysis tools for research into second languages – all freely available to anyone in the world (5,061 materials held and 49,691 downloads by December 2020 https://www.iris-database.org/).

• OASIS (Open Accessible Summaries in Language Studies): a free, searchable collection of language learning research findings, offering practitioners non-technical one-page summaries of published papers – explaining the aims of the research, the methods used, and the results found (627 summaries currently held; live since January 2018;17,333 downloads by December 2020 https://oasis-database.org/).

The Editor of the Modern Language Journal (MLJ) explains how these changes came about and why they are significant: “Emma and her colleague (Kasprowicz) published a study providing powerful and sobering evidence about the extent and nature of the lack of engagement with research among language educators... Driven by her findings, Emma gathered together approximately 16 editors of key academic journals at a conference in 2017. There and in the following few months she led the development of a new genre – the Accessible Summary – and proposed mechanisms by which journals across the field might work together and contribute such summaries to the same freely accessible digital resource [OASIS], working across publisher boundaries.

Now, the MLJ requires every author to write a one-page non-technical summary of their study, following the OASIS guidelines. These summaries are all sent to the OASIS website and made openly and freely accessible. They are also each posted online on our own journal website … The impact this change has had on the publication process and dissemination of academic research is unprecedented, extensive, and highly significant. Previously, only those with (access to) subscriptions to the journal could read about research relating to the learning and teaching of foreign languages. Now these findings are available to all … Emma’s work on the benefits of open science was influential in persuading us to introduce this new policy” [1a].

OASIS currently holds 62 summaries of research published in the MLJ, and MLJ’s authors have 248 resources held on IRIS. The MLJ is one of 38 journals that now ask authors to make materials available through IRIS, and one of 16 of the 25 most highly cited and respected MFL journals to ask authors to write summaries for OASIS. This includes journals published by all the major publishing houses: Cambridge University Press, Oxford University Press, Springer, Routledge and Wiley.

These constitute fundamental but sustainable (journal-driven) changes in the relationship between academics, publishers and practitioners, opening up a pipeline to convert research into everyday evidence-informed language education around the world. According to the President of the Center for Applied Linguistics (USA), “previous attempts to communicate academic research to practitioners have been ad-hoc and short-lived” [2] but, in the words of the Editor of Language Learning, Marsden’s creation of a sustainable new approach is “ *truly transforming the game of research in our field” [1b]**.

Impact on government policy

In recognition of her research and her success (via IRIS and OASIS) in establishing a robust new relationship between research and practice, Professor Marsden was invited by the Teaching Schools Council to be a key advisor on its MFL Pedagogy Review (2016; known as the ‘Bauckham Review’). The Chair of that review says of Marsden’s contribution: “Studies conducted by Emma and her colleagues formed a key part of the evidence base for the report’s recommendations” [3] .

Launched at the British Council and the House of Lords in November 2016, the report [4] highlighted the misalignment between the research and practice of effective MFL teaching. It was formally welcomed in the House of Lords where the Parliamentary Under-Secretary of State for Education urged schools to consider its recommendations (Hansard, 7th December 2016) [5a], and by the Schools Standards Minister [5b]. In the spring of 2017, the DfE arranged for a MFL Mobilisation Group to consider strategies for acting on the report’s conclusions. This led to a proposal by the Schools Standards Minister to establish a National Centre for Excellence for Language Pedagogy (NCELP) – with an explicit remit to ensure that “the principles and recommendations of the Bauckham Review will be followed so as to improve language teaching practice in secondary schools” [6].

The findings of the report – and the resulting plan to establish NCELP – were also cited in a report on improving social mobility through education presented to Parliament in December 2017 by the Secretary of State for Education and Minister for Women and Equalities. It states: “We will make it our mission to improve access to high quality modern foreign languages subject teaching, particularly for disadvantaged pupils, drawing on the findings of the Teaching Schools Council review” [7]. Deputy Director of the DfE’s Curriculum Policy division commented: “ The report has had significant influence on Department policy, Ofsted school inspection practice, the GCSE examination boards, Ofqual regulation practice, and the work of educationalists, education publishers and the MFL subject associations” [8] . The Chair of the MFL Pedagogy Review writes: “ The report has changed, and will continue to change, the behaviour and thinking of policy-makers, text-book writers, the school inspectorate, GCSE examination boards, Ofqual, teacher educators, and classroom practitioners [3] .

Impact on execution of policy

Drawing on her key contribution to the MFL Pedagogy Review, Professor Marsden made a successful application to set up and direct NCELP at the University of York. Since its launch in December 2018, and now with a core team of fourteen researchers, CPD providers and resource creators, NCELP has established its reach across England, delivering courses and materials to eighteen specialist teachers in nine Lead Schools from Gateshead to Exeter. Each of these nine schools cascades the NCELP training and resources to four Hub Schools (to date, a total of 308 teachers across all Lead and Hub Schools) and other local schools (a further 456 teachers) to drive a nationwide improvement in the understanding and delivery of research-informed pedagogy and curriculum design.

Over 225 NCELP professional development events have taken place to date, reaching more than 760 teachers, and a further 5,600 language professionals and learners have attended presentations by NCELP at external events. Of the teachers at NCELP’s 45 Lead and Hub schools, 92% say they are confident or very confident that they can deliver the recommendations of the MFL Pedagogy Review, and 92% also report that NCELP resources are helpful in supporting their teaching, one participant noting that: “We feel re-energised. I have had the most successful Year 7 lessons ever (in my whole career)” [9].

Chair of the MFL Pedagogy Review (also CEO of a Multi-Academy Trust, former President of the Association of School and College Leaders; Chair of the Project Board of Oak National Academy and Interim Chair of the Ofqual Board) writes that NCELP “has had a huge direct impact… Schools and teachers all over the country are using the freely available schemes of work and materials being produced by NCELP, in three languages (French, Spanish, and German). Via resources and an intensive programme of CPD (full resources for which are also freely available), the key changes in practice include: a sequenced progression of grammar including carefully revisited opportunities to practise; teaching grammar in the input via reading and listening; prioritising high frequency words and regular re-visiting of those words; a de-emphasis on teaching unanalysed chunks; an emphasis on raising awareness about L1-L2 differences; the use of information-gap activities that elicit the use of a grammar feature, making it essential and meaningful” [3].

NCELP has also allowed Marsden to build upon the impact of her previous open science initiatives by providing a searchable Resource Portal that links to both IRIS and OASIS. This database ( resources.ncelp.org) now holds 329 classroom-ready lessons, 27 tests and 21 standalone professional development sessions developed by NCELP in line with the MFL Pedagogy Review. Many link to the OASIS summaries that informed their development which, in turn, often link to the original research materials on IRIS. To date, Resource Portal materials have been downloaded more than 100,000 times, approximately 75% of which has been by language teachers, instructors, and teacher educators.

Impact on national assessment and school inspection

As part of its work NCELP has demonstrated that many of the words tested in the national GCSEs are very low frequency, that the exams overly reward rote-learning, and that teaching is heavily influenced by these factors. Partly drawing on these findings, the DfE established a panel to review the subject content of the GCSE examinations (affecting ~271,000 pupils each year), and Marsden has been one of its key advisers. The Panel Chair writes: “Emma has played a pivotal role” [3] and the Deputy Director of the Curriculum Policy Division says: “ Her input into the work of the group has been substantial, not least in recommending significant reform of how vocabulary should be treated in the new GCSE … As with the pedagogy review, Prof Marsden’s forensic knowledge of the research provided the panel with a better understanding about the nature of word frequency, word list compilation, and testing vocabulary, grammar knowledge, spoken production, and sound-writing (literacy) knowledge” [8] .

Professor Marsden has also contributed significantly to the creation by Ofsted of a new subject-specific inspection framework for MFL, providing evidence about characteristics of effective pedagogy, the types of evidence that Ofsted might gather, and the research that can help teachers to conceptualise and observe MFL learning and knowledge. By helping Ofsted to align its inspection process with the recommendations of the MFL Pedagogy Review, Professor Marsden has therefore had a direct impact on the assessment and working practices of all MFL teachers in England [3][8].

Impact on national education during the COVID-19 pandemic

As director of NCELP, Professor Marsden co-facilitated a collaboration with the Oak National Academy, which was established to deliver online teaching during the COVID-19 lockdown. The Chair of Oak’s Project Board says: “NCELP pedagogy principles and resources were used by NCELP network teachers to film over 240 fully resourced and ready-to-use lessons, across French, Spanish, and German, for years 7 and 8. The filming of 200 more is ongoing (for the remainder of year 8 and for year 9)” [3]. To date, the NCELP network has delivered 312 video lessons, which have been accessed and completed 30,699 times. The collaboration has therefore seen the impact of Professor Marsden’s work find a much larger national and international teacher and pupil audience.

Impact on provision of evidence-based teaching resources via BBC Bitesize

In September 2019, Professor Marsden conducted a workshop for the BBC Bitesize development team. An Editor of BBC Bitesize writes: “[Emma] described some of the research underpinning ... the MFL Pedagogy Review, illustrated how the recommendations in the review can be operationalised in learning materials, and she exemplified some of the schemes of work (curricula) that NCELP were developing that identify grammar to be taught and suggest sequences of learning.” Drawing on this “clear, well-evidenced framework for meaningful and accessible language learning”, the BBC team invested over GBP100,000 to produce new programmes of KS3 study in French, German and Spanish, a practice game for KS3 learners in each language, and BBC Teach videos exploring the teaching methods used in NCELP hub schools [10]. Equivalent programmes for KS2 learners in French, Spanish and Mandarin were launched in October 2020. (BBC Bitesize reached 73% of secondary school pupils in the year ending March 2020, with a considerable rise during the COVID lockdown, reaching an average of just under 4,000,000 unique users per week in Summer 2020).