Impact case study database

The EIA programme achieved large-scale systemic engagement from teachers, head Teachers and education officers across all eight divisions and 64 districts of Bangladesh [C1]. Between 1st August 2013 to March 2018 EIA dramatically increased its reach, through a shift away from direct contact with international and national English Language Teaching (ELT) experts, towards a distinctive model where ELT practice expertise was embedded in Mediated Authentic Videos accessed on mobile phones, delivered through partnership with Government Teacher Professional Development systems and staff, and supported primarily by teachers’ peers (including Teacher Facilitators). From August 2013 to March 2018, over 53,000 teachers participated in EIA: 10,563 secondary teachers and 42,708 primary teachers [C2]. Collectively, these teachers taught approximately 7.4M school children (3.1M primary and 4.3M secondary children) [C2]. More specifically, EIA has achieved impacts on practitioners, learning and public policy in Bangladesh by: i) changing the classroom practice of teachers; ii) improving the English Language competence of students; iii) shaping the content of English Language Teaching Curriculum Material and iv) influencing approaches to teacher development in Bangladesh and v) influencing approaches to teacher development outside of Bangladesh, demonstrating global impact.

1. Changing the classroom practice of teachers

There is evidence that EIA has contributed to changes in teachers’ confidence to speak English and use English in the classroom. For example, at a divisional workshop in Rajshahi in 2017, teachers underlined that EIA had triggered inner changes in them, which they saw as a pre-condition for changing students: *“I changed myself, now I am a change maker. Without a change in students, a teacher is 0” (*Assistant Teacher English/secondary and Teacher Facilitator, Rajshahi division, 23.8.2017); “EIA has developed me personally and professionally” (participant, Rajshahi Divisional Workshop, 23.8.2017). The participants also reported that they felt more at ease speaking English among themselves and with their students: “Now I am able to speak in English in any forum” (participant, Rajshahi Divisional Workshop, 23.8.2017) [C1]. EIA has also brought about changes in the classroom practices of teachers. In 2014-2015, 1,600 teachers provided self-report data, analysis of which indicated that over 71% said they used the teacher development videos every week and over 80% said they used EIA activities in the classroom every week. Government Education Officers observed 4,727 classrooms during this time and found that that 61% of students were seen using English in more than 50% of their talk-time, with 86% of lessons including pair and group work [C3]. Systematic timed observations of 346 lessons by the 2014-2015 cohort of teachers, showed similar changes in practice. The results show that both primary and secondary teachers are using a wide range of activities in the classroom and involving students in these activities. In this study, primary teachers were found to be organising 33% of the time, presenting 29% of the time, asking questions 18% of the time, and giving feedback 19% of the time. Secondary teachers were found to be organising 27% of the time, presenting 38% of the time, asking questions 21% of the time, and giving feedback 15% of the time. This is a change from the baseline study where teachers were observed to be primarily reading from the textbook and rarely involving students in activities [C4]. For 2015-2016 teachers, a quasi-experimental study (including 163 systematic timed lesson observations) found statistically significant experimental effects after only six months, just half-way through the treatment. Primary and secondary teachers spent less time ‘presenting’ to students (effect size r=0.2); student-student talk (pair and group) increased fourfold in primary lessons, from 4%pts to 16%pts (effect size r=0.3); whilst in secondary lessons, teachers and students both used more spoken English (effect size 0.2) [C5]. At a Divisional workshop in Rajshahi in August 2017, participants confirmed sustained use of the EIA interactive teaching activities and learners’ increased participation in the classes: “Before EIA, I never used body language, acting and dancing in the classroom. I thought this was only for male teachers […] that it was not for me. Now I use songs, rhymes, miming, acting, dancing. I feel very efficient in my work. My student teams are standing out in crowd at education and digital fairs. This is thanks to EIA. I changed myself and my sub-district” (teacher participating in Rajshahi Divisional Workshop, 23.8.2017) [C1].

ii) Improving the English language Competence of students

There is large-scale evidence for an improvement in learning outcomes for students who are taught by teachers who have engaged with EIA. Over 1,000 students (579 primary, 480 secondary) from the 2014 cohort took part in an end-line study of English Language Competence (ELC), carried out by independent assessors. There were statistically significant differences showing substantial improvements over baseline for both primary and secondary students. For primary students, the pass-rate rose 34.4%pts from 35.2% to 69.6%, whilst for secondary students, the pass-rate rose 8.3%pts, from 74.5% to 82.8% [C6]. For 2015-2016 students, a quasi-experimental study (1,802 students in pre- and post-test) was conducted after only six months, just half-way through the treatment. In treatment schools, the differences between pre- and post-test ELC were larger than in control schools (rising by 0.3 Graded Examination in Spoken English (GESE) grades in treatment, compared to 0.2 grades in control), but at this early stage the differences were not yet statistically significant. However, there were small-to-medium sized experimental effects for rural secondary students (r=0.2) and urban primary students (r=0.2) [C5]. Discussions with various stakeholders also indicate that teachers acknowledge that being in an EIA school has increased students’ confidence and proficiency in using English, in the classroom and among themselves: “I visited 2-3 classes in EIA schools in Manikgonj district. EIA is very fine; secondary students were spontaneously talking to us; students spoke English very fluently. In Bangladesh, children’s attitude towards English has changed. They have a courage for English and think that they can answer all questions. EIA has contributed to this. Students’ English results have also improved in the recent years. EIA has contributed to this” (Director - Secondary at Department of Schools and Higher Education (DSHE), 21.8.2017) [C1].

iii) Shaping the content of English language teaching curriculum materials in Bangladesh

The EIA Programme team has influenced both primary and secondary curriculum in Bangladesh through its contributions to the work of the National Curriculum and Textbook Board (NCTB) in Bangladesh. BMB Mott MacDonald (lead partner on EIA) conducted two studies that produced recommendations, and the textbooks for primary schools were revised to implement these. EIA then helped review the new textbooks for primary teachers, grades one to five with regards to English language teaching. EIA worked with the Centre for British Teachers, who worked with NCTB, on the textbooks. The textbook revisions included the referencing of EIA classroom audios, where relevant to particular lessons. The new textbooks were distributed to teachers in 2013 (grades one to three) and 2015 (grades four to five). All teachers received EIA classroom audio recordings, as part of the NCTB’s distribution of the revised textbooks. The 2015 Terms of Reference for The NCTB Textbook Revision acknowledge EIA inputs into the English Textbooks for grades four to five [C1, C7]. ‘Teacher Editions’ are versions of the national textbook series that include instructional guides for teachers, on how to use the textbook content to support Communicative Language Teaching. In 2014, EIA technical experts contributed to writing all five Teacher Editions for English language primary teachers (grades one to five) [C8]. Content from EIA print materials (i.e. Activity Guides) were integrated into the Teacher Editions, particularly where they gave guidance regarding how to integrate digital audio resources in classroom teaching of specific lesson content. The Teacher Editions were published in August 2016. By 2017 they had been provided in print to every primary school (two per school) and had been distributed to all local Education Offices. They were also made available in soft copy on the NCTB website [C1]. Commissioned by the NCTB, EIA produced 36 audio recordings of ‘English for Today’ listening texts that accompany the NCTB textbooks for all secondary grades. Together with EIA’s own audios, these audios were uploaded on the NCTB website [C1]. The National Curriculum Coordinator Committee approved the audios and in 2014 the Directorate of Secondary and Higher Education (DSHE) issued a circular asking all District Education Officers and Upazila Secondary Education Officers to ensure every head teacher/principal of a secondary school downloaded and used EIA audio materials [C9].

iv) Influencing approaches to teacher development programmes in Bangladesh

EIA materials and models have been integrated into in-service teacher development in Bangladesh through the Subject-Based Training (SBT) in English and the Customised Training Programme-English (CTP-E). SBT was a six-day face-to-face training for in-service teachers managed by the Department of Primary Education (DPE). The SBT Manual integrated 23 EIA A/V materials, and interactive teaching/learning activities. EIA supported the production of a Guide for trainers (including a DVD with EIA digital contents), and the training of 112 ‘Master Trainers’. Between 2014 and 2017, 115,000 Bangladeshi teachers have undergone SBT-English, giving EIA inputs a wide reach. SBT has since stopped and has been replaced by the CTP-E. The government has engaged several EIA staff as consultants in the design of the English element of this programme and its supporting materials (e.g. Training Manual) and the programme has integrated some of EIAs core components. For example, use of ICT and online/offline A/V materials is a key focus with all the trainers and teachers getting access to EIA digital A/V materials (17 EIA videos have been integrated with different sessions on different days) [C1]. Furthermore, most of the Government’s Master Trainers (MTs) for the CTP-E are people who have been oriented to EIA, including many of EIA’s teacher facilitators/teachers: “ EIA human resource is our resource” (DPE Deputy Director, 24.8.2017). Among the 31 pilot Master Trainers, 22 have been oriented/exposed to EIA [C1].

v) Influencing teacher development programmes outside of Bangladesh

Key stakeholders have testified that EIA has had an impact beyond Bangladesh: “EIA added to the global knowledge and debate around teacher professional development, by proposing a specific model based on 5 elements and by putting teachers at the centre of the classroom and evidencing what works in the classroom and to build teachers’ capacities” (DFID Education Adviser, 24.8.2017) [C1]. One example of the global impact of EIA is the Teacher Development Programme (TDP) in Nigeria. TDP, funded by DFID (2013-2019), was implemented in partnership with the Nigerian Federal Government and 5 Northern State Governments. It aimed to improve the quality of teaching for 62,000 teachers in primary and junior secondary schools and in Colleges of Education. Several EIA staff members were commissioned to work with TDP on the construction of materials such as a Facilitators Guide and a Head Teachers Guide. In addition, TDP has modelled its approach on EIA, in that both EIA and TDP are school-based teacher professional development programmes, with a blend of self-study and face-to-face training and support [C1]. In 2017 the scale and quality of impacts achieved by the EIA project were recognised through two awards: The Times Higher Education Supplement (THES) award for International Impact (November 2017) and the Pie-oneer Award for Progressive Education Delivery in H.E (September 2017). The THES judges said that EIA was a: “clear winner for its global impact on English communication abilities in difficult-to-access regions”. They commended staff at the OU for their: “innovative and pragmatic solution as well as their collaborative approach, working with intergovernmental agencies and in-country partners” and stated that “The OU’s development of teaching materials specifically designed for use on low-cost mobile phones has had a transformative impact on the regions in which they have been deployed”.

Impact has been developed by working with a range of beneficiaries including: 1) Primary school practitioners with responsibility for reading including: English Subject Leaders, teachers, head-teachers, school librarians, teaching assistants and external reading consultants or literacy specialists 2) Initial Teacher Education stakeholders including teacher educators and student teachers 3) The Department for Education and related government bodies with a role in developing reading skills in primary education. Two key pathways to impact the research team have developed are an interactive RfP website and Teachers’ Reading Groups (TRGs). Advised by a reference group of current teachers the RfP website hosts accessible resources which engage teachers with the OU research-informed approach to effective RfP. Users can upload ‘Examples of Practice’ explicitly linking change in their RfP practice to the research. TRGs are local CPD-focused groups run by volunteer leaders. The RfP research team support TRG leaders by offering training, mentoring and resources. Texts for the TRGs have been donated by the OU, UKLA and commercial publishers. TRG Members are supported to plan and implement changes to their practice as a result of their engagement with the research findings. Overall, these and other activities have had an impact on:

• The understanding and learning and practice of primary schools, primary school practitioners (and their pupils), Initial Teacher Education Providers and student teachers by transforming whole school RfP practice; shaping RfP related Initial Teacher Education provision; and influencing pupils reading practices and attainment.

• public policy of Department for Education and related bodies who have a responsibility for developing reading skills in primary education by influencing national RfP initiatives.

Transformed whole-school RfP practice: Between 2018 and 2020, 69 schools have paid for sustained support from the RfP team who have worked with the English Lead, Head Teacher, and change management team to develop their RfP practices. In 2018-2019 in Macclesfield, the RfP team worked with The Aspirer Teaching Alliance to support development across five schools. In the West Midlands they worked with the Birmingham Diocesan Multi-Academy Trust (BDMAT) to support development across eight schools. In 2019-20 the RfP team worked with eight Nottingham and eight Birmingham schools, in 2020-2021 with 27 Sheffield and Rotherham schools, linked to St Wilfred’s English Hub, and a further 13 Birmingham schools. Development days were held to enable staff from participating schools, including teachers, teaching assistants, senior leaders, and head teachers, to engage with the research (approx.1,500 practitioners in total). Consequently, RfP Lead Teachers have developed ‘Action Plans’ for their school and individual teachers have planned change to their practice. End of year one evaluation data from across the Aspirer and BDMAT Trusts revealed that 96% indicated that the RfP research had considerably influenced their own and their staff’s RfP pedagogy [C1]. At the end of year school conference, teachers from schools in the Aspirer Trust shared the practices they employed to engage the whole staff team with the RfP research such as including examples from the RfP website on the agenda for every staff meeting [C1]. Examples of whole-school practices adopted by one of these schools, Puss Bank School and Nursery, include: teachers taking time to share their reading preferences and habits with their classes, teachers creating a ‘book door’ leading into their classroom and introducing birthday book gifting [C1]. The Head teacher of Peover Superior Endowed Primary School describes how engaging in RfP research started a ripple-effect within the School and Trust leading to outcomes such as an NQT winning a RfP Teacher of the Year award and the head-teacher offering RfP in-service training to eight other schools within the wider Trust [C2]. The RfP team is currently working with 34 government funded English Hubs and their 170 English Leaders (each Hub works with 120 schools in high challenge areas) by contributing to the Hub training programme. In 2020, 141 Leaders attended five RfP training sessions led by Cremin in London, Wilmslow, Wakefield, and Bristol. After the training, 85% of attendees reported that they intended to implement what they had learnt, with examples including: ‘ staff training on not killing stories’ and ‘ getting to know my children really well’. The person responsible for leading training across the Hubs testifies that the RfP research has “ enriched teachers’ understanding of why and how they should put reading for pleasure at the heart of their schools” [C3]. The Strategic Lead for the English Hub covering North East London and Essex testifies that they run a TRG group with 18 school representatives and that evaluation responses are very positive with all participants reporting that “ it had already transformed (or was transforming) the teaching of Reading for Pleasure in their school” [C3]. Further evidence of impact on whole school practice can be found in testimonials from school leaders. For example, at St Mathews C.E Teaching and Research School, RfP is embedded through activities such as introducing ‘reading aloud’ at the end of every day, incorporating Reading for Pleasure weeks on a two-week cycle and training teachers on understanding the pedagogy behind RfP [C2]. Sneiton St Stephens Church of England Primary School have responded to the RfP research by making sure everybody (including the caretaker) was reading more current children’s literature; doing a ‘book swap’ for children on a Monday morning in the hall before school and holding reading assemblies which parents are invited to [C2]. At the University of Cambridge Primary School an RfP culture has been developed through setting up a TRG group for teachers and learning coaches, establishing a separate ‘parents RfP’ group with 20 parents, running workshops for parents and developing a RfP professional development course for 50-60 practitioners from 20 schools [C2]. The RfP research has informed whole-school approaches in a further 67 schools, as evidenced by the case studies of whole-school change that have been uploaded to the RfP website by teachers [https://tinyurl.com/ya48rfkb\]. For example, a teacher at Moorlands Primary Academy describes how they recruited 35 staff to become ‘reading buddies’ to less experienced readers in the school and how this has led to “significantly more book talk around the school”.

Shaped RfP related Initial Teacher Education provision: Since 2016, the RfP team have partnered with 30 Initial Teacher Education providers including Edge Hill University, Cardiff University, University of Glasgow, Sheffield Hallam, Roehampton, University of the West of England, and St Mary’s University College Belfast. The RfP team supports them in using RfP findings within their teaching materials, activities, or assignments. In addition, 14 universities have established their own TRG groups for ITE students, or in combination with local teachers. At the University of Greenwich, RfP is embedded in the BA QTS course (500 students since 2017); School Direct (80 since 2017); the PGCE (360 since 2017) and CPD in 11 schools has been run. In Year 3, a specialist course is run for around 20 students per year who want to develop their RfP practice, where they either examine their understanding of children’s books or their own identity as a reader [C4]. The University of Roehampton directs students to the RfP website to inform their developing practice and Year 2 BA QTS students (320 per year) are tasked to undertake a RfP case study during their placement. A senior English lecturer testifies how engagement with RfP has also influenced her own practice and the RfP ethos of 40 other staff at Roehampton, with examples including staff having posters on their doors to identify what they are currently reading [C4]. At the University of the West of England, the RfP research is shared with first year students and PGCE students (660 students since 2018). First year students focus on RfP as part of their placement and, drawing on RfP research, write an assignment that reflects on the effective teaching of reading [C4]. A 2019 survey of 20 partner Initial Teacher Education providers working with student teachers from a range of courses revealed that RfP research findings had been shared with 4,300 students and developed with 1,238 students. Regarding the impact of the RfP research on students, respondents felt there had been an impact: on students’ knowledge of children’s literature and other texts (82% agreeing or strongly agreeing); on RfP pedagogy in terms of social reading environments (94% agreeing or strongly agreeing), Reading Teachers (94% agreeing or strongly agreeing), reading aloud (88% agreeing or strongly agreeing) and independent reading (82% agreeing or strongly agreeing); and on reading communities (71% agreeing or strongly agreeing) [C5].

Transformed primary school practitioners understanding and implementation of RfP:

To date around 450 detailed accounts of RfP practice that has been influenced by OU RfP research have been uploaded to the RfP website by teachers, librarians, teacher educators and student teachers from over 25 schools in England (e.g. Bexley, Birmingham, Cambridge, Coventry, Dagenham, Grimsby, London, Northampton, Macclesfield, Manchester, Margate, Rochdale, Weston-Supermare & York) Wales, Republic of Ireland, Dubai and Pakistan [https://tinyurl.com/ya48rfkb\]. Since 2017, the RfP team have supported 200 ‘Teachers and Reading Groups’ (TRGs) across the UK and 4 international groups, involving approximately 3,600 members, who collectively are teaching approximately 900,000 pupils. A 2019 survey of 63 TRG leaders and 357 TRG members revealed that engagement in the RfP research had influenced TRG members knowledge and understanding of: children’s literature and other texts (91%), children’s reading practices (85%); RfP pedagogy-social environments (81%); RfP pedagogy-reading aloud (74%); RfP pedagogy- independent reading (75%); RfP pedagogy-informal book talk (85%); Reading teachers (87%) and reading communities (89%). TRG leaders indicate that this had contributed to an increase in confidence and a sense of being empowered to make changes in practice. TRG members also report impacts on their own knowledge and practice. For example: “ My attitude towards it [RfP pedagogy] and awareness of the importance of it has changed my practice” (literacy lead, KS1) and “ it [the TRG] has kept me focused on my aims and ensured I kept up with what I have wanted to achieve” (literacy lead, KS2) [C6]. A TRG leader in Rochdale testifies that as a result of attending the training and becoming a TRG leader they i) invited the Mayor and head of the Royal British Legion into school to read stories to the school’s most vulnerable children; ii) started the Rochdale Children’s Literature festival with 8 TRG members which involved pupils as festival ambassadors; and iii) buddied up less experienced readers with members of staff [C2].

Influenced pupils’ reading practices and attainment: Practitioner uploads of practice case studies to the RfP website, school evaluations and practitioner testimonials highlight pupils are more enthusiastic, willing, and able to voice their views about books and identify themselves as readers. Their perceptions of why we read have changed from predominantly functional to pleasurable. For example, responses to the 2019 TRG member survey included: “ some Year 2 boys formed their own book club in the playground where they were sharing books by the same author” [C6]. St. Matthew's C.E. Teaching and Research School reports that: “ we’ve found that that engagement with RfP […] has enabled our children to really take on board any type of reading that they do across the curriculum” [C2]. A teacher from University Cambridge Primary testifies: “ the RfP work was instrumental in helping us create that culture which has transformed children’s lives”. They described how a Year 6 boy who’d hated reading in Year 2 asked if he could “ run an intervention group with Year 1 children who find it difficult to read” [C2]. Elmhurst Primary School reports that in 2019 the number of pupils reaching the ‘Higher standard’ increased from 30% to 43% [C3]. Whole-school data from schools in the BDMA and Aspire Trusts shows significant shifts in readers’ attitudes through baseline/end-of-year surveys. For example, a shift from 44% to 97% KS1 students liking reading at Victoria Road Primary School; a shift from 3% to 60% KS2 students liking reading at Coleshill Church of England Primary School) and 50% to 75% KS1 and KS2 students liking reading at Nethersole Church of England Academy [C1].

Influenced national RfP initiatives: The scale and quality of impact of the RfP research has been recognised by the Department for Education (DfE) who during the pandemic invited Cremin to advise on the Reading for Pleasure component of the BBC BiteSize resources which were created to support home-schooling and to work with the Oak National Academy to produce a daily reading curriculum for 5 to16 year olds for 2020-2021. In addition, at the request of the DfE, Cremin has developed a tool for Schools to audit their RfP practice. Linked to this work Cremin has been asked to Chair the DfE RfP subcommittee and has been asked by the Minister of State for School Standards to serve as one of three ‘reading experts’ on the English Hubs Council programme [C7]. These appointments have enabled Cremin to make a significant contribution to the initiation and launch of the first National Reading Together Day (16/07/2020). Co-ordinated by the Reading Agency, the aim of the event was to encourage families and young people to read together by participating in a shared schedule of events [C7]. Cremin also initiated an OU RfP partnership with the Reading Agency to launch a Teachers Reading Challenge (Aug to Oct 2020). [https://teachersreadingchallenge.org.uk/\]. The 2,500 teachers who signed up to the Challenge were supported to use the OU RfP pedagogy to source book recommendations, take part in discussions and produce and share 2,000 book reviews. Commenting on the success of all the initiatives outlined here, the DfE English Hubs Policy Team Leader concludes that the RfP research has played a “ a highly important and influential role on our policy-making, which has resulted in excellent outputs such as the Reading Together Day and various helpful documents, products and tools for schools across the country” [C7].

The beneficiaries of the research into Tricky Topics have been schools, initial teacher education providers, universities and the teachers and lecturers working with those organisations. The work on Tricky Topics and its impact on technology-enhanced learning was awarded a WISE Technology Innovation Award 2017 (sponsored by Goldman Sachs and given by HRH Princess Anne). The Tricky Topic research has had an impact on public services by informing organisational approaches to the professional development of educators in Africa and on practitioners by influencing the knowledge understanding and practice of educators in the UK, Europe, Thailand, Myanmar and Africa.

Beneficiaries of the research into Evidence Cafés have been those responsible for the training and development of the overall police force (e.g. College of Policing); individual police forces (e.g. constabularies); those working within the police force such as front-line officers and performance analysts; and NGOs in Africa. The evidence café research has had on impact on professional services by shaping approaches to professional development within UK policing and an impact on public services by influencing approaches to organisational learning in NGOS in Africa.

Informed organisational approaches to the professional development of educators:

Although the Tricky Topics Process was originally developed in the context of STEM education in schools, it has since been adapted (i.e. removed the technological tools) and implemented in a range of organisations that have a responsibility for the professional development of educators who work in a wider range of contexts. Within higher education, the research team have worked with the Partnership for African Social Governance and Research (PASGR), an independent non-profit organisation that (amongst other things) facilitates the development of collaborative higher education programmes. The research team initially engaged with PASGR through the delivery of two Tricky Topics workshops for trained teaching staff in 2017 and 2018. One outcome of this engagement is that Tricky Topics was embedded as a core element for a professional development programme across Africa called Pedagogic Leadership in Africa (PedaL). Led by PASGR, PedaL is seeking to produce a step-change in pedagogical training, with Tricky Topics as one key element. In August 2019, the Director of the Higher Education Programme at PASGR testified that six PedaL training courses had been undertaken in Kenya, Uganda, and Nigeria (all incorporating Tricky Topics) with a total of 605 university teaching staff. Course evaluation data reveals that participants found Tricky Topics highly relevant to their practice. For one course held in February 2019, 40% of participants indicated an intention to apply Tricky Topics in their practice, and 40% said they would take action to enable others to apply it. For a course held in March 2019, 35% of participants indicated an intention to apply Tricky Topics in their practice, and 40% said they would take action to enable others to apply it. The Director concluded that: “ incorporation of tricky topics has strengthened our PedAL professional development course by enriching its content and broadening its approaches” [C1].

Influenced the knowledge, understanding and practice of educators: The Tricky Topic Process and Tools have helped educators to develop their deep understanding of students learning needs and supported them to assess if their students have developed an in-depth of understanding of a topic. A teacher testifies to the impact of the Juxtalearn project on himself and colleagues whilst working as an Advanced Skills Teacher at an Oxfordshire school: “ The Tricky Topics Process has been invaluable in unpacking the what and why of pedagogy, and how we subsequently sequence the learning process […]. The effect of using Tricky Topics on teaching practice has been seismic in terms of the shift in developing colleagues’ real understanding of what, how, why students stumble and hence we as teachers need to unpack and teach in a different ways with some topics” [C2]. A mathematics and science teacher in a school in Portugal was also introduced to Tricky Topics in 2016, through the Juxtalearn project. As a result, they have incorporated it into their teaching and used it with 200 9th grade (14 years old) students over the last 4 years. Using the Tricky Topics diagnostic tools. The teacher testifies that they are able to show that following the Tricky Topic Process, contributes to an improvement in student understanding but also that the students find the process itself engaging and motivating: “ For all of the 8 steps, the students were fully engaged, there was not a student who quit at any time. They were working on the Tricky Topic in the class and in their own time. They were also asking me about when we will do this again [...]. They were engaged, it was perfect” [C2]. As part of the TIDE project, Tricky Topics has also been introduced to university lecturers in Myanmar. Funded by the UK government, TIDE brings together universities in the UK and Myanmar to improve the quality of distance learning to result in more employable graduates. The OU is the lead partner of TIDE, and one of the many initiatives that they have co-ordinated is the running of two Tricky Topics workshops in October 2019 with 27 lecturers from the University of Yangon, Yangon University of Distance Education and Yadanabon University. Follow-up interviews indicate that the workshops increased the lecturers understanding about how to raise student’s awareness of difficult concepts and engage them in more student-centred critical thinking: “ Before the seminar, teachers never had chance to discuss with students about complicated theories and concepts. Now students can be drawn to get to understand concepts and let them think”. Some lecturers were also adapting their practices in order to engage students, by for example, using feedback tools and processes suggested in the workshop: “ I always try to use TIDE training and concepts into my classroom. Although student numbers in class is high, now [I am] using student-centred approach and feedback from them was taken as well” [C3]. In Thailand, a tricky topic workshop was run with 50 lecturers within the Faculty of Nursing at Payap University in February 2019. Follow up interviews indicates that lecturers were using Tricky Topics concepts to clarify why they were having problems in their teaching and trigger discussions with colleagues. They were also incorporating the Tricky Topics process into their teaching in order to find out the level of student understanding and adapt lectures accordingly. For example, by introducing activities that would enable students to define problems and dig down to the solution. This was having a transformative effect because: “ In Thailand we have to lecture a lot, about one hour, at least one hour for one lecture. But the Tricky Topics is just like a flipped classroom” [C3]. In August 2020, working in partnership with the African Council for Distance Education, the team contributed to a COVID-19 response project funded by GCRF which delivered a range of online workshops designed to support educators in schools, tertiary education and Teacher Education in Africa to transition to online teaching. One of those workshops focused on instructional and learning design and included an introduction to the Tricky Topics Process. 319 educators from countries that included Kenya, Nigeria, Zambia, and Mozambique engaged in the online workshops. Evaluation evidence indicates that a substantial majority of respondents stated that the course had changed their thinking about student-centred learning ‘a lot’ (87.6 (n=92), with a further 12.4% (n=13) stating it had changed their thinking ‘a moderate amount’. 77% intend to use Tricky Topics in their own practice (n=105) and 85% intended to develop their understanding further (n=98) [C4]. Finally, the team has created a free online course called: ‘Connecting tricky topics to learning design’ that is endorsed by the CPD Standards Office. Launched in July 2018, the course has had 677 enrolled and active learners, with 69 badges awarded for completion of a non-compulsory assessment. Survey results from 87 participants regarding impact on knowledge and understanding indicate that 72% (n=86) reported a significant increase in their knowledge and understanding of the Tricky Topics process and that 98% intended to share what they had learnt with their employer (n=80) [C5].

Shaped approaches to professional development and organisational learning within UK policing: Current changes in the police service in England and Wales are being driven by movements to adopt an evidence-based practice (EBP) approach to policing. At the heart of EBP is the sharing of knowledge. This had led those with a responsibility for professional development and organisation learning within policing to realise the value of the Evidence Café method and to commission Anne Adams and other OU colleagues to deliver Evidence Cafés to policing practitioners. Between March 2016 and June 2018, 21 Evidence Cafés were run with 449 participants from police forces across England and Wales including Gwent, Lancashire, Humberside and Somerset [C6]. In 2017 two Evidence Cafés were run for 45 practitioners within Hertfordshire Constabulary: one with a focus on demand management and one with a focus on evidence typologies. For the demand management evidence café, The Performance Manager for Hertfordshire Constabulary indicates that: “ The evidence shared within the Evidence Café pointed us towards the conclusion that a lot of the demand within the system was a ‘failure’ demand. This was a powerful challenge to some of our organisational thinking and forced us to confront a performance failure. A major consequence of this is that after the event, we commissioned a detailed piece of demand analysis. This resulted in a significant resource uplift to our control room of several million pounds.” For the Evidence Typology evidence café, they testify that: “ a typology was developed that allowed people to see two important points. Firstly, how different types of less quantitative data could be admitted into the canon of knowledge. Secondly, the journey from bits of data to more powerful organisational learning. These two insights were both powerful and challenging for the organisation as it had tended to value more positivist sources of knowledge. The Evidence typology introduced a way of recognising, valuing, and categorising a whole range of information. This contributed to a change in the way that some of our performance processes are managed. We asked the organisation to consider which of the metrics that were being monitored were of value and we ultimately cut about a third of the data recording within the organisation” [C7]. The Evidence Café work also led to the research team being commissioned by the Police Reform and Transformation Board (PRTB) to review national knowledge exchange and feed into the national reshaping of police transformation programmes. The resultant report made eleven strategic recommendations including sharing networks and training. The Lead for Knowledge, Research and Practice at the College of Policing (one of the stakeholders represented on the PRTB) testifies that the report provided an understanding that in order to change the culture of the police force towards engaging in independent knowledge sharing [it] needed to create a framework in order to help this happen and to help practitioners to share knowledge. One outcome of this was that the report was used to put forward a business case to PRTB to create formal mechanisms for knowledge exchange. One of these mechanisms was the creation of a network of 43 Innovation Brokers, one for each police force in the country. Their role is to help and encourage front line officers and staff to share their ideas and knowledge. Another was to begin work, with the Police ICT Company, to develop an online Knowledge Sharing Hub [C8].

Shaped approaches to organisational learning within NGOs in Africa: Between 2017 and 2018 the research team delivered four Evidence Cafés for 120 participants in the Migration for Inclusive African Growth’ (MIAG) project. This project involved partners from four countries in Africa- Kenya, Nigeria, Ghana, and Mozambique. The partners included two Non-Government Organisations with a focus on policy research and dialogue: The Network of Migration Research on Africa (NOMRA) based in Nigeria and the African Migration and Development Policy Centre (AMADPOC), based in Kenya. An employee of NOMRA testifies that the Evidence Café run in Nigeria “ reflected the voices of migrants in a way that we would not have been able to capture if we had used other methods. It also uncovered lots of assumptions and preconceptions about migrants and migration and forced people […] to ask where the evidence was to support their assumptions.” They were so impressed by the Evidence Café method that they volunteered to undergo training into order to become an Evidence Café Champion within NOMRA and their wider work [C9]. A senior manager at AMADPOC, testifies that the Evidence Café method was attractive because it allowed participants from all backgrounds to easily engage in policy discussions and that using Evidence Cafés as a method in their migration policy work has enabled them to use a different lens to try and understand the issue of migration in Kenya [C9].

SignAlong Indonesia is an approach to embedding inclusion in classrooms that includes a novel keyword signing system, a manual and a framework for training in the use of the system that focuses on teachers’ beliefs. The beneficiaries of the impact of the SignAlong Indonesia approach are: i) schools in Indonesia (inclusive, special and mainstream), the teachers within those schools and their pupils (both with and without communication needs) ii) lecturers and other university staff at the State University of Surabaya. The pathways to impact emanating from the research conducted by Sheehy and colleagues have received significant internal funding and support from the Department of Special Education in Indonesia, indicating a government commitment to promoting inclusive classrooms. This support began with the funding of the ‘Inclusive Indonesian Classroom’ project, between December 2013 and 2014, which resulted in the development of the Signalong Indonesia keyword signing system . In this project the initial vocabulary and training manual (December 2013) was largely based on British Sign Language (from Signalong UK), and then, through extensive piloting, a new vocabulary and original communication system was developed which incorporated signs from Bahasa Isyarat Indonesia (BISINDO, a widely used Deaf community language) and Sistem Isyarat Bahasa Indonesia (SIBI, created by the Indonesian Department of Education and Culture). Forty teachers from 33 of Indonesia’s 34 provinces were trained in the resulting ‘Signalong Indonesia’ and two ambassador schools were supported. The latter became centres of training and sites that sought to develop best practice. In addition, the Special Education Department of the Indonesian Ministry of Education supported nation-wide implementation of the materials by printing and disseminating multiple copies of the Signalong Indonesia manual to schools. Other examples of the commitment of the Indonesian government to the Signalong Indonesia include: i) The Indonesian Director of Special Education debating inclusive education and issues relating to Signalong Indonesia with Kieron Sheehy and a live audience in a one-hour current affairs programme called OutLook (SBO TV, 2014) and a public launch of the Signalong Indonesia manual by a government Minister, in January 2015, which was widely covered in the Indonesian media [C1]. Indonesia comprises 17,000 islands and is divided into 34 provinces (East Java is the second most populated province with around 39 million inhabitants). With the support of the Department of Special Education, the Signalong Indonesia research has had on impact on practitioners and the delivery of professional services by 1) transforming whole school approaches to keyword signing in schools in East Java and Bali 2) transforming the inclusive practices and beliefs of teachers across Indonesia and 3) influencing the inclusive practices of university personnel across 33 provinces in Indonesia; and social welfare by 4) improving the educational inclusion of school pupils in East Java.

1. Transformed school-wide approaches to inclusive education in East Java and Bali

Engaging with the Signalong Indonesia approach has transformed approaches to inclusion within a chain of four schools in Surabaya, East Java that caters for over 250 children. The chain is called “Galuh Handayani” and it consists of a Kindergarten, an Elementary school, a Junior High School, and a Senior High School. In 2015, 35 school staff undertook 3 days of training in how to use Signalong Indonesia, delivered by Sheehy and colleagues [C1]. Testimonies from the principals of each of these schools indicate that they were influenced to adopt a whole school approach to using Signalong Indonesia following engagement with the training. This approach means that all students (with or without communication difficulties) have been taught to use Signalong Indonesia and that Signalong Indonesia is implemented in every school activity including ceremonies, school assembly and scout groups. The head teachers talk of how this policy has improved the quality of the education within the schools [C1]. In 2018,183 students were using Signalong Indonesia either in the school or at home [C1] Galuh Handayani has also become a centre of training, seeking to develop best practice beyond its own classrooms. As part of this activity they have set up a website onto which they have uploaded a range of video clips documenting examples of their school-wide practice which includes using Signalong Indonesia in music and science lessons [https://signalongindonesia.org/\]. In 2019, Sheehy and colleagues were invited to work with the Bali School for the Deaf which is a national centre of excellence in Indonesia and children from across Indonesia attend. This collaboration began with the training of 40 teachers and has resulted in the school committing to use Signalong Indonesia in their religion and music education curriculum. The head teacher writes that this work is of national importance given the government directive for classrooms to be inclusive and is enabling them to “ create materials that reflect the diversity of Indonesian society and develop effective and innovative ways of teaching” [C2].

2.Transformed the inclusive classroom beliefs and practices of teachers across Indonesia

Between December 2013 and January 2020, The Signalong Indonesia team have delivered a range of training events for 898 schoolteachers (who collectively teach around 22,629 students) and 2,520 university teachers across 33 provinces [C1, C3]. Survey results indicate being trained to use Signalong Indonesia has positively changed teachers practice and that they hold positive beliefs about the value of keyword signing for all children. In a 2019 survey of 105, 104 teachers (mostly from East Java) indicated that were using Signalong Indonesia in their classrooms and 103 indicated that they had been trained to use Signalong Indonesia. 74% indicated that Signalong Indonesia had changed their teaching; This change was seen as positive by 85%. This positive impact appears to be where it has helped them to teach and communicate with children with special educational needs [C4]. In addition, using Signalong Indonesia in their classroom has transformed how teachers at Galuh Handayani communicate with children with special needs, teach difficult concepts to children with communication difficulties and promote active engagement of students in the classroom. For example, one teacher from the Senior High School testifies: “ Signalong Indonesia can also improve the quality of education services, because Signalong Indonesia helps me to communicate and socialise with students” [C1]. Another Galuh Handayani teacher shares: “ *If I communicate orally, I just use 50% of my power [...] but if I use Signalong together with oral I use my full power- 100%*” [C5].

3.Influenced the inclusive practices of the State University of Surabaya (UNESA)

At the State University of Surabaya, Signalong Indonesia has been implemented across a volunteer programme within the university called PSLD. PSLD is made up of volunteers from various departments in the university, who support disabled university students in their study. Between 2017 and 2019, 280 UNESA lecturers; 120 members of the D/deaf community and 430 volunteers have been trained to use Signalong Indonesia through the PSLD programme. The Vice President for Planning and Partnership at UNESA testifies that the embedding of Signalong Indonesia within PSLD activities has “ significantly supported UNESA’s drive to be an accessible university, able to educate a diverse student group”. In addition, staff from UNESAs special education department routinely train schoolteachers in Surabaya (and beyond) how to use Signalong Indonesia. Between 2017 and 2019, 1,720 teachers received training [C6].

4.Transformed the experience of inclusive classrooms for pupils in East Java

Teachers and parents have spoken of how the use of Signalong Indonesia has also transformed the learning experience of Indonesian pupils with special educational needs. Testimonies from teachers at Galah Handayani School describe how using Signalong Indonesia has increased the confidence and engagement of children with special educational needs: “ In addition to learning in the classroom Signalong Indonesia is also applied in the school assembly, scout and extracurricular activities. And that makes me happy, through Signalong Indonesia special needs students can be more confident when communicating with other students” [C1]; “ There is a real difference. When the teachers and students weren’t using Signalong the students were less excited. Some of them rested their heads on the table and daydreamed. But with Signalong there was no daydreaming, no sleeping because they were using their bodies” [C5]. Case studies presented on the Galah Handayani Signalong Indonesia website [https://signalongindonesia.org/\] reveal that the active and passive communication skills of four 7^th grade pupils with autism improved over a period of 12 months following training in how to use Signalong Indonesia. In addition, their science assessment scores improved by 40%. Parents of pupils at Galuh Handayani have spoken of how the use of Signalong Indonesia has increased the confidence and social interaction of their children: “ He has Asperger’s Syndrome, usually my son is very shy and sits at the back. But last week when I came to the class, he is much more confident. He was sitting at the front and could now understand well because of the signing” [C5]. Children have also spoken of how using Signalong Indonesia has helped them ‘ learn things for myself” and ‘ understand things better” [C5]. Interviews with nine teachers from two inclusive schools in two different cities in East Java indicate that using Signalong Indonesia has made learning engaging and fun [C7]: “ The key thing is that they learned together […] using Signalong […] has made the learning moment, learning process be more enjoyable […] it is easy for disabled and non-disabled students to learn together”; “ If they give happiness to the students, they will receive the lesson more easily, they will remember easily what the teacher gives to them”; “The students felt happier […]. So, it is very good even when it is used by able-bodied students”. In 2019, members of the Inclusive Indonesian Classroom designed a story book (called Big Book) that could be used by mainstream kindergarten teachers. In keeping with the Signalong Indonesia method, a single sign was created for each page of the story. Five kindergarten teachers were trained how to use the Big Book stories with Signalong Indonesia. An evaluation involving five kindergarten classes (one control class that used the Big Book but not the signs) and 83 children (with and without communication needs) revealed that use of Signalong Indonesia improved understanding of the story. Correct answers to five comprehension questions were 82%, 92% 75%, 77% and 79% in the Signalong Indonesia classes compared to 5% in the Big Book only class. Teachers also reported that using Signalong Indonesia was fun for the children, made learning easier and that the children were more engaged [C8]. In addition, in September 2020, copies of three Big Books with Signs (plus a training manual) were sent to 274 kindergarten teachers. In November 2020, a three-hour online training workshop was attended by 70 kindergarten teachers. In December 2020, an evaluation questionnaire was sent to all participating teachers. Results from 168 responses indicate that 98% were using SignAlong Indonesia with their pupils (3,322 pupils in total) and that 38% used the Big Books once per week, 18% used them two or three times per week and 10% used them four to five times per week. 98.5% indicated that using the Big Books had changed the way they teach and 98.4% said they planned to use the approach with other teachers. Furthermore, 89.1% indicated that use of the Big Books supported inclusion of disabled children in their classrooms; 90.5% reported that using the Big Books made learning more fun for the children and 97% reported that their use helped the children remember the stories [C9].

Two main beneficiaries of the research reported in this case study are: i) organisations responsible for supporting the professional development of language teachers and ii) language teaching practitioners. A significant organisation that the research has impacted on is the European Centre for Modern Languages (ECML). ECML is funded by the Council for Europe and representatives from the Ministries of Education of each of the member states sit on the Governing Board. The strategic mission of ECML is to act as a catalyst for reform in the teaching and learning of languages across 33 member states. In collaboration with ECML, the DOTS team has delivered a professional development programme underpinned by the research to language teachers working in schools, colleges, and higher education as well as teacher-educators and advisors. The research therefore has achieved an impact on practice in two key areas: it has shaped the professional development agenda of those responsible for online language teaching and transformed teachers’ understanding and practice of online language teaching.

Shaped the professional development agenda of those responsible for online language teaching.

Between 2013 and 2019, ECML funded the researchers (with other European and Canadian partners) to run national workshops for key local disseminators in ECML member countries with the aim of cascading the DOTS framework. This pathway to impact was called ‘ICT-REV’, and ICT-REV workshops implemented the DOTS framework for the professional development of language teachers who teach online that had been developed over the previous ten years of research. Typically, an ICT-REV workshop would be requested by a representative from a Ministry of Education (i.e. a member of the ECML board) who would then take responsibility for convening the workshop within their country. Between August 2013 and November 2019 43 professional development workshops were run in 21 member countries (e.g. Sweden, Czech Republic, Bulgaria and Croatia) with 740 participants; and a further 4 non-member countries (United Kingdom, Spain, Italy, Belgium) with 102 participants [C1]. An independent evaluation of the ICT-REV project in 2014 concluded that a clear benefit of the ICT-REV impact pathway “ is the cultivation of multipliers together with the establishment and maintenance of a dynamic digital collaborative resource for future on-going sharing and networking” [C2]. The Head of Administration at the ECML testifies that the ICT-REV workshop is one of the most popular workshops that ECML offers its members and that it is highly regarded by the governing board who: “ constantly send us extremely positive feedback about the workshops that have taken place in their different countries [...] we do get it first-hand from people who are working in Ministries of Education, and that’s very important for us”. He also testifies that one key consequence of the partnership that ECML has developed with the ICT-REV is that it has worked with them to build: “ an inventory of ICT tools, a set of Open Educational Resources, which is very popular both within the workshops and just for people looking for good quality tools to use in their teaching” [C3]. The Deputy Executive Director at ECML testifies that: “ very few projects have had such long-standing and continuous involvement with ECML which highlights the longer-term relevance of the current ICT-REV activity in the context of promoting quality language education in Europe”. They add that “ on the website we also monitor the 10 most-frequently accessed tools, and the ICT-REV inventory is currently number 1” [C3]. Between January 2019 and January 2020, the inventory was accessed 155,522 times. An ECML board member testifies that they drew the ICT-REV inventory to the attention of a European Commission working group that was developing a set of recommendations for a comprehensive approach to the teaching and learning of languages. The value of the inventory as a model for supporting continuous professional development for language teachers across Europe was endorsed by this working group and incorporated into one if its recommendations (no.19) [C4, C5].

Testimonials from Members of the ECML board who requested and convened ICT-REV workshops, comment on how participants were inspired not only to change their own practice but to change the practice of others [C4]. A member from the National Centre for Education in Latvia writes that ICT-REV materials are published on its website, that ICT-REV training has fed into new content that has been developed and that 50 German language teachers attended two ICT-REV workshops. This, combined with localised cascading of what was learnt in the ICT-REV workshops, means that: “ the training has been undertaken by the vast majority of German-language teachers in Latvia”. The Director of Education at the Swedish National Agency for Education testifies that demand for the ICT-REV workshops was high and that participants in two ICT-REV workshops held in Sweden were inspired to cascade what they had learnt through in-service training for their colleagues. They conclude: “ The workshops were really influential because they enabled teachers to build on their previous experience and take ownership of the pedagogy to apply to their particular situations. Many of the teachers said after the workshop that they were more confident in the use of ICT on teaching and would use it more in teaching”. A member from the Institut National Des Langues in Luxembourg testifies as to how staff who were usually resistant to change within their organization were persuaded, through engagement with the ICT-REV workshops to change their practice. They also comment on a “snowballing” effect where many of the tools and ideas were being formally and informally passed on to other teachers. A member of the Ministry of Education, Science and Sport in the Republic of Slovenia testifies as to how the two ICT-REV workshops which their department convened “ left a very positive impression” and inspired attendees to replicate the training framework in further professional development initiatives.

There is also evidence of impact beyond the ECML community. For example, a freelance consultant testifies as to how they have utilised resources from the DOTS project within their activities. Of their work as a consultant, they report how content from the DOTS project is included: “ in the reference materials for and embedded in the activities of a number of our CPD course activities for in-service language teachers and educators from a range of EU countries. We have been able to run 3 courses specifically involving references and activities related to DOTS materials, thus reaching some 40 EU in-service teachers” [C6]. The success of the European focused ICT-REV workshops has led to invitations to train practitioners in China (n=120) and Jamaica (n=40). In August 2017, the team led a blended workshop in China. 120 English teachers at all levels from different parts of China attended the workshop face-to-face; around 27,000 people followed the event through a live stream, and about 5,000 subsequently viewed recorded webcasts. The convenor of the workshop in China and Dean of the Institute of Online Education at the Beijing Foreign Studies University, testifies that the success of this workshop convinced them to convene a second workshop in 2019. They write: “ I would say that ICT-REV enhanced not only our beliefs, but also our practice in teacher development” [C7].

Transformed teachers’ understanding and practice of online language teaching.

Participants who have engaged in the ICT-REV workshops have testified to an impact on their practice. For example, a six-month follow-up questionnaire was sent to 138 participants from Lithuania, Austria, Sweden, Spain, Ireland, and Bulgaria who had attended ICT-REV training in 2015. Results revealed that: 63.62% indicated that they were using ICT tools and activities with more confidence and on a more frequent basis in the classroom; 72.72% indicated that they were using ICT tools, activities and ideas presented in the ICT-REV workshop, when teaching; 54.54% indicated that they were using the ideas and concepts promoted at the event to identify and apply appropriate ICT tools in their teaching and/or teacher training, and 63.63% indicated that they had acted as a multiplier to promote the use of ICT tools and activities to others in the field [C8]. A follow-up questionnaire sent to participants from Serbia, Armenia, and Latvia who had attended ICT-REV training in 2016-2017 revealed that 46% intended to integrate ideas learnt during the event into their work practice [C8]. A follow up survey of 77 participants who had attended ICT-REV workshops between 2013 and 2019 revealed that following the workshop: 73%[n=75] had used the resources introduced in the workshop; 68% [n=75] had gone on to develop their own materials using online tools and 82%[n=73] had worked with others to develop teaching materials and 82% [n=58] and delivered training related to the DOTS framework to others [C8].

Interviews conducted in 2017 with workshop participants evidence a change in their confidence, awareness, pedagogy and use of tools; as well as a motivation to disseminate to other colleagues [C8]. A participant of a workshop held in Ireland said: “ I think it [confidence] has increased [...] helped me actually to teach less and less and to create situations for learning more and more”. A participant of a workshop held in Croatia said: “ Yeah, it [my practice] has changed in the way that DOTS gave us some guidelines in how to choose ICT, and how to choose appropriate tools for the classroom. I think that was the most important thing. And I also organised three seminars about using DOTS in the classroom. [...] For a start I talked about the ECML and the different projects. And then we focused on DOTS projects. And so, I gave them the information, the website, showed them the tool. And then we […] tried out different tools and presented them to other groups.” A participant of a workshop in Bosnia-Herzegovina testifies how, following the workshop, they, and a colleague “ delivered a 90-minute-long lecture-cum-workshop for 200 EL teachers from the Sarajevo Canton in which we talked about the ICT-REV workshop we had in Sarajevo, showed specific ways in which ICT can be used in ELT […] and finally directed them to the ECML’s repository of ICT tools & OER”. A Latvian workshop participant said: “ *It had an immense impact on me, because I was totally ignorant [...] I remember that Moodle was mentioned quite often and then I went home and immediately I asked for my school administration that they […] download Moodle to our server. And then my first job I think was making a Moodle course […] And those first years were very successful.*”

The COVID crisis forced language teachers across Europe to take their teaching online. In May 2020, the DOTS / ICT-REV team, through ECML, ran a series of free webinars (live and recorded) for language teachers of English, French, and German to show the basics of online language teaching. The live event was attended by 2,850 participants, the majority of whom were from Greece, where the Ministry of Education had recommended the webinar to language teachers. Subsequently, in June 2020 the team received four requests for four follow-up events: including one from the education authority in East Macedonia [C1]. Evaluation evidence from 135 participants indicates that attending the workshop had increased their confidence to teach languages online by 30%. 50% reported that since the webinar they had used tools introduced in the webinar in their teaching practice [C8].

Key beneficiaries who have engaged with Tatlow-Golden and her research are: i) national and international public health organisations and government agencies who have invited Tatlow-Golden to act as an expert advisor and contribute to high level policy workshops, working parties, reports and consultations and ii) parents and educators who have engaged in educative material that Tatlow-Golden has designed. By engaging with these beneficiaries, the research has had an impact on public policy, law and services through shaping the agenda of public health organisations regarding food marketing to children and supporting governmental and non-governmental organisations to generate evidence regarding the impact of food marketing. In addition, it has had an impact on the understanding and learning of parents and educators by raising their awareness of the marketing strategies used to influence children’s emotions and behaviours.

Shaped the agenda of public health organisations regarding food marketing to children

The Head of the European Office for the Prevention and Control of Non- Communicable Diseases at the WHO Regional Office for Europe states that Tatlow-Golden’s research has “ contributed to a large extent” to the WHO’s understanding that there is a connection between exposure to marketing and impact on children’s health and development. Her research has supported the integrity of WHO work in this area and enabled them to make a robust defence against stakeholders (particularly from industry) who might seek to discredit WHO’s position. They also note that Tatlow-Golden’s research has shaped their understanding of the subject [C1]. Tatlow-Golden has collaborated with the WHO on a number of reports that translate her (and others’) research including the WHO’s first report on the digital marketing of foods and non-alcoholic beverages to children in 2016 [O2] and the 2018 evaluation of the WHO set of recommendations on the marketing of foods and non-alcoholic beverages to children. The 2018 report concluded that member states needed to adopt a more comprehensive approach to HFSS food marketing regulation, one which incorporated the digital sphere of advertising and extended to include adolescents, not just younger children. In May 2019, a forum of US and EU consumer organisations which develops and agrees on joint consumer policy recommendations to the US government and European Union (Trans-Atlantic Consumer Dialogue) drew on the 2018 WHO report in their resolution on the protection of children from digital food marketing [C2].

The Technical Officer for Nutrition at the WHO Regional Office for Europe testifies that the 2016 report led directly to policy dialogues with “ countries, with funding agencies, and with other inter-governmental organisations” [C3]. For example, WHO was invited to present the report at a meeting of a high-level group of the European Union, with senior representatives of 28 member states working on the topics of nutrition and obesity. Following the publication of the WHO report, two EU presidencies (Malta and Bulgaria) have included the topic of digital marketing on their agenda. Tatlow-Golden was invited to present to the Maltese presidency, again with high level policy makers from 28 countries present (February 2017), and the Bulgarian presidency included digital marketing in their Council Conclusions. They attribute a funding call from CHAFEA, the public health agency at the European Commission level, as a direct consequence of the digital marketing report. This call addressed a key message from the report; that there are currently no adequate measures to monitor online marketing to children. Tatlow-Golden has also been an invited expert to both WHO and European Commission expert workshops on marketing of food to children [C3]. As a result, the Head of the European Office for the Prevention and Control of Non- Communicable Diseases notes that “ We have seen the number of member states that have increased their attention to these issues rise dramatically”, for example [in the European region] 44% of countries have increased activities that limit marketing of unhealthy foods through actions such as strengthening regulations, implementing monitoring tools and adapting the WHO nutrient profiling. They are “ strongly convinced” this is due to the work undertaken by WHO and supported by Tatlow-Golden [C1]. Furthermore, the Technical Officer for Nutrition notes, “ really our work as WHO Europe is recognised as a benchmark globally and that is due in large part to the contributions from Tatlow-Golden“. They add that Tatlow-Golden’s research has “directly contributed to work in countries and to discussions at a political level”, and “ has transformed the debate around food marketing to children […] and helped shape the agenda and dialogue on these topics” [C3].

In October 2017, the National Institute of Public Health (NIPH) in Slovenia funded a meeting in Ljubljana that included Tatlow-Golden, the WHO and The European Commission. This workshop explored digitalisation across six lifestyle areas. In an interview, a Head at NIPH stated, “ really [Mimi] has opened completely new areas in digital marketing”, and that her sense of urgency to take action on marketing with respect to nutrition, and the use of big data to influence people’s lives, was a catalyst for NIPH to recognise the nature and scale of the challenge and to take action on these topics across all its lifestyle areas. They also noted that in May 2018 the WHO held a general assembly to debate a resolution for reducing pressure of marketing to infants and young children. Despite opposition, the resolution was eventually passed. Evidence from Tatlow-Golden’s research was used to support a robust argument in favour of the resolution [C4].

A Co-Director of the US-based Center for Digital Democracy (CDD) testifies that Tatlow-Golden’s research and policy contributions have had ‘an enormous impact on civil society advocacy and government policy making’ and are critical to CDD’s efforts which have affected policies in major companies. They give the example of CDD’s 2018 complaint to the Federal Trade Commission regarding YouTube’s failure to comply with the US Children’s Online Privacy Protection Act, which restricts gathering personal information from children. A settlement resulted in a “ significant change in YouTube’s policies that restricts food marketing to young children” [ C5].

Supported governmental and non-governmental health organisations to generate evidence regarding the impact of food marketing

Underlying all regulation and its successful implementation and evaluation is the need for structured monitoring of the marketing of unhealthy food and beverages to children in order to provide evidence of the nature and extent of the issue. In 2018, The Council of Nordic Ministers funded the Norwegian Directorate of Health to produce a protocol to monitor marketing of unhealthy food and beverages towards children and young people. Tatlow-Golden was one of the expert contributors to this protocol. An Advisor to the Norwegian Directorate of Health, testifies that Tatlow-Golden was “ a key person to get this through”. The outcome of that evaluation contributed to an assessment of existing policy, where the decision was made to uphold policy with no change needed at this time. The protocol is also being used by advocacy groups such as the NCD Alliance (a network of 2,000 organisations in 170 countries), Norwegian Consumer Council and the Norwegian Cancer Society, who all want to see stronger regulation of marketing to children [C6]. In June 2018, Tatlow-Golden was one of the invited experts at a meeting convened in Moscow by the WHO (Regional Office for Europe) on the monitoring of digital marketing. The outcome was the 5-step ‘CLICK Monitoring Framework, an overarching set of principles and approaches to monitoring digital marketing of unhealthy products to children, which can be adapted to different national contexts [C7].

Alongside the ‘CLICK Monitoring Framework’, Tatlow-Golden (with Boyland) developed a more practical, detailed set of WHO protocols, embedded within a training toolkit that offers step-by-step monitoring guidance and support; teaches about marketing, child development and related psychological issues; and provides the tools for countries to generate comparable evidence of the impact of food marketing practices. Intended users of this ‘WHO Monitoring Training Unit’ (https://tinyurl.com/y3p96j64\) include public health practitioners, NGOs and policy analysts. UNICEF has supported the implementation of this training toolkit in Asia and Latin America. As part of this initiative Tatlow-Golden (with Boyland) ran two training workshops for 19 UNICEF staff and research agency staff (Manilla, February 2020 & Buenos Aires, May 2020). Following this, commissioned by UNICEF, research agencies in Argentina, Mexico and Philippines are using the toolkit to collect data about digital marketing to children. The Nutrition specialist at UNICEF who commissioned the training and use of the toolkit testifies that in these countries, as well as Russia: “ The training toolkit has met a recognised need [...] it is the only thing that is out there, as a common good, for people to use.” He also states that it generates the knowledge necessary to start conversations with governments [C3]. The co-ordinator of the Mexican component of this work states that the protocols have stimulated them to look at variables they had not considered before and as a result has enabled them to report findings that have engaged the Mexican media and contributed to public discourse around the lack of regulation of digital marketing. The Argentinian co-ordinator states that the protocols have enabled them to “ be sure we would have robust evidence, by using a methodology you know you can defend”. The Philippines co-ordinator testifies that without the protocols they would not have attempted to build an evidence-base for legislation change in the way they have [C8].

In March 2019, the UK Department of Health and Social Care (DHSC) and the Department of

Culture, Media, and Sports (DCMS) launched a consultation regarding the potential introduction of further advertising restrictions on TV and Online for high HFSS products. The consultation document referred to the WHO 2016 paper [O2] in acknowledging the potential of a combination of online and broadcast marketing to amplify the effectiveness of HFSS marketing. Also contained within this consultation document were assumptions regarding the minimal impact of online marketing on children which was based on commissioned work from a consultancy company. In order to inform their response to the consultation and this novel assumption in particular, the Obesity Health Alliance (OHA) asked Tatlow-Golden (with Parker) to analyse the validity of the methods used by the consultants to draw their conclusions. This analysis revealed that the exposure and thus potential impact had been significantly under-estimated [O5]. This evidence was submitted to the consultation exercise and as a result, in October 2019 OHA and Tatlow-Golden were invited to meet with DHSC and DCMS officials to discuss their evidence regarding the true scale of the impact of digital marketing in more detail. Due in part to this evidence, a 9pm watershed for HFSS advertising on TV was part of the announcement for the Obesity Strategy in July 2020. The Government also committed to a further consultation into whether there should be a total ban on HFSS advertising online. When the consultation was published in November 2020, it was with a revised estimate into children’s exposure to HFSS advertising online (an increase from 0.7bn to 15.1bn). The lead for the OHA, testifies that Tatlow-Golden’s research: i) enabled the OHA to make informed decisions about the position it wanted to take ii) made it difficult for the government to deny there was a problem ii) informed the method the government used to revise its estimate of exposure [C9].

Raised awareness among parents and educators of marketing strategies used to influence children’s emotions and behaviours

In 2018, Tatlow-Golden developed an online Open Learn course called ‘Children and young people: Food and food marketing’ which was designed to informed parents and educators of the marketing strategies used to influence children’s emotions and behaviours. The course has achieved 2,885 enrolments and 1,766 completions. A survey of 198 participants who have completed the course revealed that taking the course had increased their awareness and understanding of: i) issues relating to food marketing of children (99%); ii) children’s rights (94%) iii) what needs to be done relating to issues about food marketing for children (96%). In addition, 81% said they were more likely to talk to their child or pupil about food marketing. Follow-up interviews with six teachers, nursery or youth workers indicated that their increased awareness was informing their practice. Examples include seeking out more interesting health eating books to “combat the flashy marketing of unhealthy food’; contacting the local council about an illegal ad featuring unhealthy foods and creating a food marketing resource pack for colleagues [C10].

Beneficiaries of research conducted by the OER Hub include i) open education organisations (e.g. open textbook providers, online course providers); ii) higher education institutions, professional bodies and networks with an interest in supporting educators in their use of OER or OEP; and ii) educators working in schools, further education and higher education with an interest in developing their own personal OEP or with responsibility for developing institution-practice. The OER Hub has implemented a range of pathways to impact including: i) modelling best education practices by making openly available their survey tools and datasets and courses aimed at informing and transforming open education practices ii) implementing a Research Fellowship Scheme for employees of Open Education organisations and iii) running a range of practitioner workshops. The influence and value of these pathways in supporting the development of a community of OER practitioners has been widely recognised. For example, in 2015 an independent evaluation of the work of the Hub concluded that: “ the OER Research Hub has built a solid reputation among a community of OER researchers and practitioners as leader, expert, and role model” [C1]. In February 2015, the Hub won a RCUK/ OU Engaging Research Award with judges commenting on how: “ publics were engaged meaningfully in this densely participative research”. More specifically, the OER Hub has had an impact on the delivery of professional services and performance of Open Education organisations by informing: the design and delivery of their OER; their policies and strategies and their internal professional development initiatives and practitioners by increasing their awareness of OER and OEP; increasing their capacity to implement OEP and stimulating them to develop new teaching strategies.

Informing the performance and delivery of professional services of Open Education organisations

As part of establishing an initial evidence-base, the OER Hub worked collaboratively with large Open Education organisations to help them undertake action research that would offer a better understanding of their learners and how they use OER. Examples include: BCcampus, a project in Canada developing open textbooks for high population undergraduate courses (in 2018/9 103,330 students used their textbooks); OpenStax in the United States who are the largest open textbooks provider (their textbooks are being used in 56% of college and universities in the U.S. and over 100 countries); The Saylor Academy, a foundation developing 100+ open online resources and courses for learners and Siyavula, a provider of maths and science open textbooks for K12 in South Africa (users have completed over five million Mathematics and Science exercises). These collaborations were realised through funded fellowships, whereby a researcher in the team worked with the organisation, and an individual in the organisation visited the Open University. The OER Hub developed their research skills and worked on a specific research question (related to the eleven hypotheses) they wished to pursue. In each case the collaborations produced evidence that helped shape future implementation of learning resources within the organisations. For example, The Managing Director of OpenStax testifies that a key outcome of the collaboration with the OER Hub is that: “A range of design iterations were applied to some OpenStax products to address some of the concerns in the surveys. As such the findings supported intelligent investment in improving the problem areas and potential context gaps” [C2]. The Associate Director of Open Education at BCcampus states that the collaboration with the OER Research Hub was invaluable because “ the recommendations […] provided concrete evidence to share with Ministry contacts and Institutional leaders and directly impacted the University of British Columbia’s decision to add the creation of open educational resources in their promotion and tenure guide for educational leadership” [C2]. The Director of the Community College Consortium for Open Education Resources (CCOER) reports that their fellowship helped CCOER gain a “ better understanding of how OER activities were being conducted at community colleges and faculty perceptions of the associated outcomes which proved very helpful for support and expansion of our community of practice” [C2]. The Director of Product and Research at Creative Commons testifies that the findings from their visiting fellowship were fed back into continued running of the ‘School of Open’ and that it was of benefit to “ leading, managing, and running courses and course organizers for the ‘School of Open’, which ended up impacting people in various regions around the world”. Illustrative examples of the scale of this benefit include: the launch of ‘School of Open Africa’ in Kenya, Tanzania, Nigeria, South Africa (8 programs; 400 participants) and the design and launch of 100 courses, workshops and programs with 3,000 participants to date [C2]. The person responsible for learning research and analytics at Siyavula (2015-2018) testifies that the research they conducted with the OER Hub as part of their Fellowship provided data that helped Siyavula understand the nuances of interacting with teachers and their perceptions regarding open texts. This informed future “ collaborative content development workshops that we ran with them. For example, we included some focused sessions on the various technology tools so that they could get more out of the experience and view it as a professional development exercise. This enabled us to generate a continuous relationship with them” [C2].

The OER Hub has also helped to inform the OER policy and strategy of OER related institutions. For example, working with the Association of Learning Technology (ALT) the OER Hub has helped develop a policy guide for OER adoption. ALT is the largest member organisation for educational technology practitioners in the UK with over 3,500 members. The ALT CEO stated that: “ALT’s role as an independent voice in the development of national policies has been informed by the work of the OER Hub, resulting in the publication of a document: “ Open Education and OER - A guide and call to action for policy makers” and also ALT’s formal response to the Government’s consultation on Data and Ethics in Education. The OER Hub’s approach to open practice and the evidence base they provided for the benefits of OER have been key elements in the acceptance of ALT’s policy work in this area” [C3]. The Program Officer for Hewlett Foundation (2014-17) writes: “ The findings of this research provided me and my foundation colleagues with valuable information about the general state of the OER field, as well as specific insight into key questions being studied. This information informed the grant-making strategy related to OER at the Hewlett Foundation. The Hub research helped us understand where OER use was less effective or less prevalent and adjust our approach to funding organizations that could help improve efficacy and prevalence of use. Nuanced changes in our strategy setting could be implemented as a result and provided good guideposts to help navigate our complex global grant-making work” [C4].

Influencing the open practices of educators in a range of contexts

Through a range of impact activities such as the Fellowship Schemes, workshops and supporting the development of community networks the OER Hub has influenced the open practice of educators by raising their awareness of the potential of OER, enabling them to make informed decisions about whether and how to adopt OER and increasing their ability and confidence to develop and evaluate OER policy and practice in their own communities.

The Director of Non-Profit Flipped Learning Network testifies to the influence of the OER Hub Fellowship scheme: “ Without the background and knowledge I gained though my work with the OER Research Hub Project at the Open University, I would not have been qualified, or possessed the skill set, to work with the U.S. government” on a national project which aimed to identify public schools in the U.S. that were leaders in the practice of using OER in their classrooms” [C2].

As part of the OEPS project (2014-2017), the OER Hub team carried out seven workshops encouraging exploration of open practice at H.E and F.E institutions across Scotland. There is evidence that engaging in the OEPS project stimulated practitioners to develop new or different teaching strategies. For example, a Senior Lecturer at The University of the West of Scotland recalled how attending an OEPS workshop influenced his decision to utilise open textbooks [https://tinyurl.com/rby9m26\]. The UK Open Textbook project (2017-2019) conducted 16 workshops promoting open textbook adoption at ten institutions with participants from almost 20 UK HEIs. In addition, the project participated in seven exhibitions/presentations showcasing OpenStax materials. As a result of the project nine known adoptions are attributable in whole or in part to UK Open Textbook activity and there were 16 confirmed UK recommendations of OpenStax resources [C5]. Working in partnership with the Open Textbook Network (OTN), one branch of the project ran workshops across universities in the UK and the Republic of Ireland introducing the value of Open textbooks in teaching and learning.14 workshops were held at eight institutions with invited staff from 18 universities. Of the 116 attendees, nearly half of those attending completed an OTN follow-up survey, and of those, 37% answered ‘yes’ or ‘maybe’ to whether they would adopt an open textbook [C6]. A senior lecturer who attended the workshop at Glasgow Caledonian University in 2017 testifies that it raised awareness about the potential of open texts and open education more generally. It also highlighted issues that the institution needed to address in order to support implementation of open textbooks. They are now working as an independent consultant and draw on OER Hub Resources in their work with universities, considering them to be one of her trusted resources: “ Because of the range of projects and experiences that the OER Hub has, there are quite a wide range of things on offer which gives you more of an opportunity to tailor your recommendations to colleagues” [C6]. In August 2020, OER Hub staff contributed to a COVID-19 response project funded by GCRF which delivered a range of online workshops designed to support educators in schools, tertiary education and Teacher Education in Africa to transition to online teaching. One of those workshops focused on OER. 175 educators from countries that included Kenya, Nigeria, Zambia, and Mozambique engaged in the online workshop and evaluation evidence (n=82) indicates that 91% fully or mostly understood the potential of OER and how to incorporate it into their own practice and 57% were planning to re-use, create or share OER in their own teaching [C7].

The OER Hub co-ordinates The Global OER Graduate Network (GO-GN), a network of international doctoral candidates whose research includes a focus on open education. These doctoral researchers, many of whom are teachers, are at the core of the network; around them experts, supervisors, mentors and interested parties connect to form a community of 269 practitioners who draw on OER Hub research to inform their practice. Members of the GO-GN Network have testified as to how their practice as teachers has been influenced through membership of the network. For example, the Manager in Open Educational Practice at the University of Southern Queensland writes: “ Within my institution, I have been actively working on shaping policy through development of an action plan in relation to Open Educational Practices (OEP), and drawing upon the Go-GN network has been particularly helpful in this [2019] [...]. I am very doubtful that the documentation would have been as comprehensive had I undertaken this search alone” [C8]. A lecturer at Tangaza University College in Kenya states that “ Participating in the GO-GN has also helped me to support other staff within my institution [2019]. I now coach other faculty members on how important it is to incorporate ICT and OER in teaching and learning, which I feel this has enhanced the quality of my own teaching” [C8]. An Edtech advisor at the Saxion University of Applied Sciences in the Netherlands testifies: “ Last year [2018] I set up a project within my institute based on their projects and outcomes. It was really helpful. Based on existing OER content, we developed our own professional development program for teachers on OER as well as a supporting website […] I have also sought to implement changes within my institution […] In addition to the website development, every Instructional Designer (12 in total) received training on OER so that they apply it within their work as well […] We also created our own open online courses that are not only used within our own institute, but also nationally […] by at least 6 Universities of Applied Sciences” [C8].

The TESS-India project was designed to strengthen and transform professional development and classroom practice in seven target states in India: Assam, Bihar, Karnataka, Madhya Pradesh, Odisha, Utter Pradesh and West Benghal. At the core of the TESS-India approach was an Open Educational Resources toolkit to support the professional practice-based learning of primary and secondary teachers of Literacy, Maths, English and Science (www.tess\-india.edu.in\). In addition to the toolkit, a six-week task-based MOOC was designed which utilised a ‘hybrid’ support model combining online support through course forums with face-to-face support through regular contact sessions in the seven target states across India.

The key beneficiaries of the TESS-India Project are: State Resource Groups comprising teacher educators, leading teachers and teaching education stakeholders; teacher educators in District Institutes of Education and Training (DIETS) and local teacher advisers (called Block Resource Coordinators (BRCs) or Cluster Resource Centre Coordinators (CRCCs)). The TESS-India project team has worked with these key beneficiaries in order to: 1) write, refine, prepare, film, translate and localise the OERS within the toolkit. 2) persuade each state government (State Council of Education, Research and Training - SCERT) to officially approve the materials and mandate their use 2) identify opportunities to embed the materials in teacher educator programmes across India 3) identify NGO partners to collaborate with in order to extend the reach of TESS-India in teacher education programmes.

The overall outcome of this work with beneficiaries is that 26,028 sets of OER have been distributed as print, SD card, CD/DVD and pen drives. The TESS India Massive Online Open Course (MOOC) has extended the reach and impact of the TESS-India approach enhancing digital literacies and building teacher communities. Through 2015-17, two iterations of the TESS-India MOOC (in Hindi and English) attracted over 43,000 registrations, with approximately 95% of these from India including the 7 target states, and with a completion rate of over 50% (higher than the average of 12% for MOOC completion) [C1]. Data from the TESS-India YouTube channel indicates that there were 13,700 subscribers and over 3 million views (26/11/20). More specifically, the TESS-India project has had an impact on:

1. public policy through the endorsement of the use of TESS-India OERs in teacher education programmes by state governments and by influencing approaches of non-governmental organisations in and beyond India to teacher professional development.

2. the understanding and learning of trainee teachers, teachers and teacher educators by developing their understanding of active learning.

The scale and quality of this impact has been recognised through two major awards. In 2015, TESS-India won the Bond Innovation award, with judges commenting how the project was using ‘innovation through decentralisation to address a stubborn challenge. In March 2017, TESS-India was awarded the Guardian University Award 2017 in the ‘International Projects’ category. The judges gave the following comments in support of their decision: “ We think this project is outstanding in its innovative and impactful teaching and learning approach. In terms of impact, the initiative contributes to development of new forms of teacher education, using web based, globally available technology”.

Influenced teacher education policy through state endorsement of TESS-India OERs.

The success of TESS-India OERs and MOOC has led to changes in public policy, specifically how teacher education is conceptualised and delivered in key states in India –moving away from cascade models towards professional learning, situated in the school classroom and supported by various local experts. A 2018 independent evaluation of TESS-India activity concluded that it has demonstrated an “ innovative and path-breaking approach to professional development of the teacher community’ and ‘ it has been successful in influencing teacher education activity in alignment with the current thinking in the country” [C1]. State endorsement of TESS-India approaches to teacher education can be evidenced by the inclusion of specific TESS-India activities in Teacher Education Annual Work Plan and Budgets which SCERTS submit to the National Ministry for approval and once approved are then transformed into ‘Orders’. For example in Madhya Pradesh, following government approval, the SCERT issued orders in 2016 and 2017 to use the TESS-India OER in all Teacher Education Institutions and purchased micro-SD cards containing the TESS-India OERs for all 286,000 elementary teachers at a cost of 1,432 lakhs (approximately GBP1.5 million) [C2]. In Utter Pradesh, the activities agreed in the 2015/16 Annual Work Plan included capacity building of all teachers and teacher educators on the use of the OER and three training programmes utilising the TESS-India OERs were incorporated into the pre-service curriculum for courses offered by 75 DIETs (District Institutes of Education and Training) in the state reaching 10,00 teachers and teacher educators per year [C3]. Four states (Assam, Odisha, Bihar and Madhya Pradesh) have integrated the TESS-India OER into pre-service programmes which are using the resources for lesson planning, practice teaching and project work. In five states (Assam, Odisha, West Bengal, Karnataka and Madhya Pradesh) the TESS-India OER have been integrated into large-scale in-service teacher education programmes [C1]. In Bihar, in addition to embedding the OER materials in a 2-year pre-service teacher education programme the OER have been mapped to four set textbooks to aid teacher use of the OER [C4].

Influenced approaches of NGOs to teacher professional development

The TESS-India OER have been utilised by several NGOs and community organisations in their work with teachers. For example, in 2015, STIR (an NGO working across India and Uganda) integrated the materials into a project with teachers designed to improve teacher motivation through the use of micro-innovations in their classrooms [C5]. In July 2015 TESS-India materials were also integrated into a training module for 100 UNICEF ‘intervention schools’ that was designed to promote student-centred and activity-based learning in classrooms [C6]. The National University for Educational Planning and Administration has integrated the Tess-India Leadership OER into the national programme for school leadership training to support ontological shifts in practice (February 2014) [C7]. In 2020 the OU entered into a collaboration agreement with the large Indian NGO, Gurushala, to develop an online course based on the TESS-India materials. The course is called ‘Teaching English in Secondary School’ and there are 4 modules, each with 12 hours of teacher learning. Beyond India, the TESS-India OER are heavily utilised in the ‘Transforming Teacher Education and Learning (T-TEL) programme which aims to transform programmes of initial teacher education in all public colleges of Education in Ghana. TESS-India resources are incorporated into training materials for College Tutors and handbooks to guide Student Teachers during their three teaching practice periods. T-TEL was highlighted in the DFID Education Policy (2018) for its high quality open digital teaching resources [C8]. Finally, funded by the Global Partnership for Education’s Knowledge and Innovation Exchange (KIX), a teacher professional development programme in Ghana, Honduras and Uzbekistan is adapting the TESS-India model (2020) [C9].

Developed trainee teachers’, teachers’ and teacher educators’ understandings of active learning

The TESS-India OER have reached over 1 million teachers across 7 states in India through a variety of means [C1] including: micro-SD cards distributed to 286,000 teachers in Madhya Pradesh; state sponsored school improvement programmes such as ‘Shaala Saddhi’ in Madhya Pradesh involving 25,000 schools; within in-service programmes delivered by the Sarva Siksha Abhiyan (SSA) focussing on 10,000 Maths and Science teachers in Odisha and 170,000 Maths, Science and English teachers in Madhya Pradesh and the use of TESS-India Language and Literacy OER with 235,000 elementary teachers in the Padha Bharat Badhe Bharat (Storytelling) initiative in Madhya Pradesh as part of a national early reading directive. Evidence indicates that engagement with the OERs has had a significant impact on classroom teachers understanding of how to engage children in active learning experiences. For example, 2018 data from videos, classroom observations, lesson plans and interviews with a sample of 21 pre-service teachers, teachers and teacher educators in Madhya Pradesh, Odisha and Bihar indicate changes in classroom practices where teachers are utilising activities and drawing on ideas from the TESS-India OERs [C10]. All the practitioners report changes in their own practices through generating lesson plans with a wider range of activities for their pupils involving new, creative uses of different locally available resources. For example, one teacher from Odisha said: “By using the key resources students became active and took interest to share questions. There was less gap between pupils and me. It gives me more ideas to make my class more creative and successful”. Furthermore, the whole sample consistently talked about how these changes had led to greater pupil engagement indicating that they understood how changes in their practice impacted directly on the active learning of the children. Comments include: “ Students attendance is getting better day by day due to their interest with the introduction of TESS-India materials”; “ Children are not scared to talk to the teachers as they were before” and “ The materials help in arousing children’s curiosity and allows them to do things as it encourages teachers to conduct activities’ and ‘ the child’s fear of the teacher has gone away through the strategies suggested in the OERs and videos”. The TESS-India model for systemic change centres on relational change in the practice of teacher educators and teachers. Movement in the practice of teacher educators is therefore important for sustainable change. DIET faculty have used the resources extensively in their pre-service teacher training and encouraged student teachers to use the resources for lesson planning, practice teaching as well in their project work. One outcome of this is that, informed by their observations of student teachers, teacher educators have enhanced their own understanding of active learning. For example, one teacher educator from Madhya Pradesh shared: “The [...] techniques seen in the videos have influenced my thinking and led to a lot of change in me. It has changed the way I had previously thought classrooms should be like and I looked at them differently. I also realized that I needed to change a lot as a Teacher Educator” [C10].