Impact case study database
Advancing Education Policy and Practice to Teach Science and Maths in Argentina
1. Summary of the impact
Professor Facundo Albornoz’s research on ‘Teacher Professional Development’ is a pioneering econometric study, providing experimental evidence on the efficacy of the main candidate strategies used in teacher training in Argentina (specifically, modes of structured curricula and coaching) with an explicit comparative cost-benefit analysis. Furthermore, it is also unique in qualifying and quantifying the effect of teacher experience. As stated in a recent survey by the World Bank, the methods of training evaluated in Albornoz’s research, are one of only two programs ‘that explicitly targeted teachers based on their experience, both of which resulted in student learning gains’ [H].
The impact of this research manifests in how it is currently used to inform education policy for training to teach Science and Maths; as stated by the Secretary of Education, Argentina;
‘Facundo’s insightful research was crucial in the design of content and forms of delivery of the training policy implemented by the Ministry of Education since 2018 for the secondary school system.[A]’ This work resulted in cost-savings for the Ministry of Education and led to a transformational impact on the professional development of teachers across Argentina. In addition, the research has led to a new virtual postgraduate degree, ‘Specialisation in Science Education’ being launched by the Universidad de San Andrés (UdeSA) in 2019. This is the first postgraduate degree of its kind in Argentina, and has resulted in improvements to the quality of teaching and learning, as well as enabled UdeSA to extend its reach across Argentina and Latin America. In addition, the research and its achievements have been recognised by the World Bank and UNESCO, who have disseminated the findings as best practice across the Latin American region.
2. Underpinning research
In 2015, Professor Facundo Albornoz led on a collaborative research project with academics from the School of Education, Universidad de San Andrés in Argentina. This was in response to a request from the Secretary for Innovation and Quality based at the Ministry of Education in Argentina, who was looking to develop new strategies for training teachers of STEM subjects.
The research team generated field experimental evidence on the effects of different strategies of teacher training on student learning in science in the City of Buenos Aires. The research findings were presented in a working paper published by CREDIT in 2017 [R2] and the World Bank [R3], before being developed into a full paper [R1] and accepted for publication by the World Bank Economic Review in 2018 [R1].
The research [R1, R2, R3] uses a randomised control trial implemented in state schools in Argentina to estimate the learning impact and cost-effectiveness of different teacher training methods, specifically, the use of structured curricula and coaching. A growing body of research demonstrates that teachers are among the most important determinants of student learning (e.g. Hanushek and Rivkin, 2010; Chetty, Friedman, & Rockoff, 2014). Providing individualised, repeated teacher training, associated with a specific method or task were among the most recommended interventions for improving student learning (Evans and Popova, 2016; Kremer, Brannen, and Glennerster, 2013). As discussed in Popova, Evans, Breeding and Arancibia (2018), the development of ‘pedagogical content knowledge’ and ‘ongoing mentoring’ are increasingly adopted in teacher professional development programmes around the world. Yet, despite the growing consensus on their importance, empirical evidence is scant and little is known on the impact of these training practices on student learning. The need for research to address this gap in knowledge has been identified in recent surveys on the topic (Popova, Evans, Breeding and Arancibia, 2018; Kraft, Matthew A., David Blazar and Dylan Hogan, 2016).
The team led by Professor Facundo Albornoz examined learning outcomes of 7200 students exposed to teachers randomly assigned to one of the following three groups: teachers receiving the training, teachers receiving the training and a follow-up 8-week ongoing mentoring, and a control group of teachers who did not receive any of those treatments. During 2016, the team collected information about students, teachers and administrative data about the participating schools, before and after the intervention, to guarantee that the random groups were comparable and that the ex-post outcomes were effectively associated with the intervention. Students' learning outcomes were evaluated by a written test designed by a team of specialists in science education.
The key research insight from [R1, R2] finds that there is a substantial gain in terms of learning outcomes for students when teachers are trained using structured curricula and coaching (between 55% and 64% of a standard deviation more than those students in the control group). Coaching teachers is not a cost-effective intervention since the unit cost per 0.1 standard deviation is more than twice the cost of using a structured curriculum only. However, additional coaching is particularly relevant for relatively inexperienced teachers.
3. References to the research
[R1] Albornoz, F., Anauati, M. V., Furman, M., Luzuriaga, M., Podesta, M. E., & Taylor, I. ‘Training to teach science: Experimental evidence from Argentina’, World Bank Economic Review, 2019, pp. 1-25 (available at https://academic.oup.com/wber/advance-article/doi/10.1093/wber/lhy010/5318602)
[R2] Albornoz, F., Anauati, M. V., Furman, M., Luzuriaga, M., Podesta, M. E., & Taylor, I. (2018). ‘Training to teach science: Experimental evidence from Argentina’, CREDIT Research Paper, N. 17/08 (available at https://www.nottingham.ac.uk/credit/documents/papers/2017/17-08.pdf )
[R3] Albornoz, F., Anauati, M. V., Furman, M., Luzuriaga, M., Podesta, M. E., & Taylor, I. (2018). ‘Training to teach science: Experimental evidence from Argentina’, Policy Research Working Paper, 8594 (available at http://documents.worldbank.org/curated/en/992241537378173492/pdf/WPS8594.pdf). This publication is a product of the Strategy and Operations Team, Development Economics Vice Presidency; it is part of a larger effort by the World Bank to provide open access to its research and contribute to development policy discussions around the world. Policy Research Working Papers are also posted on the web at http://www.worldbank.org/research
4. Details of the impact
Collaborative research (R1-2) led by Professor Albornoz on Teacher Professional Development programmes, has directly informed and been adopted within education policy at the Ministry of Education in Argentina. As corroborated by the Secretary for Innovation and Quality at the Ministry of Education, the work became “a cornerstone to structure the teaching programmes run by the National Ministry”. This resulted in cost-savings and a transformational impact on the professional development of teachers [A].
During 2015-16, Albornoz and researchers at Universidad de San Andrés piloted the quantitative and experimental studies in 70 schools across the City of Buenos Aires. The delivery of the training deployed a workshop format and a course format. The workshop format involved three 4-hour sessions during a timeframe of 6-8 weeks; the course format consisted of 60 hours of training delivered throughout 8-10 weeks.
Before the research intervention [R1, R2, R3], teacher training in Argentina involved costly coaching interventions as a main tool. R1 and R2 demonstrated that in most cases, a structured sequence yields similar gains in terms of learning to coaching, but at a much lower cost. Specifically, the estimated savings per students resulting from the change in the training method from coaching (USD14.7 per student) to structured sequence (USD4.6 per student) is USD10.1, an approximate 70% saving.
Following the successful pilot in Buenos Aires and discussions of the research [R1, R2, R3] with senior ranked officers, the Ministry of Education adopted the recommendations in R1, R2 and R3 and redesigned the content and forms of delivery of the teacher training policy for Sciences, which further led to a restructure of the teacher training policy for Mathematics. The commitment to teacher training and new structure of the programme (which follows the proposed research recommendations), is outlined in the Ministry of Education’s National Programme and Action Plan 2017-2021 [B]. The restructure of the teacher training policy for Mathematics (which again follows the proposed research recommendations) is outlined in the National Framework for the Improvement of Learning in Mathematics [C].
The new policies were fully implemented into the secondary school system nationally across Argentina in the academic year (Jan-Dec) in 2018, and have already resulted in significant cost-savings highlighted in the table below and corroborated by the Secretary for Innovation and Quality in Education, [A]: “We are delighted with the success of the new training programmes in 2018, which are continuing during 2019 and beyond. During the 2018 academic year, the total savings estimated for the teaching to teach Science and Maths are over USD313,000 for the first year…Should this research had not been available, the Ministry of Education would have run the risk of misusing efforts and wasting costly resources.”
National Institute for Teachers’ Training (Instituto Nacional de Formación Docente (INFOD), 2018. Figures [D]
National Implementation | Schools/ Institutions | Teachers trained | Allocated Budget | Savings* |
---|---|---|---|---|
Workshop Format (Science) | 6705 | 3993 | ARG 3,672,914 (USD 250,000) | ARG 2,416,795 (USD 40,239) |
Course Format (Science) | 4010 | 8031 | ARG 2,356,216 (USD 160,300) | ARG 4,860,858 (USD 81,113) |
Maths (Both formats) | 16,826 | 19,000 | ARG 11,499,991 (USD 191,900) |
* Savings calculated from the baseline in the research [R1, R2, R3] and corroborated by Ministry of Education in Nov 2019 [A].
In addition, R1 shed light on how the new improved practical methods and learning materials changed perceptions and better engaged the teachers in the teaching of Sciences, which was later realised in professional teaching practice: “Another useful finding was that the structured curriculum units were sufficient to affect teacher perceptions, since teachers expressed that they enjoyed teaching Sciences more, taught more hours of Sciences and that their students developed more skills. This is a crucial piece of evidence as it shows that despite an initial reluctance to adopt new teaching pedagogies, teachers do incorporate the knowledge transmitted in the training”. Secretary for Innovation and Quality in Education, Ministry of Education [A]
In January 2018, the government launched the National Plan for the Teaching of Math (Plan Nacional Aprender Matemáticas - E) in which the coaching technique outlined in [R1] was set into action through a cascade approach. This programme trains trainers, namely teachers who then train further teachers. Each trainer trains four groups of 35 teachers each throughout six sessions. This is a cost-effective model has seen significant participation in its first year with substantial savings, as confirmed by figures provided by the Secretary for Innovation and Quality in Education at the Ministry of Education [A].
National Plan for the Teaching of Math (Plan Nacional Aprender Matemáticas) 2018 Figures [E]
Schools/ Institutions | Trainers | Trainee teachers | Allocated Budget | Savings* | |
---|---|---|---|---|---|
Train the trainer programme | 4720 | 228 | 20370 | ARG 16,150,710 (USD 930,000) | ARG 12,467,201 (USD 208,039) |
* Savings calculated from the baseline in the research [R1, R2, R3] and corroborated by Ministry of Education in Nov 2019 [A]
Albornoz has worked extensively with researchers from the School of Education at the Universidad de San Andrés (UdeSA), which has led to instrumental impacts. Based on the research evidence [R1, R2] on the benefits of the structured sequences to train teachers, in March 2019, UdeSA launched a new virtual postgraduate degree, ‘Specialisation in Science Education.’ As corroborated by the Director of the School for Education: ” The research evidence highlighted a substantial gain in terms of student learning outcomes of the implementation of the structured sequences technique and provision of guides for teachers to be used in the classroom. The benefits from the structured sequences technique were critical in our decision to launch a new postgraduate degree at UdeSA aimed at training future teachers, and teacher trainers across all levels of the education system.[F]”
This is the first postgraduate degree in Argentina to train Science teachers organised around the training of structured sequences. Aimed at Hispano-American students, the eleven module 18 month course focuses on training a new cohort of teachers in the use of structured guides for practical use in schools. UdeSA have highlighted improvements in the quality of teaching and the learning process, leading to excellent attainment levels for the first cohort, as stated : “The new structured sequences technique has improved the quality of the teaching and learning process. The teachers are trained and assessed (in each module and in the final dissertation) based on the method of structured sequences. This method enables the development of a deeper pedagogical content knowledge combined with the training on the inquiry-based teaching approach….We are delighted with the results achieved from our first cohort of 56 students in 2019. All of the cohort passed, with over 73% scoring 90% and over.”) [F, G]
In addition, the new postgraduate degree is the first virtual degree offered by UdeSA, enabling the University to extend its reach and recruit students across Argentina and more widely in Latin America. Since 2019, two cohorts have joined the course, enrolling 173 students; 150 from Argentina and 23 from neighbouring countries. The monthly fee for Argentinians is ARG 4600 (about USD 77), whereas non-nationals pay USD 200 per month, totalling an estimated income of USD 290,700 “..the new postgraduate degree is the first virtual degree offered by UdeSA, and it has allowed the School of Education to reach a new and broader audience. For example, there are students from all the provinces in Argentina and from other countries in the region.” [F, G].
The research [R3] has also gained interest beyond Argentina. The World Bank’s Lead Economist in the Chief Economist's Office for the Africa Region and former Senior Economist in the Human Development Department in the Latin America and the Caribbean Region , discussed the paper [R1] extensively in a recent World Bank Policy Research Working Paper‘. In the article, it is argued that ‘[Cole, Duflo and Linden (2007), and Albornoz et. al (2018) [R3] are] the only two programs that explicitly targeted teachers based on their experience, both of which resulted in student learning gains.’ [H]. Note that the World Bank Policy Research Working Paper series disseminates research findings with applicability; hence, it supplies countries with evidence-based recommendations to inform their policies. In addition, a World Bank specialist tweeted the paper [I].
In 2017, this research [R1] also attracted the attention of the United Nations Educational, Scientific and Cultural Organization (UNESCO). The Coordinators of the Regional Programme in Science, Technology and Innovation Policy and of the National Programme Specialist for Education at UNESCO contacted the research team when they were commissioning a policy paper on how Sciences are taught in the Latin American countries. Drawing on the findings in R1, the aim of the assignment was to propose pedagogical changes and new training strategies across the region. The commissioned policy report [J], which cites R1 extensively, has been used by UNESCO to recommend best practice concerning the new teacher training across the Latin American region, as stated by the UNESCO Regional Programme Specialist. “the research directly informed our policy brief (Aprender Ciencias en las Escuelas Primarias de América Latina, 2018) which recommended these new pedagogical techniques as best practice to Education Ministries and government departments across the Latin American region. In my interactions with policymakers in the areas of Education and Science & Innovation, I base my policy recommendations to improve science literacies on the findings and implications of this study.” [K].
5. Sources to corroborate the impact
Letter from the Secretary for Innovation and Quality in Education, Ministry of Education, Culture, Science and Technology of Argentina.
Res CFE 316-17 - Formación Situada (Pg 4, PROGRAMA NACIONAL DE FORMACIÓN PERMANENTE NUESTRA ESCUELA PLAN DE ACCIÓN 2017-2021) (In Spanish)
Marco Nacional para la mejora del aprendizaje en Matemática (2018) (In Spanish)
Instituto Nacional de Formación Docente (INFOD), Ministry of Education, Culture, Science and Technology. (In Spanish)
Plan Nacional Aprender Matemáticas, Ministry of Education, Culture, Science and Technology. (In Spanish)
Letter from the Director of the School of Education at the Universidad de San Andrés.
Universidad de San Andrés Course Prospectus and Notas finales- Especialización en Educación en Ciencias. (In Spanish)
Popova A., Evans D., Breeding M. and Arancibia V., ‘Teacher professional development around the world: The gap between evidence and practice’, World Bank Policy Research Working Paper N. 8572 https://doi.org/10.1596/1813\-9450\-8572.
World Bank Tweets.
‘Aprender ciencias en las escuelas primarias de América Latina’, UNESCO-CILAC Policy Paper (2018). (In Spanish)
Letter from the Regional Programme Specialist, Science, Technology and Innovation Policies, UNESCO Montevideo.