Skip to main

Impact case study database

The impact case study database allows you to browse and search for impact case studies submitted to the REF 2021. Use the search and filters below to find the impact case studies you are looking for.

Search and filter

Filter by

  • King's College London
   None selected
  • 23 - Education
   None selected
   None selected
   None selected
   None selected
   None selected
   None selected
Waiting for server
Download currently selected sections for currently selected case studies (spreadsheet) (generating)
Download currently selected case study PDFs (zip) (generating)
Download tags for the currently selected case studies (spreadsheet) (generating)
Currently displaying text from case study section
Showing impact case studies 1 to 4 of 4
Submitting institution
King's College London
Unit of assessment
23 - Education
Summary impact type
Societal
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Approximately 20% of children in English schools have diverse language backgrounds and many experience lower academic achievement associated with their reduced English language proficiency. Research conducted by Professor Leung at King’s College London has underpinned the development of the English Language Assessment Framework for Schools (AFfS), which has been designed to address the language needs of these students. The Framework provides teachers with formative assessment tools to accurately record the language progression of English as an Additional Language (EAL) learners, and performance descriptors to chart their achievement across the curriculum. The AFfS, which is free to download, is the first theoretically underpinned and empirically validated framework for assessing EAL language learning available for use by UK schools. It is a web-based platform maintained by a national education charity and is recommended for use in schools by local and national governments. AFfS is transforming EAL learner assessment and support in the UK and has received a British Council Innovation Award for English language education.

2. Underpinning research

21% of primary and 17% of secondary school children in England have diverse language backgrounds and many experience lower academic achievement associated with their reduced English language proficiency. In a substantial body of research built up over two decades, Professor Leung has examined the ways in which learners with English as an additional language have been assessed, and how their progression might be more accurately recorded and their learning better supported.

The assessment of EAL learners

In a critical analysis of EAL and Second Language Acquisition research and assessment frameworks from international locations over the last 20 years, Leung’s research has shown that mainstream English-medium classroom activities and assessment practices have not, by themselves, provided sufficient opportunities for meaningful English language development. For example, this research has demonstrated that the use of curriculum tests and tasks normed on English as a first language can result in inaccurate assessments of the academic performance of EAL learners. Specifically, Leung’s analysis [1] showed that the dominant psychometrically orientated, standardised testing approach to learning is inappropriate because it misconstrues EAL proficiency. Further theoretical research on the conceptual limitations of psychometrics in relation to assessing language proficiency [2] highlighted the need for more context-sensitive assessment approaches that meet the needs of individual learners in their local settings. An analysis of historical EAL policy and provision [3], meanwhile, critiqued practices whereby student performance across all curriculum areas is assessed against a common set of criteria that do not differentiate between EAL and English as a first language learners, called for greater support for teachers who are expected to make judgements with limited training, and further drew attention to inconsistencies in the level and nature of specialised provision afforded to EAL learners.

Developing a Framework

The Assessment Framework for Schools (AFfS) was developed by Prof Leung and his colleagues at the University of Cambridge – Dr Michael Evans, Dr Neil Jones and Dr Yongcan Liu. It comprises a curriculum embedded EAL learner development map with assessment descriptors which is designed to be used as a formative assessment tool that enables teachers to chart pupil performance to support progress. These descriptors have been designed for use by teachers of all subject areas with limited understanding of EAL learner trajectories. The Framework also provides a means of systematically recording pupil performance that can serve as the basis for producing a summative report of achievement. The Framework is conceptually underpinned by Leung’s analyses, summarised above, of the ways in which EAL learners’ language development and progression may best be recorded and supported [1,2,3]. The Framework is further grounded upon the following research:

  • Ethnographic research investigating learner participation and discourse in school and university classrooms which challenged existing conceptualisations of communicative competence and emphasised the need for more contingent, formative assessment mechanisms focussed on the efficacy of communication in local contexts [4].

  • Case study research investigating teacher-pupil and pupil-pupil classroom discourse data [5], which mapped and analysed the interactional pathways that can support formative assessment during classroom learning activities including scaffolding techniques on the part of teachers to guide learners’ rejection of incorrect answers and identification of correct answers through reasoned discussion.

  • Research that applied the concept of dynamic assessment (drawing on the Vygotskian approach to sociocultural theory) to language development in curriculum tasks [6]. This work has provided the conceptual basis for designing tasks to track learners’ progress and identify problems that require further teaching support.

The development of the Framework was supported by the analysis of 12,000+ teacher judgements of gradations of EAL proficiency. The teachers, recruited at language education conferences and professional meetings, were invited to rank proficiency descriptors on a web platform. Their judgements on the relative difficulty levels of the descriptors were compared with other internationally used EAL scales to moderate the selection and use of the descriptors.

3. References to the research

  1. Teasdale, A. & Leung, C. (2000). Teacher assessment and psychometric theory: a case of paradigm crossing? Language Testing, 17(2), 163-184. [Winner of Best Research Paper Award 2000, International Language Testing Association]. DOI: 10.1177/026553220001700204

  2. Jenkins, J. & Leung, C. (2019). From mythical ‘standard’ to standard reality: The need for alternatives to standardized English language tests. Language Teaching, 52(1), 86-110. DOI: 10.1017/S0261444818000307

  3. Leung, C. (2016). English as an additional language – a genealogy of language-in-education policies and reflections on research trajectories. Language and Education, 30(2), 158-174. DOI: 10.1080/09500782.2015.1103260

  4. Leung, C. & Lewkowicz, J. (2012). Language communication and communicative competence: a view from contemporary classrooms. Language and Education, 27(5), 389-414. DOI: 10.1080/09500782.2012.707658

  5. Leung, C. & Mohan, B. (2004). Teacher formative assessment and talk in classroom contexts: assessment as discourse and assessment of discourse. Language Testing, 21(3), 335-359. DOI: 10.1191/0265532204lt287oa

  6. Leung, C. (2007). Dynamic assessment – assessment as teaching? Language Assessment Quarterly, 4(3), 257-278. DOI: 10.1080/15434300701481127

4. Details of the impact

The EAL Assessment Framework for Schools (AFfS) was developed to provide teachers with the tools to support EAL learners whose classroom and curricular needs may not have previously been met. Grounded on Leung’s conceptual analysis [1,2,3] and empirical classroom research [4,5,6], the AFfS provides a phase-specific development map enabling teachers to chart EAL learners’ performance and progress, and performance descriptors to support EAL learner achievement across the curriculum.

The development of the AFfS was funded by the Bell Foundation, an education charity that aims to enable speakers of EAL to overcome disadvantage through language education. In January 2017 the Foundation published the AFfS on a website along with guidance for its practical implementation in schools and classroom resources for primary and secondary teachers – the latter also having been developed by Leung and his Cambridge colleagues. The Framework is free to download and is the first theoretically underpinned and empirically validated tool for assessing the language progression of EAL learners available for use by schools [A].

Changing assessment practices in schools

Between 1 January and 1 December 2020 alone, the AFfS was downloaded 31,582 times. During 2020, support for teachers keen to use the Framework was provided by webinars: 496 attended webinar sessions in April and May; a further 2,250 viewed recordings of these webinars; and 113 attended assessment-related, online training courses [B].

The AFfS is promoted by the Bell Foundation’s five national Centres of Expertise for EAL assessment and professional development [C p.4,10] located in areas across England which have been identified as having high need for targeted EAL support and professional development training. The Bell Foundation is now extending the AFfS internationally. As of March 2020, 475 international schools and other educational organisations had downloaded the Framework from across 82 countries [B].

Educators have described the Framework as a ‘game changer’ [C p.11] in helping them to set appropriate targets for their learners. One EAL coordinator explained in a testimonial on the project website that the AFfS “has helped the EAL department show real progress [by] our students, as opposed to tracking them against the National Curriculum [ie standardised attainment criteria predicated upon English as a first language learners]. Using the old system, we couldn't see the progress as much but with The Bell Foundation assessment we can share the results with the students and parents to show real progress” [A].

Informing national guidance

The value of the AFfS has been recognised by local and national governments in the UK. In England, for example, in response to the Rochford Review (2016), which had called for further research into how teachers can be better supported to assess EAL learners, the Department for Education (DfE) cited the AFfS as an example of positive action being taken [D p.23].

Ethnic Minorities and Travellers Achievement Services based within local education authorities in England have recommended that schools use the Framework to enable staff to conduct more accurate assessments of EAL learner progression and thereby also help inform the setting of more appropriate targets for EAL learners [E], while in Scotland, the Framework is recommended by Education Scotland’s 2020 report Learning in 2(+) Languages [F] which provides guidance to education practitioners working with EAL learners. The Framework has also been recommended to schools in Wales as part of Statutory Guidance for the 2022 Welsh National Curriculum. Supporting documentation for the new curriculum, accompanying the Government of Wales’ website, states that “The MEALA Group [Minority Ethnic Achievement Local Authorities Group which represents all local authorities in Wales] can recommend The Bell Foundation’s assessment tool which is free to use for schools” [G p.7].

The Framework is also actively promoted by the National Association for Language Development in the Curriculum (NALDIC). As the UK’s subject association for EAL, NALDIC provides a professional forum for teachers of English as an additional language and promotes the development of EAL classroom practices to support bilingualism and raise the achievement of ethnic minority learners. After the DfE removed the requirement for schools to report a Proficiency in English scale for all EAL pupils in 2018, NALDIC advised schools to continue to assess learners for internal purposes, specifically recommending that they use the AFfS [H].

In 2018, a joint report by the Bell Foundation and the Education Policy Institute, an independent, evidence-based research institute that aims to promote high quality education outcomes for young people, identified prior inadequacies in EAL assessment in England and emphasised the important role played by the Framework in supporting schools to overcome these challenges by enabling teachers to conduct more rigorous assessments of students’ language proficiency [I p.12]. In recognition of its quality and significance, the Framework was awarded the 2018 British Council ELTons International Award for Local Innovation. The British Council is the world’s leading authority on English language education, and in presenting the award, the panel noted that the AFfS “ enables teachers within primary and secondary settings to effectively report English language proficiency of EAL learners to the DfE and gives practical strategies on how best to support and track progression” [J].

5. Sources to corroborate the impact

  1. The Bell Foundation (2021). EAL Assessment Framework [webpage].

  2. Testimonial from EAL Programme Officer, The Bell Foundation, 1 December 2020.

  3. Bell Educational Trust Limited. Annual Report and Financial Statements (year ended 31 December 2019).

  4. Department for Education. (2017). Primary school pupil assessment: Rochford Review recommendations: Government consultation response.

  5. Hampshire County Council. (2017). EMTAS (Ethnic Minority and Traveller Achievement Service) Position Statement.

  6. Education Scotland. (2020). Learning in 2(+) languages. Ensuring Effective Inclusion for Bilingual Learners. Good Practice of Teachers, Educational Establishments and Local Authorities.

  7. Welsh Government (2019) Language acquisition needs assessment survey toolkit: English as an additional language (EAL) (Annex C: Training presentation on EAL NAS assessment).

  8. NALDIC. (2020). EAL Assessment Framework for Schools [webpage].

  9. Hutchinson, J. (2018). Educational Outcomes of Children with English as an Additional Language. Bell Foundation & Education Policy Institute.

  10. British Council. (2018). ELTons Innovation Awards 2018 [video available online at 36:00-45:00 minutes].

Submitting institution
King's College London
Unit of assessment
23 - Education
Summary impact type
Societal
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Following a decade of spending cuts, grassroots open youth work (youth clubs, street-based and online youth work) has faced significant challenges. In providing an evidence base for the value of youth work and more youth-centred evaluation practices, research led by Dr Tania de St Croix at King’s College London has contributed to new investment in the sector and prompted change to accountability and evaluation policies and practices. Further, the research has bolstered youth-centred developments in the daily practice of open youth work and the training of professional and volunteer youth workers, ultimately benefitting some of society’s most marginalised young people.

2. Underpinning research

A decade of substantial cuts to youth work provision across the UK has resulted in the depletion of a resource that is open to young people by choice, engages with them on their terms, and attracts marginalised young people who are less likely to access mainstream services. At the same time, there has been an increasing policy emphasis on assessing youth services through predetermined, measurable outcomes. Two research studies led by de St Croix have examined the impact of these developments on grassroots open youth work; that is youth work that takes place in youth clubs, street-based youth work, and youth work with specific groups (eg young women, young refugees, black young people, LGBTQ young people, or disabled young people). Specifically, the research has documented practitioners’ and young people’s perspectives and experiences, highlighting, mapping and analysing how a combination of austerity and the increased use of outcome-based monitoring procedures has transformed practice [1,2,3,4] and is threatening the long-term future of youth work [5]. The research has also identified existing good practice in evaluation and articulated the case for more youth-centred qualitative techniques [5] and grassroots narrative methods [6] that are more congruent with the values of youth work.

Grassroots youth work practice in a changing policy context

Through policy analyses, participant observation and in-depth qualitative interviews and focus groups with 35 youth workers, the first study, published as a monograph, Grassroots youth work: Policy, passion and resistance in practice, showed how part-time and volunteer youth workers, who work directly with young people facing complex and challenging circumstances, are themselves marginalised in decision-making relating to policy and practice [1]. The research also illuminated the ways in which target cultures have distorted their practice by, for example, requiring formal evaluation methods (such as form-filling and database tracking). This has made it more difficult for them to develop person-centred, trusting relationships with young people who are inclined to be suspicious of formal measurement and intrusive tracking mechanisms [2]. In a co-authored paper with Ian McGimpsey from the University of Birmingham, researchers at King’s have conceptualised these changes as being informed by neoliberal logic, resulting in services and projects being ranked and compared according to their economic outcomes [3]. This contrasts with a tradition of youth work that is based on a commitment to critical dialogue with young people and the creation of informal education opportunities that start from young people’s agendas [4].

The value and evaluation of youth work

A follow-up study, funded by the ESRC, ‘Rethinking Impact, Evaluation and Accountability in Youth Work’ [7] examined how evaluation and accountability mechanisms are used in youth work, how young people and practitioners conceptualise the value of youth work and what approaches to evaluation and accountability are most appropriate in relation to practice. Qualitative data collection comprised participant observation in a variety of youth work settings in five regions across England (including urban and rural areas with varied demographics) alongside in-depth interviews, focus groups and youth participatory research with 140 young people, practitioners, managers and policy influencers. The study took place in a policy context in which youth organisations were encouraged by government (through the Centre for Youth Impact, see section 4) and funding agencies to prove their effectiveness through outcomes measurement. Conventional outcomes measurement approaches are problematic in youth work settings because they determine efficacy on the basis of predefined, discrete outcomes, whereas in youth work, there is no specific beginning or end and diverse needs may only emerge during the process of engagement.

The study generated new understandings and articulations of how youth work makes a vital contribution to young people’s everyday lives in the here and now, by providing an in-depth analysis of the value and importance of providing trusted relationships, safe yet informal spaces, and critical conversations around personal challenges and wider social issues [5]. The research has, in addition, revealed and unpacked the ways in which evaluation and monitoring practices are experienced by young people and youth workers as inappropriate: intrusive, time-consuming, overly formal, impractical; and as reinforcing adult-defined deficit narratives about young people [5]. The research has shown how, in contrast, young people and youth workers value accountability mechanisms that are rooted in practice, are sensitive to the complexities of young people’s lives and avoid deficit labels [5].

The research has also identified and illuminated the problematic ways in which outcomes measurement shapes overall provision towards short-term targeted projects and away from long-term open provision [5]. This is intensified by what de St Croix, with colleagues from King’s and the University of Birmingham, have named a ‘social investment machine’, in which evaluation techniques developed within and outside government attribute a financial value to outcomes and services, further disadvantaging practices that are less amenable to this kind of evaluation [3].

To address the need for a more equitable, nuanced approach to accountability that reflects the complexities of practice and the skilled nature of youth work, the research has made the case for, and recommended, mixed qualitative evaluation processes (comprising, for example, interviews, focus groups, storytelling and creative methods) and the greater involvement of young people and practitioners in decisions about evaluation and accountability [5,6].

3. References to the research

  1. De St Croix, T. (2016) Grassroots youth work: Policy, passion and resistance in practice. (1st ed.) Bristol: Policy Press. DOI:10.1332/policypress/9781447328599.001.0001

  2. De St Croix, T. (2018) Youth work, performativity and the new youth impact agenda: Getting paid for numbers? Journal of Education Policy, 33(3), 414-438. DOI:10.1080/02680939.2017.1372637

  3. De St Croix, T., McGimpsey, I. & Owens, J. (2019) Feeding young people to the social investment machine: the financialisation of public services. Critical Social Policy, 40(3), 450-470. DOI:10.1177/0261018319854890

  4. Taylor, T., Connaughton, P., de St Croix, T., Davies, B. & Grace, P. (2018) The impact of neoliberalism upon the character and purpose of English youth work and beyond. In: P. Alldred, F. Cullen, K. Edwards & D. Fusco (Eds.), The Sage Handbook of Youth Work Practice (pp.84-97). SAGE Publications Ltd. DOI:10.4135/9781526416438

  5. Doherty, L. & de St Croix, T. (2019) The everyday and the remarkable: Valuing and evaluating youth work. Youth and Policy: The Journal of Critical Analysis. Online open access publication published 18th November 2019 at: youthandpolicy.org/articles/valuing-and-evaluating-youth-work/

  6. De St Croix, T. (2020) Reimagining accountability: Storytelling workshops for evaluation in and beyond youth work. Pedagogy, Culture and Society. DOI:10.1080/14681366.2020.1855231

Funding

  1. De St Croix, T. (2018-2021). Rethinking impact, evaluation and accountability in youth work. Economic and Social Research Council (ESRC): GBP218,250. ES/R004773/1).

4. Details of the impact

In the largely practice-based field of youth work, the research has provided an evidence base for reforms to funding, policy and practice. The research impact is rooted in a collaborative approach, based on de St Croix’s lengthy professional experience in the field, and extensive engagement with young people, youth workers and policy makers/influencers throughout the research process (planning, data collection and engagement/dissemination). Research findings are shared in a variety of accessible formats including policy briefings, blog posts, workshops at professional conferences and creative methods such as film.

Strengthening the case for new investment

In the context of spending cuts that have disproportionately affected grassroots open youth work, de St Croix’s research [1,4] has helped to articulate the case for further youth work investment and amplify the voices of young people and youth workers calling for increased funding and more responsive systems of support at local and national levels of government. For example, the Chair of the Institute for Youth Work (the professional association for youth workers in England) has commented that: “We as youth workers have been empowered by Dr de St Croix’s research to push back against commissioners… Dr de St Croix’s work speaks to the frame of reference that youth workers need in order to inform the agency they have in such a system” [A].

De St Croix’s research documenting and conceptualising the value of open youth work [1] was also used by the 2019 All Party Parliamentary Group on Youth Affairs as evidence of the highly skilled role played by youth workers in providing young people with trusted relationships, safe spaces and stimulating activities [B1 p.19]. Three months after its publication in April 2019, the APPG report prompted a general debate in the House of Commons on the current sufficiency of youth services, in which new funding, including GBP500,000 in bursaries for up to 400 youth work students, was announced [B2].

Informing policy: reframing accountability and evaluation in youth work

By providing evidence of the harmful effects of the policy emphasis on target cultures and outcomes measurement and demonstrating and articulating the value of alternative ways of conceptualising and practising accountability in youth work, the research [5,6] has directly contributed to moves towards more youth-centred approaches that are more in keeping with the philosophy of youth work.

For example, de St Croix has worked in partnership with the Centre for Youth Impact, which was set up by the Cabinet Office in 2014 to encourage impact measurement in youth organisations and which works collaboratively with practitioners, funders and policymakers to improve services and support for young people across the UK. Influenced by the research [2,5] and through a ‘critical friend’ relationship with de St Croix (including regular meetings, mutual advice and invitations to speak at events and contribute to essay and blog series), the Centre for Youth Impact has moved away from recommending standardised outcome-based measurement approaches in youth work towards an approach that is more sensitive to, and better aligned with, the traditional values and practices of youth work [C1 p.18]. The Centre for Youth Impact has put this change into practice in their commissioned evaluation of one of the biggest investments in open access youth provision in recent years, the 2017-20 Youth Investment Fund [C2 p.2,4] (a joint investment between the Department for Digital, Culture, Media and Sport and The National Lottery Community Fund of GBP40,000,000 to support voluntary, community and social enterprise youth organisations to deliver, expand and create high quality local youth provision in targeted communities across England). Rather than imposing a model of evaluation from outside and restricting the evaluation to outcomes measures, they have taken a participatory approach (working in collaboration with youth organisations) and have focused on young people’s feedback and indicators of quality practice, both of which were absent from government-funded youth work monitoring and evaluation in the past. The CEO of the Centre for Youth Impact, commenting on de St Croix’s influence on the Centre’s thinking and practice, has noted that the continuation and responsiveness of its work is due in large part to de St Croix’s research: “If we had kept trying to push out some of the ways of working that Tania critiques, we wouldn't have been around…[T]he fact that we have integrated so much of the kind of thinking that Tania represents – and tried to walk a sensitive and thoughtful line – is the reason why we exist” [D].

Insights from the research also fed into the development of Labour Party policy ahead of the 2019 general election, through de St Croix working closely with the Political Advisor to the Shadow Youth Minister, giving advice on issues including: the distinctive contribution of grassroots open youth work; training and workforce development; challenges and opportunities in scaling up youth work after a period of under-investment; and the need for appropriate evaluation and monitoring. By identifying the danger of tying funding to quantitative outcomes, in contrast to previous Labour policy, Labour’s (2019) Only Young Once policy paper and its subsequent 2019 manifesto committed to reversing the funding cuts and moving towards longer-term, contextually-sensitive forms of evaluation [E1]. As acknowledged by the Political Advisor to the Shadow Youth Minister “[de St Croix’s] research on accountability and evaluation processes in youth work and her informal feedback helped shape Labour’s policy positions on youth work evaluation and contributed to wider youth policy development…[H] er expertise played an important role in shaping Labour’s commitment outlined in the Only Young Once Report to move towards stronger, more effective evidence of youth work that focuses on the long term, moving away from previous attempts to tie quantitative measures around outcomes and impact of universal youth work. Following the publication of this report, Labour at the 2019 General Election committed to build a statutory youth service that focused on non-formal learning and ensured all young people have access to high quality youth work provision that matches their needs” [E2].

In addition, the research findings have provided youth workers and organisations with an evidence-based rationale for more qualitative, youth-centred approaches to evaluation. Practitioners (youth workers, managers and evaluators in local government, charities and social enterprises) have widely engaged with King’s research on youth work, including academic articles [especially 1,2,5], blogs, essays and talks at practitioner events, and de St Croix has held meetings with over 25 organisations rethinking their approach to evaluation. This influence on practice has been recognised by the National Youth Agency – the national body for youth work in England, with wider influence internationally – which works with youth work providers to champion youth work and raise its profile among policymakers, educators and employers; train youth workers and offer accreditation; and further facilitate youth work by arranging collaboration and funding with commercial and public sector partners. The National Youth Agency’s Research and Learning Manager has acknowledged the importance of de St Croix’s research and its strong impact focus, commenting that: “de St Croix’s work helps organisations think about the implications of their practice and evaluation processes in a way that considers the practical context within which youth work operates, but places individual young persons at the centre of the process … [I]t improves the overall outcome for young people by ensuring their needs are understood as the most important element in the youth work process, and [shows] that… sometimes the ‘everyday’ contribution that youth work makes in young people’s lives is just as valuable as the ‘extraordinary.’ … [H]er mixed methodologies fill a significant gap in evaluation processes… She is regularly referenced in our advocacy reports due to the strong empirical evidence her work provides… We regularly consult with Dr de St Croix’s on various matters that concern the youth sector” [F].

Pioneering new approaches to practice

To embed research findings [5,6] in youth work, de St Croix has collaborated with charities and practitioners’ groups to develop and share alternative evaluation tools. For example, in 2020 she worked with London Youth (a London-based network of 650 community youth organisations) to deliver training in qualitative interviewing for youth workers, to enable practitioners to build their skills and confidence in using qualitative research methods for evaluation. De St Croix also worked as part of In Defence of Youth Work (a grassroots practitioners’ forum) to devise and implement a radically different approach to conventional quantitative evaluation – a qualitative and reflexive storytelling methodology. The storytelling workshop method has been used for practice development, critical professional reflection and accountability purposes by local government and third sector organisations. The approach has been shared through workshops involving 1416 participants across nine countries, and resources to run the workshop have been made available on an opensource website aimed at practitioners [G]. The workshops were developed as a form of resistance to the dominance of neoliberal outcomes measurement in youth work, as theorised by researchers including de St Croix [4], and their adaptation as a form of qualitative evaluation draws on de St Croix’s articulation of the value of alternative evaluation methodologies and her research on the importance of democratic accountability [2,6].

The storytelling resource has had an impact on practice internationally. For example, a research team at Hosei University, Tokyo, who have researched the impact of youth work in Europe in order to develop youth work practice in Japan, have taken a particular interest in the storytelling methodology and invited de St Croix and a practitioner to Japan in 2019 to lead seminars with over 200 practitioners and local policy makers. The professor who led this project has acknowledged the use of this storytelling approach in Japanese youth work and described how de St Croix’s work has “helped to develop local practice and highlight ways of communicating what youth work can achieve” [H].

5. Sources to corroborate the impact

  1. Testimonial from: Chair of Institute for Youth Work (national youth sector organisation), 3 February 2021.

  2. Documents from All Party Parliamentary Group on Youth Affairs: [B1] All Party Parliamentary Group on Youth Affairs (2019) Youth work inquiry: Final report including recommendations and summary. Leicester: National Youth Agency; [B2] UK Parliament. (2019). HC Debate (24 July 2019). vol. 663, col. 1371.

  3. Documents from Centre for Youth Impact: [C1] New Philanthropy Capital (NPC) & Centre for Youth Impact (2019) Youth Investment Fund: Learning and Insight Paper One. London: NPC; [C2] New Philanthropy Capital (NPC) & Centre for Youth Impact. (2020). Youth Investment Fund: Learning and Insight Paper Four. London: NPC.

  4. Testimonial from: CEO of the Centre for Youth Impact, 3 December 2020.

  5. Documents from the Labour Party: [E1] Labour Party (2019) Only Young Once: The Labour Party’s Vision for Rebuilding Youth Services; [E2] Testimonial from: Former Labour Party Political Advisor to the Shadow Youth Minister, 15 February 2021.

  6. Testimonial from: Research and Learning Manager, National Youth Agency, 3 February 2021.

  7. In Defence of Youth Work (IDYW) (2020) IDYW storytelling workshops: A review [website].

  8. Testimonial from: Professor at Hosei University, Japan, 18 November 2020.

Submitting institution
King's College London
Unit of assessment
23 - Education
Summary impact type
Societal
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

Despite targeted funding, participation in science-related study and extra-curricular activities in post-compulsory education remains patterned along gender, ethnicity and social class lines. To explain and better understand these patterns, researchers at King’s College London proposed the concept of ‘science capital’ – an aggregate of one’s science-related knowledge, attitudes, experiences and connections. The concept and associated research have provided the underpinning rationale for national governmental and third sector strategies aimed at supporting aspirations and public engagement in science, as well as informing national and international science assessment measures.

2. Underpinning research

The impact reported in this case study has been underpinned by a ten-year ESRC-funded programme of research led by Professor Louise Archer – the ‘Young People’s Career Aspirations in Science’ studies (ASPIRES 1 and 2) [7,8]. The research followed a cohort of students in England from ages 10 to 19 in order to better understand the factors affecting the realisation, or otherwise, of their science-related aspirations and trajectories. It began at King’s College London’s School of Education, Communication and Society in 2009 and moved to the UCL Institute of Education in 2017.

The research in the ASPIRES projects comprised a series of cross-sectional quantitative surveys and in-depth qualitative interviews. The surveys were completed by a cohort of students in five phases spanning the period from when they were in their last year of primary school (Year 6) to when they were in the final year of compulsory education (Year 13). Each survey was nationally representative of the school population in England (although students participating in one survey had not necessarily completed a previous survey). For ASPIRES 1, over 19,000 surveys were completed: 9,319 by Year 6 students, 5,634 by Year 8 students and 4,600 by Year 9 students. For ASPIRES 2, a further 20,434 surveys were completed: 13,421 by Year 11 students and 7,013 by students in Year 13 or the equivalent. The qualitative interviews were conducted over the same five phases with a smaller group drawn from the same cohort of students. These enabled the researchers to track over 80 students and their parents in ASPIRES 1 as they moved from Year 6 to Year 9 and over 60 of these same students and their parents in ASPIRES 2 when they were in Year 11 and Year 13.

The data showed that most young people reported liking and valuing science and that their parents valued science too. The data also showed that young people’s aspirations were relatively consistent over time, but few aspired to become scientists [2]. Analyses further showed that aspirations and personal perceptions of one’s ability in science are directly shaped by social class, gender and ethnicity [2]. For example, across all age groups, the students who were significantly more likely to express science aspirations and perceive themselves to be good at science came from socio-economically advantaged families. Boys were also significantly more likely to report wanting to study science in the future and to perceive themselves as being good at science. By contrast, girls were less likely to aspire to study science in the future, and more likely to state that they were not good at science and that they did not see themselves as ‘sciencey’. Ethnicity was also found to be significant in determining the likelihood of young people having science-related aspirations, with Black students overwhelmingly seeing the careers of scientist or science teacher as ‘not for me’ despite being interested in science [3]. Irrespective of class, gender and ethnicity, students with a family member with a science qualification and/or science-related job were significantly more likely to have science-related aspirations and to perceive themselves as good at science. The intersectionality of contributing factors produces a complex picture: Asian male students with a family member working in science were found to be most likely to have science-related educational and career aspirations. Conversely, students found to be least likely to aspire to a science-related future tended to be white and female with no family members working in science-related fields [1,2,3,4].

To explain these varied patterns in science aspirations and perceptions, Archer and her team proposed the concept of science capital [4]. Science capital encapsulates all the science-related knowledge, attitudes, experiences and connections that an individual acquires and builds throughout their life. It equates to what you know about science, how you think about science, what you do that is science-related and who you know who works in or engages with science. Inspired by the work of sociologist Pierre Bourdieu on the role that different forms of capital play in the inter-generational reproduction of social and economic advantage, the concept of science capital was developed to direct attention to wider structural factors affecting a young person’s opportunities to access science, and to provide an explanation for disengagement that does not rest on a deficit construction of young people as simply lacking interest.

The concept was tested empirically by comparing the ability of the science capital construct to predict science aspirations-related outcomes against a measure of cultural capital [5]. Science capital was found to be the better predictor. The concept and its application to educational contexts was also tested in practice by working in partnership with teachers and museum education practitioners in a follow-up project (‘Enterprising Science’, 2012–2017 [9]). The findings of this project [6] highlighted the constituent components of the science capital construct that educators found clear and less clear, and identified effective pedagogical practices for building science capital, such as involving families in science learning experiences and promoting the broad possibilities of science-related futures.

The concept of science capital developed and refined by Archer and colleagues provides the field of science engagement with an analytic tool that both explains and illuminates why some students feel able to participate in science-related activities and study while others do not. Their research findings have additionally shown the importance of building a science identity from a young age and confirmed the importance of how a student’s ability to ‘get on’ in life is shaped by the diverse resources they acquire from home, community and school.

3. References to the research

  1. Archer, L., DeWitt, J., Osborne, J. F., Dillon, J. S., Wong, B. & Willis, B. (2013). ASPIRES Report: Young people’s science and career aspirations, age 10–14. London: King's College London.

  2. Archer, L., DeWitt, J. & Wong. W. (2013). Spheres of influence: What shapes young people’s aspirations at age 12/13 and what are the implications for education policy? Journal of Education Policy, 29(1), 58-85. DOI:10.1080/02680939.2013.790079.

  3. Archer, L., DeWitt, J. & Osborne, J. (2015). Is science for us? Black students’ and parents’ views of science and science careers. Science Education, 99(2), 199-237. DOI:10.1002/sce.21146

  4. Archer, L., Dawson E., DeWitt, J., Seakins, A. & Wong, B. (2015). ‘Science Capital’: A conceptual, methodological, and empirical argument for extending bourdieusian notions of Capital beyond the Arts. Journal of Research in Science Teaching, 52(7), 922-948. DOI:10.1002/tea.21227

  5. DeWitt, J., Archer, L. & Mau, A. (2016). Dimensions of science capital: exploring its potential for understanding students' science participation. International Journal of Science Education, 38(16), 2431-2449. DOI:10.1080/09500693.2016.1248520

  6. King, H., Nomikou, E., Archer, L. & Regan, E. (2015). ‘Teachers’ understanding and operationalisation of ‘Science Capital’’. International Journal of Science Education, 37(18), 2987-3014. DOI:10.1080/09500693.2015.1119331

Funding

  1. Osborne, J., Archer, L. (2009-2013). Science aspiration and career choice: age 10-14 (ASPIRES 1). Economic and Social Research Council (ESRC): GBP745,592.

  2. Archer, L., DeWitt, J., Francis, B., Osborne, J. (2014-2019). Young People's Science and Career Aspirations age 14-19 (ASPIRES 2). ESRC: GBP842,092.

  3. Archer, L., Dillon, J., (2012-2017). Enterprising Science Programme. BP Chemicals Ltd: GBP1,600,088.

4. Details of the impact

The concept of science capital and the detailed findings from the ASPIRES research have formed the bedrock for the design and implementation of new policies, initiatives and assessments aimed at widening participation, which recognise the importance of building learners’ science identities from an early age and the role played by families and schools in forming young people’s science-related aspirations.

Science identity and science capital at the heart of international and national assessments

On the international stage, ASPIRES research has had a direct impact on how the OECD’s Programme for International Student Assessment (PISA) evaluates science education. PISA, implemented every three years, assesses 15-year-olds’ performance in the fields of reading, mathematics and science. The results are widely reported and debated in the media and often used as a measure of a country’s educational success. Science capital is listed as a key indicator in the new ‘Scientific Identity’ dimension that is being introduced as part of PISA 2024. This new dimension reflects the extensive evidence linking identity to science learning and attainment and is identified as “an important and urgent priority” in the PISA 2024 Strategic Vision and Direction for Science report [A p.12]. The report further states that science capital “plays a key role in shaping the extent to which a young person experiences science as being ‘for me’ or not, and significantly relates to post-16 science aspirations and progression.” [A p.14]. 79 countries currently participate in PISA and many shape their curricula in response to its specific dimensions.

Domestically, ASPIRES findings have impacted assessment practices in primary schools across England with science capital now a key criterion in the Ofsted-endorsed Primary Science Quality Mark (PSQM) [B1]. This national award scheme recognises schools that demonstrate effective and confident science leadership and is delivered as a year-long teacher professional development programme. Clare Warren, a senior PSQM leader, reported that “Science capital is an important concept for primary schools to consider, because children aged 10 with low science capital, who also expressed no interest in STEM careers, were very unlikely to have changed their minds by the age of 14. It is for this very reason that, when the PSQM criteria were amended in 2017, the new criterion, ‘L3: There is a commitment to developing all children’s science capital’, was incorporated” [B2 p.22]. The criterion assesses the ways in which schools build children’s science capital through extended outreach work with parents and involving children in extracurricular activities. The PSQM impacts 240,000 children and 9,000 teachers annually.

Influencing strategic plans aimed at building young people’s science aspirations

Insights from ASPIRES findings regarding the social factors shaping young people’s participation in science, and the root causes of disengagement, have directly influenced governmental strategies for diversifying and widening participation in science education and careers. In England, the 2017 House of Commons Science and Technology Committee’s report on ‘Science Communication and Engagement’ was commissioned to enhance engagement with the public on funding, policy and future research directions in science. The report draws on the concept of science capital to frame its discussion on the opportunities for promoting greater science awareness in the public sphere [C1 p.3,9,10,11,26]. The report was informed by evidence submitted by the National Forum for Public Engagement in STEM (a collective of key funders and organisations involved in setting the agenda for public engagement in STEM nationally). The Forum’s evidence highlighted the significant contribution of ASPIRES research findings in focusing their member organisations’ attention on the ways in which children’s science identities are formed and how, for some children, intersectional factors combine with the result that science careers are seen as ‘not for me’ [C2 p.5,7].

ASPIRES research findings relating to socio-economic disadvantages were used in the 2017 Scottish Government’s STEM Strategy to frame the broad societal challenges that need to be overcome to widen participation in STEM education and careers in Scotland. The Strategy referenced the ASPIRES research as part of its ‘Evidence Base’ [D p.17], in particular its identification of the role that ‘key influencers’ (ie teachers and parents) play in promoting science careers awareness and the need for STEM-related subjects to be promoted across all school ages. The social contexts in which science capital is acquired and built are also noted in the 2019 Scottish Parliament’s inquiry into ‘STEM in early years education’ [E1 p.3]. The report of the inquiry highlights correlations between the inclusion of STEM education in the early years curriculum and children’s continued competence and interest in STEM into adulthood, noting the importance of STEM education in the early years for children who are socio-economically disadvantaged and/or have limited access to STEM capital in the home. This inquiry prompted the subsequent creation of a new online module for early learning and childcare staff across Scotland to help them inspire young children in science and maths [E2].

Refocusing outreach and engagement programmes

National organisations, science museums, learned societies and charities have used the concept of science capital, and the findings of associated research on how it can best be built, to direct their engagement and outreach activities. For example, the Institute of Physics – the professional body and learned society for physics in the UK and Ireland – used the research to refocus its programming upon families with primary-aged children. The Public Programmes Manager at the Institute states that, “The work of the ASPIRES project, and particularly the definition of Science Capital, has been invaluable in guiding and shaping the family outreach and public engagement work done by the Institute of Physics. A key pillar of the IOP’s work is in raising public awareness and appreciation of science in everyday life and how science is addressing the challenges facing society. In response to the ASPIRES findings, the focus for outreach and engagement activities are families with KS1/KS2 children to try and reach them at a time when we could have the most impact. Our approach is to engage the whole family, with follow-on activities and resources for the home, to support families to have conversations about the relevance of science, empowering them to gradually build their own science capital” [F].

The British Science Association (BSA), the learned society founded in 1831 to aid the promotion and development of science, has similarly used ASPIRES findings to change its public engagement priorities. The Chief Executive of the BSA notes, “The ASPIRES findings showed very clearly that our [previous] approach was flawed: most children had already decided they did not want to ‘be’ a scientist by the time they finished primary school. We decided to prioritise primary-aged children instead of secondary-aged children and switched the majority of our education programme resources to our under-developed primary CREST scheme instead. Since 2016, we have reached over 50,000 primary school children directly, and many more indirectly. We brought many of our CREST partners and stakeholders with us on this journey and the ASPIRES ‘killer slide’ [1, figure 2, p.2] was part of our standard presentation template during this period, to explain the rationale behind our change of strategy” [G].

Changing the direction of organisations’ thinking

ASPIRES research findings have been used by organisations to underpin their strategic objectives. For example, the Science Museum Group has described its commitment to growing the science capital of their audiences as a core priority, stating, “We use the principles of science capital to shape all science engagement experiences… our organising principle is to build science capital to enrich people’s lives and enhance their contributions to society” [H p.16]. The BSA, meanwhile, acknowledges that the research has changed the very working practices of their organisation. Its Chief Executive explains that “the work of Professor Archer and her team has helped us understand that the BSA itself needs to change, if we are to succeed in our goal of enabling more people to develop a stronger ‘science identity’. This led to us putting equality, diversity and inclusion at the heart of our mission. We recognise that we are part of the field and therefore we need to change. We have committed to improving the diversity and inclusivity of our workforce. We have invested in our own audience analysis to better understand how the most under-served communities relate to science… [ASPIRES research] has had a significant effect on our work and our approach, and I believe it will continue to do so for many years” [G].

The impact of the ASPIRES research programme continues to grow. The research has been awarded the British Educational Research Association Public Engagement and Impact Award in 2018, was a finalist in the ESRC Outstanding Societal Impact prize of 2019 and has seeded subsequent studies and initiatives nationally and internationally. The Impact Case Study submitted by the UCL Institute of Education further describes the impact of research which utilises the concept of science capital to guide more equitable science engagement initiatives.

5. Sources to corroborate the impact

  1. Organisation for Economic Co-operation and Development (OECD) (2020) PISA 2024 Strategic Vision and Direction for Science.

  2. Documents associated with Primary Science Quality Mark: [B1] Primary Science Quality Mark (PSQM) Awards Criteria and descriptors; [B2] Association for Science Education (ASE) (2019) Primary Science Special Issue: Primary Science Quality Mark (PSQM).

  3. Documents associated with House of Commons Science and Technology Committee’s report on ‘Science Communication and Engagement: [C1] House of Commons Science and Technology Committee (2017) Science communication and engagement: Eleventh Report of Session 2016-17 (HC162); [C2] Written evidence submitted to the House of Commons Science and Technology Committee by the National Forum for Public Engagement in STEM.

  4. Scottish Government (2017) Science, Technology, Engineering and Mathematics (STEM): education and training strategy.

  5. Documents associated with Scottish Parliament’s inquiry into ‘STEM in early years education’: [E1] Scottish Government (2019) Education and Skills Committee: STEM experiences in early years education. Submission from Education Scotland; [E2] Scottish Government (2020) Learning in STEM in early years [press release].

  6. Testimonial from: Public Programmes Manager, Institute of Physics (IOP), 23 October 2019.

  7. Testimonial from: Chief Executive, British Science Association, 18 February 2021.

  8. Science Museum Group (2020) Inspiring Futures: Strategic Priorities 2017-2030.

Submitting institution
King's College London
Unit of assessment
23 - Education
Summary impact type
Health
Is this case study continued from a case study submitted in 2014?
No

1. Summary of the impact

During public health crises, the public’s adherence to health guidelines plays a significant role in reducing illness and death. However, the public may misunderstand or be confused by health messages and fail to comply with guidance. Prof Rundblad’s research at King’s College London’s School of Education, Communication and Society on safe water use has investigated the ways in which health messages should be communicated for optimal comprehension. Its findings have led to water and health bodies in North America and Australia making changes to the wording, phrasing and structuring of the advice they disseminate, resulting in reduced illness and death.

2. Underpinning research

Rundblad’s research systematically examines the clarity of and compliance with health and safety guidance related to water use. Prior research in this area is limited and Rundblad’s work has played a key role in developing new approaches to identify the optimal content and structure of communications to instil public trust and compliance.

Through four funded projects, Rundblad has devised and tested a new technique known as cognitive discourse analysis. This involves systematic analysis of messages through a focus on the interplay between explicit language construction, intended meaning and audience comprehension. For example, the use of metonymy was found to confuse audiences and cloud the lines of responsibility, resulting in reduced adherence to health advice [1]. (Metonymy is a figurative form of language where a linked term, eg ‘the authorities’ is used instead of stating what is actually meant, eg ‘scientists working for the local water and health bodies’.) Rundblad’s research has also identified significant communication shortfalls arising from the use of words, such as ‘contaminant’, which have a particular (often neutral) connotation in scientific contexts and a very different (often scary) everyday meaning [2].

To complement her linguistic analysis, Rundblad has analysed how such messages are understood by the intended recipients, the public, through comparison and correlation with analyses of additional qualitative and quantitative data. For example, a web-based questionnaire, in which participants were asked to respond to a hypothetical public health situation, was used to determine the likelihood of public compliance with official guidance [3]. In other studies, the percentage compliance with guidance was compared with the mode and provenance of the communication to determine the relative efficacy of message formats [4,5].

Rundblad’s research directly addresses specific public health crises. For example, following the 2007 Gloucestershire flooding disaster, the Leverhulme Trust funded [7] research to examine the reasons for the dangerously low levels of compliance with public health recommendations relating to tap water consumption. Rundblad identified the use, misunderstanding and omission of particular words and instructions that resulted in unhealthy behaviours. For example, although individuals were told not to drink tap water, they were not told not to drink boiled tap water (contrary to common belief boiling does not always render water safe) [3,4].

Subsequent funding from the Department for Environment, Food and Rural Affairs [8] enabled the first ever comparison of compliance with official health recommendations relating to water use following natural disasters with those relating to human-error incidents [5]. The findings indicate that the public are more likely to follow precautions with respect to human-error incidents than with respect to natural disasters, as the latter normally give rise to multiple risks requiring multiple messages. These findings underscore the need for greater specificity of advice at times when the situation is complex and wide-reaching.

In two studies funded by the US Water Research Foundation [9,10], Rundblad compared the language used by the water industry, health authorities and the media. In the first study, Rundblad found that the form and simplicity of language in media reports adversely affected public perceptions of water contamination [1,6]. For example, she found that, without the full scientific explanation of terms, the public were uneasy with processes described as ‘unregulated’ and that the inclusion of the term ‘unknown’ in communication messages resulted in significantly less trust in the content of such statements. In the second study, Rundblad researched ways of ameliorating messaging shortcomings identified in the initial study. For example, attention was directed to the efficacy of communications targeted at women, and recommendations were made about how specific terms should be used to avoid the communication of ambiguous messages [6].

3. References to the research

  1. Rundblad, G., Tang, C., Knapton, O., Ragain, L., Myzer, M., Tytus, A. E., Breedlove, J. & Cooke, R. (2013). Consumer Perceptions and Attitudes Toward EDCs and PPCPs in Drinking Water. Water Research Foundation.

  2. Tang, C. & Rundblad, G. (2020). A media brew of implied, hidden and unknown risk claims: cognitive discourse analysis of public health communication. In Demjén, Z. (Ed.) Applying Linguistics in Illness and Healthcare Contexts (pp. 242-268). London: Bloomsbury Academic. DOI: 10.5040/9781350057685.0018

  3. Rundblad, G. (2008). The semantics and pragmatics of water notices and the impact on public health. *Journal of Water and Health, 6(*S1), 77-86. DOI: 10.2166/wh.2008.130

  4. Rundblad, G., Knapton, O. & Hunter, P. R. (2010). Communication, perception and behaviour during a natural disaster involving a 'Do Not Drink' and a subsequent 'Boil Water' notice: a postal questionnaire study. BMC Public Health. 10, 641. DOI: 10.1186/1471-2458-10-641

  5. Rundblad, G., Knapton, O. & Hunter, P. R. (2014). The causes and circumstances of drinking water incidents impact consumer behaviour: comparison of a routine versus a natural disaster incident. *International Journal of Environmental Research and Public Health, 11(*11), 11915-11930. DOI: 10.3390/ijerph111111915

  6. Rundblad, G. (2017). Terminology Guidance for Water Professionals (or, What You Say Is Not What People Hear...). Water Research Foundation.

Funding

  1. Rundblad, G. (2008-10). The impact of language and cognition on compliance during a natural disaster. Leverhulme Trust: GBP93,472.

  2. Rundblad, G. (2009-10). The Impact of language and cognition on compliance during a natural disaster - Improving communication on cryptosporidium and 'Boil Water' notices: lessons from Pitsford. Department for Environment, Food and Rural Affairs: GBP12,000.

  3. Rundblad, G. (2011-2013). Consumer perceptions and attitudes towards EDCS and PPCPS in drinking water. US Water Research Foundation: GBP117,592.

  4. Rundblad, G. (2015-2019). Terminology for Improved Communication Regarding CECs. US Water Research Foundation: GBP61,308.

4. Details of the impact

Rundblad’s research has helped improve compliance with public health messages in the US and informed the construction of risk communication protocols by national and state agencies in the US, Canada and Australia.

Improving compliance with public health messages

Crafting effective health messages involves making decisions about content and rhetorical structure: what should be revealed, and what concealed? The efficacy of a message can be gauged by the public’s compliance and their trust in the issuing and/or mediating authority. Rundblad’s research has highlighted the importance of consistency in use of terms, and the need to explain why particular measures must be taken and what will happen if measures are not heeded. These insights and the more specific findings from her research have directly fed into the construction of guidance by water authorities in the US, preventing illness and death from waterborne diseases and other water-related morbidities, and concomitantly reducing levels of anxiety and stress. This impact has been achieved through Rundblad’s work with the Water Research Foundation, a charitable research and development organisation, part funded by individual water companies, which has sponsored Rundblad’s research and invited her to present her findings at water industry conferences, and to lead professional training programmes through a series of workshops and webcasts, resulting in her recommendations being widely taken up.

For example, Rundblad’s recommendations were adopted during a Legionella outbreak in Arkansas in 2018. Legionella bacteria are spread through droplets and can cause Legionnaires’ disease – a kind of pneumonia with a high mortality rate. The local water company, Central Arkansas Water – the largest purveyor in the state covering around 450,000 people – used Rundblad’s research to create new consistent messaging, particularly taking note of her findings regarding choice of words, the use of explicit instructions and the quantification of risks [4,6]. This new messaging was specifically noted for its role in effectively communicating appropriate actions and was described as having diffused a “volatile situation” regarding the public’s reaction to the threat and having prevented Legionella-caused mortalities [A]. In a statement expressing appreciation of Rundblad’s research, the company commented that “our most valuable commodity is trust from our customers… [W]e have benefited from the work Rundblad has done and we have dealt with a lot of controversial issues in which the guidelines have been a dream” [A].

The Philadelphia Water Department (PWD), serving water to 1.7 million residents, used Rundblad’s research [1,6] on the appropriate use of terms and the need for explicit messaging to shape its communication strategy during the 2015–2016 nationwide public health scare concerning lead contamination of water. In particular, the research informed the Department’s use of key words such as ‘chemical’ and ‘plumbing system’ to ensure that consumers were ‘savvier’ in their decisions to reduce their exposure to lead [B]. PWD Director of Laboratory Services credits Rundblad’s work [2] for the Department’s new approach to their communications with the news media and public, stating that “Rundblad’s work has made us reconsider the use of certain terms and phrases as she found that they are perceived by consumers to be ‘scary’. Instead, we have matured our communication materials to use wording that Rundblad’s work indicates will be well-received by the public” [B].

The Metropolitan Washington Council of Governments (COG), an independent, non-profit association made up of 24 local governments, including Maryland and Virginia state legislatures, used Rundblad’s research [1,4] when communicating with their 5.5 million consumers and governmental agencies including the US Congress and the Pentagon. The Principal Water Resources Planner at COG stated, “outputs from Rundblad’s publications and consultations, whether they take the form of tangible guidelines or training and presentations, have the singular forceful message that clarity in language and thought are integral to communicating to the public… [T]he importance of such work can be further gauged by the way it has been mobilized at the various local and national levels to communicate with a total of 300 million Americans about the water they consume” [C]. In 2019, COG partnered with the Centers for Disease Control and Prevention and the National Association of County and City Health Officials to use Rundblad’s research [4,5] in risk communication to train 110 members how to respond to and communicate with consumers if there was a water outage. Rundblad has also advised COG on how best to communicate messages around either low or high levels of endocrine-disrupting compounds and pharmaceuticals, and personal care products, that may be present in drinking water due to farming and industry pollutants or wastewater contamination.

Informing the construction of risk communication protocols

In the US, the Environmental Protection Agency requires public water utilities to issue drinking water ‘advisories’ when the water is, or may be, contaminated. In 2016, to support the utilities, the Centers for Disease Control and Prevention (CDC), the Environmental Protection Agency and other health and water organisations produced a Drinking Water Advisory Communication Toolbox, known as the CDC Toolbox [D1]. Drawing directly on recommendations from Rundblad’s invited presentations to the CDC, and her publications [3,4], the CDC Toolbox comprises information about when to publish public alerts and examples of appropriate text to use. Providing guidance to the 151,000 public water systems in the US and also used across Canada, the toolbox has enabled the CDC to develop and refine risk communications around water-related emergencies, including: hurricanes Irma and Maria, wildfires in California, an earthquake in Puerto Rico in 2017 and flooding in Houston in 2019 [D2]. A CDC Deputy Branch Chief stated, “Part of the value of the toolbox is giving local health and water utility leaders a framework for transparently communicating risk in a way that effectively addresses community concerns about the safety of their water… During crises, public health officials need science-based approaches based on good research” [D2].

Rundblad’s work [1] on message development and consumer behaviour also informed the production of a 2018 US-wide Risk Communications Toolkit for Cyanotoxins and other algal bloom-related water contamination. Half of all US states report harmful algal blooms in freshwater bodies each year, with incidence rising due to increased global temperatures. Toxins produced by algal blooms can affect the nervous system, brain and liver, and there is no treatment available for animals, including livestock and pets. The toolkit authors solicited the input of Rundblad to inform their framing of messages [E] but additionally note that she provided them with key insights regarding how vulnerable consumers are likely to perceive cyanotoxin risk management messages, and the importance of explaining why water should not be boiled (as this would concentrate the toxins). As with the CDC Toolbox, the Cyanotoxin Toolkit provides clear guidance to water and health practitioners drafting health messages about the framing of instructions and the potential for misunderstandings. The COG’s Principal Water Resources Planner and a key stakeholder in the production of the Toolkit stated, “When Rundblad consulted [o]n the Cyanotoxin risk Toolkit in 2017, the result was a clear guide for public health organizations, water utilities, and government bodies to follow when communicating to the public about algae blooms in water” [C].

In Australia, Rundblad’s research [3], communicated via invited presentations to New South Wales Health Water Unit informed the development of risk communication protocols for use across the state with its 8 million consumers. Specifically, Rundblad’s work on compliance with health advice during different types of water incidents [4,5] guided the design of an ‘alert template’ that the Chief Health Officer uses to issue drinking water advisories, for example during seasonal bushfires or at times of drought. The Manager for Water Unit NSW Health explained; “Through the protocol, consumers in New South Wales are provided with the facts in a way that’s understood and that is a great reassurance to them. Rundblad’s work enhanced our consumers’ well-being and trust” [F]. This manager also reported that the protocol and alert template have successfully prevented illness from waterborne diseases.

5. Sources to corroborate the impact

  1. Testimonial from: Compliance Manager, Central Arkansas Water, 3 June 2020.

  2. Testimonial from: Director Bureau of Laboratory Services, Philadelphia Water Department, 6 February 2020.

  3. Testimonial from: Principal Water Resources Planner, Metropolitan Washington Council of Governments, 17 June 2020.

  4. Documents corroborating the development and use of the Drinking Water Advisory Communication Toolbox: [D1] US Center for Disease Control and Prevention, the US Environmental Protection Agency, the American Water Works Association, the Association of State and Territorial Health Officials, the Association of State Drinking Water Administrators and the National Environmental Health Association (2016) Drinking Water Advisory Communication Toolbox (CDC Toolbox), 3rd Edition; [D2] Testimonial from: Water Preparedness and Response Deputy Branch Chief, Waterborne Disease Prevention Branch, CDC’s National Center for Emerging and Zoonotic Infectious Diseases, 26 March 2020.

  5. Water Research Foundation and American Water Works Association. (2018). Four Steps to Effective Cyanotoxin Communications: A Risk Communications Toolkit.

  6. Testimonial from: Manager, Water Unit, Environmental Health Branch, New South Wales Health, 26 March 2020.

Showing impact case studies 1 to 4 of 4

Filter by higher education institution

UK regions
Select one or more of the following higher education institutions and then click Apply selected filters when you have finished.
No higher education institutions found.
Institutions

Filter by unit of assessment

Main panels
Select one or more of the following units of assessment and then click Apply selected filters when you have finished.
No unit of assessments found.
Units of assessment

Filter by continued case study

Select one or more of the following states and then click Apply selected filters when you have finished.

Filter by summary impact type

Select one or more of the following summary impact types and then click Apply selected filters when you have finished.

Filter by impact UK location

UK Countries
Select one or more of the following UK locations and then click Apply selected filters when you have finished.
No UK locations found.
Impact UK locations

Filter by impact global location

Continents
Select one or more of the following global locations and then click Apply selected filters when you have finished.
No global locations found.
Impact global locations

Filter by underpinning research subject

Subject areas
Select one or more of the following underpinning research subjects and then click Apply selected filters when you have finished.
No subjects found.
Underpinning research subjects