Skip to main

Impact case study database

The impact case study database allows you to browse and search for impact case studies submitted to the REF 2021. Use the search and filters below to find the impact case studies you are looking for.
Waiting for server

A new evidence synthesis methodology for informing agricultural and environmental decision-making

1. Summary of the impact

Policy-makers, and land managers try to make decisions based on the best available evidence. One way to answer a question is to collate existing research, and synthesise the findings. Systematic reviews do this in a structured and comprehensive manner, but are often unsuitable for broad topics.

We established and developed systematic mapping methodology for environmental and land-based decision-making. Rather than answer specific questions, systematic maps investigate the state of research within a topic, can incorporate diverse evidence, and can identify knowledge gaps.

Our methodology is now used globally to collate and summarise evidence for environmental and agricultural decision-makers.

2. Underpinning research

Decisions should be informed by the best available evidence, but often that evidence is difficult for users to find, access or even interpret. An evidence synthesis (a type of research method) uses robust and transparent methods to collate research information from a variety of sources and collate and summarise their findings for decision-makers. Until recently, this method had been little used in land and agricultural management.

Developed a searchable database of research on farmland biodiversity

Systematic reviews, which usually aim to answer specific questions, are probably the most well- known evidence synthesis method. In 2012 we were carrying out a systematic review on farmland biodiversity, but found the research too broad and disparate to successfully use the methodology. Instead, we developed a method of collating and summarising the research data into a searchable database to create a ‘map’ of research and gaps3.1.

We later illustrated how our method could be used to inform policy for organisations such as Department for Environment Food and Rural Affairs (Defra), and used our new method to address broad questions set by them for example farmland interventions to mitigate for water quality issues3.2.

In 2016, we published a methodology paper3.3 which has now been cited over 200 times, primarily in papers where researchers have used the methodology. The paper appears on the 94th percentile on altmetric, which indicates that many other online-users are sharing this research.

Methodology incorporated into international guidance

Our methodology has been incorporated into international guidance, and has been taken up by other European and global organisations (see impact section). The use of systematic mapping alongside existing systematic review methods enables users to not only use existing evidence to answer very specific questions (as in many traditional systematic reviews), but also to identify areas for future research priority, for further synthesis or identify research studies that can then be used to inform decisions. We have used systematic mapping methods for organisations from the UK, Europe and globally. These include, Defra, the Scottish government, the Environment Agency, The EU, and Canadian development organisations.

Biodiversity systematic map used to address agricultural policy questions

The method has also been incorporated into more extensive research. For example, Dicks et al.3.4, describes how our biodiversity systematic map3.1 was combined with expert opinion, to address agricultural policy questions. This paper has been cited over 100 times. Stewart et al.3.5 used systematic mapping methodology as part of a three-stage review, where a review of reviews was first carried out to identify where existing syntheses were missing, a systematic map was carried out in one of the sub-areas to identify priority areas for further research, and to identify which specific questions might be feasibly answered through further synthesis, and then a systematic review was used to suggest policy and management implications.

International application of methodology

Our method is also used internationally. For example, systematic mapping formed part of the evidence gathering part of Ceres 2030, a multidisciplinary partnership led by Cornell University, the International Food Policy Research Institute and the International Institute for Sustainable Development, and funded by Federal Ministry of Economic Cooperation and Development and the Bill and Melinda Gates Foundation. The aim is to inform more sustainable food systems globally. This includes identifying priority areas for future funding such as in water scarce regions3.6.

3. References to the research

3.1. Randall, N.P. and James, K.L. (2012) The effectiveness of integrated farm management, organic farming and agri-environment schemes for conserving biodiversity in temperate Europe - A systematic map. Environmental Evidence 1: 4. DOI: 10.1186/2047-2382-1-4.

3.2. Independent systematic review of impact of on-farm mitigation measures for delivering an improved water environment - WT0965 http://sciencesearch.defra.gov.uk/Default.aspx?Menu=Menu&Module=More&Location=None&Completed=0&ProjectID=18183

3.3. James KL, Haddaway N and Randall NP (2016) A Methodology for Systematic Mapping in Environmental Sciences Environmental Evidence 5: 7. DOI: 10.1186/s13750-016-0059-6. (200+ citations)

3.4. Dicks, L. V., Hodge I, Smith R.K., Scharlemann, J.P.W., Siriwardena G., Randall N.P., Smith H.G., and Sutherland W. J. (2013) A transparent Process for Evidence Informed Policy-making. Conservation Letters 7: 119-125. DOI: 10.1111/conl.12046.

3.5. Stewart R, Erasmus Y, Zaranyika H, DaSilva N, Korth M, Langer L, Randall NP, Madinga N & de Wet T (2014) The size & nature of the evidence-base for smallholder farming in Africa Journal of Development Effectiveness, 6: 58-68. DOI: 10.1080/19439342.2013.877060.

3.6. Ricciardi V., Wane A., Sidhu B.S., Goode C., Solomon D., McCullough E., Diekmann F., Porciello J., Jain M., Randall N., & Mehrabi Z. (2020) Scoping review of research funding for small-scale farmers in water scarce regions. Nature Sustainability. 3: 836-844. DOI: 10.1038/s41893-020-00623-0.

4. Details of the impact

  1. Methodology recognised by UK and international organisations

  2. Collaboration for Environmental Evidence (CEE) incorporated methodology

Systematic mapping is now a recognised evidence synthesis methodology by CEE, an international collaboration of organisations and individuals forming part of their methodology guidelines5.1 since 20135.2 following publication of our first systematic map (2012). The CEE now has published around 60 systematic map papers.

  1. Natural Environment Research Council (NERC) methodology adoption

The methodology has been taken up by Research Councils, e.g. NERC made £400,000 available for people to carry out systematic maps5.3 using the CEE guidance5.1 as a basis.

  1. Methodology helped inform EU and others on biodiversity and ecosystem services

Systematic mapping is one of 20 methods that has been summarised by Eklipse (an EU mechanism originally funded by the EU and now managed by Alternet, the European Science-Policy Interface on Biodiversity and Ecosystem Services), to answer requests from European governments and others, to help inform their decisions on biodiversity and ecosystem services in Europe5.4.

  1. Further development and expansion of methodology by other groups and disciplines

CEE systematic mapping3.1 has been further developed and expanded by other groups that have, for example, developed reporting guidelines5.5 visualisation software.

Our methodology paper3.3 has been used extensively for systematic mapping to inform specific interventions for projects funded by policy and management organisations. Extensive adoption of the methodology by other disciplines has occurred, with one example seen in the field of chemical policy and risk management5.6.

  1. Method used internationally, to help inform more sustainable food systems globally

The method is also used internationally. For example, Ceres 2030, a multidisciplinary partnership led by Cornell University, the International Food Policy Research Institute and the International Institute for Sustainable Development, and funded by Federal Ministry of Economic Cooperation and Development and the Bill and Melinda Gates Foundation, combined systematic mapping and other evidence syntheses (e.g. 3.6) with economic modelling to calculate the costs of meeting SDG 2 targets. It is now able to advise donors more accurately on the funding that is needed to inform more sustainable food systems globally.

  1. Systematic mapping projects by HAU

  2. Informed worldwide policy questions, such as microplastics in drinking water

Our methodologies have been used to inform specific policy questions leading to further impacts. For example, in 2018 we used the methodology to collate information on microplastics for the EU leading to further EU calls for tender for research to fill the knowledge gaps identified It has been discussed in a G7 round table meeting, influenced the World Health Organisation’s ‘Microplastics in Drinking Water’ report; has been cited in various government reports around the world, (including the European Commission, the Government of Canada, and the UK Department for Environment, Food and Rural Affairs); and informed policy briefs by the Society of Environmental Toxicology and Chemistry5.7.

  1. Adapted for UK policy guidance

The method has also been adapted for policy guidance in the UK and more widely to inform decisions. Defra created guidance for rapid versions of evidence synthesis, partially based on our methodologies, and two pieces of work that we carried out for Defra are referred to in their guidance5.8. We were also asked to pilot the guidance and provide feedback on this guidance.

  1. Applied in Defra projects, such as water availability and quality

In 2012-13, we carried out a systematic map for Defra to investigate the value of different on-farm interventions for improving water quality3.2. The project was specifically referred to in the Defra Water Availability and Quality Evidence Plan for the period 2013-20185.9. Our research was an example of the type of evidence used by the Water Availability and Quality Programme that aimed to deliver Government policy priorities at that time.

  1. Used by advisory organisations to inform management policy on farmland

The method has been used and adapted to inform specific management policy by advisory organisations. Specific policy agricultural advice/decisions that systematic mapping has been used to inform include Defra/ Environment Agency advice on cover cropping and buffer strips to reduce water pollution from farmland, Environment Agency advice on managing covers on slurry stores to reduce ammonia emissions, and Scottish Government action to support organic farming.

The slurry store study has been used as the basis of changing guidance on slurry management by the Environment Agency, who say that the study “…You may be interested to know that as well adding to our understanding of the anthropogenic aspects of the nitrogen cycle your findings have had a direct impact on our regulatory approach to slurry management...One very specific outcome is that we have withdrawn the use of slurry separation as a recognised ammonia control method.”5.10.

5. Sources to corroborate the impact

5.1. Current Collaboration for Environmental Evidence guidelines.

https://environmentalevidence.org/information-for-authors/2-need-for-evidence-synthesis-type-and-review-team/

5.2. Collaboration for Environmental Evidence 2013 guidelines (separate attachment)

5.3. NERC 2018 Environmental Evidence for the Future call (£400,000) for people to carry out systematic maps using EEJ guidelines (separate attachment)

5.4. Methods of knowledge synthesis in Eklipse https://eklipse.eu/methods/

5.5. Reporting standards for Systematic Evidence Syntheses in environmental research https://www.roses-reporting.com/

5.6. Wolffe, T.A., Whaley, P., Halsall, C., Rooney, A.A. and Walker, V.R., 2019. Systematic evidence maps as a novel tool to support evidence-based decision-making in chemicals policy and risk management. Environment international, 130: 10487. DOI: 10.1016/j.envint.2019.05.065.

5.7. Science Advice for Policy by European Academies. A scientific perspective on microplastics in nature and society. https://www.sapea.info/topics/microplastics/

5.8. Rapid evidence assessments and quick scoping reviews guidance created by Defra (separate attachment)

5.9. Defra (2013) Water Availability and Quality Evidence Plan. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/221053/pb13933-evidenceplan-water-availability-quality.pdf

5.10. Testimonial from EA (separate attachment)

Additional contextual information

Grant funding

Grant number Value of grant
1 £89,816