Impact case study database
Shaping Policy Learning on Regulatory Reforms: Co-Creating the World Bank’s Global Regulatory Impact Assessment Awards
1. Summary of the impact
Building capacity for better regulation is highly valued by international organisations working with developing economies to support transparent and effective governance. The World Bank (WB) is the global leader in promoting whole-of-government regulatory reforms through regulatory impact assessment (RIA). Capacity challenges in developing countries means the use and quality of RIA is patchy. Research carried out by Dunlop and Radaelli demonstrates governments do not learn how to use RIA if the impulse comes exclusively ‘from the top’, that is, donors and institutions. Top-down approaches are also poor at detecting innovations at the local level. Dunlop and Radaelli worked with the WB to co-create and transform these lessons into the ‘Global RIA Awards’. The Awards represent an international learning platform, assisting policymakers in 45 of the 154 developing and transition economies to share and diffuse local innovation, providing models of best practice and highlighting changed practice in regulation and policy making across a number of countries, including Armenia and Brazil .
2. Underpinning research
Regulatory Impact Assessment (RIA) of proposed policies (laws and regulations) is a policy instrument that screens proposals in terms of their efficiency, effectiveness and likely impacts on a wide range of stakeholders and the environment. This evidence-based approach improves the quality of legislation and demonstrates to foreign investors that legislative proposals are seriously tested by the government before being implemented. For these reasons, RIA is promoted to developing and transition economies by the WB. For nearly two decades, the University of Exeter’s Centre for European Governance (CEG) political scientists have used their research findings on regulatory reform to inform the agenda of these international organisations. Taking a co-creative approach, lead researchers, Dunlop and Radaelli, have used their research on the quality of impact assessment processes and the problem of policy learning within and from international organisations to shape WB initiatives.
The need for learning spaces, in which policymakers can explore different forms of RIA, contest quality in constructive ways. The creation, sharing and adaptation of international organisations’ templates is one of the central conclusions of this work.
The need for RIA processes that are sensitive to administrative and political context is a long-standing research theme. How policymakers define innovation, adopt and edit regulatory reforms is highly contingent. Though ‘how to’ guides from international regulatory consultants are commonplace, Dunlop and Radaelli’s unique contribution moves beyond lists of ‘dos and don’ts’ of RIA and conceptualises quality as a problem of policy learning [3.2, 3.3]. Using models of learning in public policy systems, they analyse the ways in which policymakers develop regulatory tools and learn from their own and others’ experiences. They have developed and applied this perspective in novel analysis of regulatory impact assessment in different contexts, showing that emulation and political learning often dominate over problem-focused learning.
Two specific findings, on individual policymakers’ learning processes, pinpoint why and how context matters.
First, their research identifies the limits of formal training in assisting policymakers’ learning about regulatory tools. In a rare experiment with actual policymaker participants, Dunlop, Kamkhaji and Radaelli [3.6] identified that, while training interventions help to reinforce existing understandings of innovation, they generate only shallow (so-called ‘single loop’) learning about how to model that practice. For deeper level lessons, participants gain more from lived experience. Thus, if international organisations are to diffuse the ‘right’ lessons [3.1] they need local, context sensitive, bottom-up approaches.
Their second key finding, in two comprehensive reviews of the triggers of policy learning and failure, demonstrates how the learning context policymakers experience is contingent and dynamic. This generates variation that ensures drawing lessons and identifying best practice is extremely complex [3.4, 3.5]. Dunlop and Radaelli demonstrate that learning is triggered by, among other things, individual policymakers’ openness to reflect and critically engage with their own work and others. International organisations can facilitate such learning by taking a reflexive approach, curating best practice exemplars, rewarding innovation and creating cooperative informal settings for communities of practice to grow.
Rather than a top-down imposition of lessons about regulatory innovation, international organisations should align epistemic goals with policymakers’ own structures of social meaning.
3. References to the research
3.1. Radaelli, C.M. (2004) ‘The Diffusion of Regulatory Impact Analysis – Best Practice or Lesson-Drawing’, European Journal of Political Research 43(5): 723-747. DOI: 10.1111/j.0304-4130.2004.00172.x
3.2. Radaelli, C.M. (2009) ‘Measuring Policy Learning: Regulatory Impact Assessment in Europe’ Journal of European Public Policy 16(8): 1145-1164. DOI: 10.1080/13501760903332647
3.3. Dunlop, C.A. and Radaelli, C.M. (2013) ‘Systematising Policy Learning: From Monolith to Dimensions’ Political Studies 61(3): 599-619. DOI: 10.1111/j.1467-9248.2012.00982.x
3.4. Dunlop, C.A. and Radaelli, C.M. (2018) ‘The Lessons of Policy Learning: Types, Triggers, Hindrances and Pathologies’ Policy and Politics 46(2): 255-272. DOI: 10.1332/030557318X15230059735521
3.5. Dunlop, C.A. and Radaelli, C.M. (2019) ‘Policy Instruments, Policy Learning and Politics: Impact Assessment in the European Union’, in Capano, G., Howlett, M. and Ramesh, M. (eds) Making Policies Work: First and Second Order Mechanisms in Policy Design Cheltenham: Edward Elgar. DOI: https://doi.org/10.4337/9781788118194.00016
3.6. Dunlop C.A., Kamkhaji J.C. and Radaelli C.M. (2015) ‘Regulators and Reform: A Quasi-Experimental Assessment of the Effects of Training Inspectors’, International Public Management Journal, 18(2): 304-322. DOI: 10.1080/10967494.2015.1023912
Grants supporting the underpinning research:
Radaelli, C.M. Regulatory Impact Assessment in Comparative Perspective, ESRC (RES-000-23-1284) 2005-2009 – £174,000.
Radaelli, C.M. and Dunlop, C.A. ALREG – Analysing Learning in Regulatory Governance, European Research Council (ERC) Advanced Grant 2009-2013 – €948,447.74
Dunlop, C.A. and Collins, N. Policy Learning Beyond the West – Kazakhstan, British Council Overseas Researcher Programme 2014-2015 – £39,600
Radaelli, C.M. and Dunlop, C.A. PROTEGO – Procedural Tools for Effective Governance, European Research Council (ERC) Advanced Grant 2016-2020 – €1,792,778
Radaelli, C.M. and Dunlop, C.A. Learning to Design (L-2-D): Regulatory Impact Assessment and Learning Processes in International Organizations, ESRC IAA Impact Cultivation Award 2016 – £2,998
Quality of underpinning research:
Outputs [1, 2, 3, and 4] are from ERC ALREG project, outputs [5, 6] are from the ESRC (2005-2009) project.
4. Details of the impact
Co-Creating the World Bank ‘Global RIA Awards’
Regulatory Impact Assessment (RIA) is one of the fastest spreading administrative tools for a government to appraise the impacts of different courses of action before it commits resource. As such, they are a vital part of securing transparent and effective governance. A growing number of developing countries use it, not just to analyse impacts of proposed regulation, but also to structure the regulatory policy process.
The WB leads on the global delivery of regulatory reform programmes. Dunlop and Radaelli have a longstanding advisory relationship with the WB (since 2008 continuing to the present day) [5.1]. This in itself is noteworthy as the World Bank’s academic collaborations are usually with economists and sector specialists as opposed to policy and regulation scholars.
Dunlop and Radaelli worked with the WB’s Regulatory Policy and Management team to design and manage the Global RIA Awards [5.1]. Launched in 2017, the Award is a transnational learning platform for policymakers engaged in regulatory reform and open to all public organisations in 45 of the 154 of the World Bank’s developing and transition client countries that currently use RIA. As a result of the Awards, changes in regulatory practice have been identified in several countries [5.7, 5.8].
The RIA Awards were co-created through an extensive programme of exchange including: an intensive study visit funded by an ESRC IAA (e.g. World Bank, Washington, 2016), research presentations (from both sides – Dunlop and Radaelli and the WB officials, e.g. ECPR Regulation and Governance conferences, Exeter 2012, Tilburg 2016), and multiple conference calls and email exchanges with the Regulatory Policy and Management section of World Bank Global Governance Practice division (World Bank, 2015- 2019). The process is grounded in Radaelli’s research on the contested concept of quality in regulatory reforms, and Dunlop and Radaelli’s research programme on learning in public policy, specifically research on the limitations, failures and problems of learning in and from complex organisations.
The research on regulatory quality and learning informed the implementation of the award. Dunlop, Radaelli and three WB officials worked together to establish the submission requirements, three award categories and their assessment criteria [5.2]. On submission requirements, research findings on the centrality of context [3.1; 3.2; 3.6] informed the resulting portfolio approach – whereby public officials submitted not only the RIA technical documentation but supporting commentaries and data of how the RIA was received by stakeholders and their own narrative [5.2].
The three award categories [5.2] – ‘influential RIAs’, won by Armenia; ‘data innovation in RIA’, won by Brazil; and ‘regulatory alternatives’, won by Bosnia and Herzegovina – are rooted in findings from Dunlop and Radaelli’s body of work on learning [3.4; 3.5]. Regulatory reform innovations are diverse, contested by stakeholders and frequently the source of new, but often over-looked, policy instruments adapted for local circumstances. The joint production of assessment criteria was facilitated by intense interaction, drawing on the WB team’s knowledge of how RIA operates in developing contexts and Dunlop and Radaelli’s research on how the stages of RIA production can trigger learning episodes.
The Global Lead Regulatory Policy and Management, confirms: “The team from the [Centre for European Governance at the University of Exeter] have been a key partner to the program. The insights brought from the Centre’s research projects on policy learning and regulation were instrumental in shaping the Global RIA Awards, and provided critical quality assurance in the evaluation of Awards’ contributions. Beyond the Global RIA Awards, it is worth mentioning that the critical insights from the Centre’s research on policy learning and regulation have had continued influence on many aspects of the World Bank’s work on evidence-based rulemaking, including our approach to capacity-building and implementation in this” [5.1].
Changing practice in regulation and policy-making
The Awards provide an informal mechanism that gives policymakers conducting RIAs, in contexts which are often low in analytical capacity, with feedback on specific cases of impact assessment which they consider to be their best efforts. Following Dunlop and Radaelli’s research, the Awards’ fundamental aim is to promote learning by sharing RIA stories and innovative approaches with RIA experts and fellow policymakers as opposed to purely looking at design features of RIA systems from the top [5.2, 5.3]. In fact, for the Brazil team “participation raised a discussion, leading to an important reflection on the RIA performance, resulting in an improvement in the RIA activity” [5.8].
The Awards highlighted and celebrated notable changes in regulatory practice and heightened engagement amongst the participants.
The 2017 edition produced a shortlist of 21 entries. The panel – including Dunlop and Radaelli – assessed each portfolio entry and worked together to provide feedback and determine the category winners. Positive press coverage was extensive [5.4], and the award-winning RIAs now feature in WB training documents as exemplars of good practice [5.5] used with developing and transition economies.
The Global Lead Regulatory Policy and Management from the WB discusses the value of the Awards: “The Global RIA Awards is successful in demonstrating and communicating tangible benefits of how impact analysis can contribute to better policy decisions. Although it is inherently difficult to attribute specific impacts of such initiatives, it is noteworthy that two of best performing countries – Armenia and Russia – have made continued improvements to their RIA systems. The media and policy attention generated by the Awards is likely to have contributed to this” [5.1].
The WB’s Regulatory Policy and Management Lead confirms: “We received an overwhelmingly positive feedback from parties within and outside the Bank. Many of the winning institutions and honourable mentions appeared very encouraged by the recognition. This provided them not only with a confirmation of the value of their work, but also an opportunity to raise the profile of RIA’s in their respective countries” [5.6].
RIA success stories: Armenia and Brazil
Success stories from Armenia and Brazil exemplify the power of the Awards as a learning platform to change and re-shape regulatory practice.
Armenia’s ‘influential RIA’ award highlighted the radical reforms for addressing the serious issue of food safety in meat. These new evaluation processes are embedded in regulatory and inspection regime. Before the Awards, RIA in Armenia were poor quality and gave very little attention in the decision-making process [5.5]. When speaking of the learning process and recognition of good practice enabled by the Awards the Head of Regulatory Impact Assessment Department, Government of the Republic of Armenia, comments how the Government of Armenia has a strengthening confidence in the role of the RIA and its benefits and “has accelerated the ongoing process of launching a new RIA decentralised model in the country with a direct Prime Minister’s office oversight function… As a first winner of the Global RIA Awards, we assure that the impact of such events is great, it is instrumental for developing countries to underpin the RIA process towards ensuring better governance and regulatory reforms. Practicing such research and rewarding events would serve as a platform for information sharing that make more and more regulators from different regions understand the benefits of introducing RIAs into their regulatory systems” [5.7].
As winners of the ‘regulatory alternatives’ Award, Brazil’s National Institute of Metrology, Quality and Technology (INMETRO) focussed on the demand for regulation of sports helmets in particular for cycling. The process INMETRO completed identified no need for regulation but rather an information campaign to increase awareness among consumers. The former Director of the Products Regulation Branch INMETRO explains how they “considered participation and, specifically, winning the Global RIA Awards an important recognition of our activities related to regulatory impact assessment.” But more fundamentally, the learning processes embedded in the methodological components of Awards process “shed light on the importance of seeking alternative sources of data for RIA … it became evident that using only databases is not enough. A specific study on the object to be regulated, in order to better understand the object’s characteristics and non-conformities, is an important support to be considered” [5.8].
5. Sources to corroborate the impact
5.1. Global Lead Regulatory Policy and Management (RPM), Global Practice, World Bank Group (by email)
5.2. World Bank (2017) The 2017 Global RIA Awards – Highlighting Innovation and Impact of Regulatory Impact Assessments (RIA) in Developing and Transition Countries https://web.archive.org/web/20210114143005/https://www.worldbank.org/en/events/2016/12/20/the-2017-global-ria-award
5.3. World Bank (2017) The 2017 RIA Innovation Awards Guidance Document, Washington, DC: World Bank
5.4. Client Country Press Coverage Summary Document
5.5. World Bank (2018) Making RIA Work in Armenia, Washington, DC: World Bank
5.6. Regulatory Policy and Management Lead (2017), World Bank Group (by email)
5.7. Head of Regulatory Impact Assessment Department, Government of the Republic of Armenia (by email)
5.8. Former Director of the Products Regulation Branch INMETRO, Brazil, currently Executive Analyst at ANP (by email)
Additional contextual information
Grant funding
| Grant number | Value of grant |
|---|---|
| RES-000-23-1284 | £174,000 |
| 28889 | £90,000 |
| 28604 | £120,000 |
| 230267 | £820,330 |
| 150716198 | £39,600 |
| 694632 | £1,549,841 |
| 0000 | £2,998 |