Impact case study database

The What Works Centre for Wellbeing (WWCW) was established to inform policies to improve wellbeing. WWCW marked a shift in Government thinking in how to evaluate policies across all departments from using purely monetary metrics to incorporating wellbeing outcomes as a measure of social impact. As a foundational programme of WWCW, the UEA Work and Learning team, led by Daniels, have ‘… been instrumental in consolidating and ensuring the sustainability of the Centre, its place in the What Works network and as an influencer of policy at all levels of government, the third sector and employers’ ( S1, Executive Director of WWCW).

UEA’s research has shaped new national policy frameworks; led to changes in organisational practices; and influenced standards and professional practice:

1) By providing a robust evidence base, the research has contributed to building a public dialogue and allowed informed policy discussions. As a foundational programme of WWCW, UEA’s research has ‘…raised public and policy awareness of wellbeing and related issues such as the importance of the workplace as a source of social connections and resilience, and the contribution of wellbeing to organisational performance’ ( S1). As well as reach through WWCW ‘… engagement with over 12k people per month through our [the WWCW] website, of which 75% are across the regions of the UK and 25% are overseas; 13k followers on Twitter; courses delivered by WWCW staff have attracted over 4000 attendees; there have been over 18k of downloads of WWCW products’ ( S1), the UEA team’s direct reach into non-academic dialogue is by over 130 citations to UEA led research in policy reports, the professional and popular press ( S2).

UEA research has identified gaps in Government departments’ evidence. The Lead Analyst at the Employers, Health & Inclusive Employment division of the Department for Work and Pensions (DWP) reported that a “… *significant element of the work has been to help us identify gaps in the evidence base, specifically around return to work, which informs our own research and has been cited in a DWP consultation.*” ( S3). Writing to Daniels, Economic Adviser at the Labour Market Directorate, Department for Business, Energy and Industrial Strategy (BEIS) stated that that “…I wanted to comment on the value of your research relative to BEIS’s current policy priorities… As a result of the Taylor Review of Modern Employment Practices and the Good Work Plan, BEIS now places equal importance on the quality as well as the quantity of work. Wellbeing is closely linked to job quality, the WWC states that ‘having a job is good and having a good quality job is miles better’. Your research has helped to inform our evidence base in this area.” ( S4).

2) Informed by the research, the UEA team, and WWCW more broadly, have worked with policy and practice groups on standards, guidance and practice. UEA team members ‘… have been responsive to requests for information by central Government departments, including the Cabinet Office and No.10’ ( S1). Key stakeholders benefit from consultations that draw on UEA research (e.g. Public Health England, DWP, BEIS, Department of Education {Teacher & Leader Wellbeing Advisory Group}, All Party Parliamentary Group on Wellbeing Economics; Federation for Small Business, Royal Society of Arts/Carnegie Trust Short Life Working Group on Measuring Job Quality, Chartered Institute of Personnel, Society of Occupational Medicine, S1).

Shaping New National Policy Frameworks in Key Government Departments.

The UEA team has helped make the case for policies to promote wellbeing through enhancing the quality of employment ( S3 DWP; S4 BEIS; S1 WWCW; research R1; R2; R3; R6) and informed BEIS and DWP’s own policy research ( S3; S4): ‘The Work and Learning team have helped shift policy debates around employment, wellbeing and mental health so that there is an increasing focus on the quality of jobs as determinants of wellbeing and health rather than treating employment/unemployment as a binary contributor to health and wellbeing’ ( S3).

These policy debates have been translated into action based upon recommendations of the research. For example, the UEA team’s findings have been referenced in answers from MPs and Lords in response to Parliamentary questions:

‘The What Works Centre for Wellbeing has found that being in a job is good for wellbeing, when measured by life satisfaction, but being in a good quality job is even better. We are creating fairer, inclusive and flexible workplaces so everyone has the chance to succeed as well as balance work and home life’ (Nadhim Zahawi MP, Parliamentary Under-Secretary, S2 p4 entry 1, relates to R1) and

‘The What Works Centre for Wellbeing has found that mindfulness training can have a positive impact on wellbeing’ and attached a link to the WWCW briefing report from R6 (Lord O'Shaughnessy, Department of Health and Social Care, S2, p4, entry 2).

Examples of significant impact include:

Stevenson-Farmer recommendations for workplace mental health ( S2, p4, entry 3). R1 informed WWCW “ advice to the Stevenson-Farmer Review” ( S1, S10). Specific reference to UEA research is made on p31, p49 and the features of good work (p69). The Prime Minister and Cabinet Secretary accepted the recommendations of Stevenson-Farmer for the Civil Service as an employer, as did the NHS. Office of National Statistics data indicates the recommendations apply to over 2.2 million workers employed in these institutions. ‘Civil service organisations and the NHS have agreed to follow the recommendations of Stevenson-Farmer in respect of the management of workplace mental health’ ( S1)

Influence on Department of Work and Pensions’ (DWP) new reporting standards and guidance ( S2, p5, entry 4), through influence on Stevenson-Farmer and WWCW recommendations ( S3; S1): ‘The work of the team has been influential on DWP recommendations for reporting standards for disability, mental health and wellbeing and our guidance for Disability Confident leaders’ ( S3). According to DWP statistics posted on DWP’s website, over 19000 workplaces have signed up for the Disability Confident Leaders scheme.

Evidencing the case for statutory guidance on why employer support for employees returning to work after sickness absence is important for reducing long-term sickness absence ( S3; R3 cited on p22 of DWP’s Health is Everyone’s Business consultation, S5).

Evidence-based Changes to Organisational Practice

The UEA team helped effect nationwide changes in police forces in England and Wales and contributed to the development of a new police wellbeing service, ‘Oscar Kilo’, through work on grant G2 ( S6, College of Policing). More specifically, findings from R4 led to the adoption of a mindfulness app across all forces: ‘As a result of the evaluation, one of the apps (MindFit Cop) has been made available to over 120,000 officers and 68,000 staff in England and Wales through the National Police Wellbeing Service. The process evaluation also informed amendments to the design of MindFit Cop and provided evidence to inform purchasing decisions of the other commercially available and well-established app Headspace. ‘We estimate savings of £1127 per year to policing for each person using the app’ (Knowledge Research and Practice Lead, College of Policing, S6)

The UEA team were involved in developing national guidelines for officers with line management responsibilities, that drew upon UEA expertise in review methods ( R5) and informed the College of Policing’s research underpinning the framework ( S6). The national guidelines include: ‘…for the first time, guidelines on supervisory practices that protect and enhance line reports’ wellbeing. The UEA team contributed their expertise in systematic review processes and roll-out of new guidelines during the review. The new guidelines will apply to around 115,000 first line supervisors in the police’ ( S6)

Daniels was the academic consultant for the WWCW wellbeing assessment tool, ensuring a wider set of indicators of employment practices were included than in traditional risk assessment approaches ( R1; R2). DWP, Department of Transport and Ministry of Defence (MoD) ( S1) have used the tool. The tool informed OFSTED’s research on teacher wellbeing. Page 11 of the Ofsted report ( S2, p5, entry 6) states: *‘We adopted the conceptual well-being framework from the What Works Centre for Wellbeing for this study. It informed our development of data collection instruments (see Appendix 4)*’. This assessment tool is also used in employer reporting standards: ‘DWP have incorporated a short-form of our workplace wellbeing indicators in the reporting standards for disability, mental health and wellbeing’ ( S1).

The WWCW briefing developed from R1 is included as a resource in NHS Employers’ guidance on developing wellbeing strategies: ‘ The What Works Centre for Wellbeing have summarised the range and quality of evidence demonstrating a link between high quality roles and better workforce wellbeing. Read the full report to find out more about the evidence relating to each of these elements of job design’ ( S2, p5, entry 5).

The UEA team have developed an evidence-based approach to working with a significant business unit in the MoD ( S7), advising on implementing and evaluating wellbeing related initiatives (drawing on R1; R4; R6), which included ‘launching wellbeing related research projects conducted by staff in Defence Intelligence … evaluating cost-effectiveness as part of our large-scale Mental Health First Aid Training Programme’ and; ‘identifying a suitable evaluation framework for our Mindfulness Programme and alignment with the methodology used by UEA in a high-profile randomised control trial at the College of Policing’ (Defence Intelligence Head of Wellbeing, Diversity and Inclusion, MoD, S7)

Influence on Standards and Professional Practice

Daniels is a member of the British Standards Institute (BSI) committee that developed the latest guidance on managing occupational health for International Standard 45001 ( S8). The UEA team’s research informed the standard’s definition of wellbeing (p2, R2) and elements related to management ( R1) and risks ( R1). The bibliography includes the WWCW website. The UEA team’s research is also cited in a BSI Publicly Available Specification ( S9, drawing on R1, p iii, R1, R6 p5, WWCW listed as a resource on p23) that provides guidance on evidence-informed actions on improving workplace health and wellbeing.

The growth of the Tesco Local range over the previous decade had afforded hundreds of small-scale food and drink producers the opportunity to access mainstream supermarket distribution. Tesco’s ‘Project Re-set’, launched in 2016, was the largest range rationalisation project in their history, seeking to reduce the number of products listed by 15% per annum over the following three years. This signalled a significant change in Tesco’s strategy, leaving small-scale producers vulnerable to de-listing if their products under-performed.

Against this backdrop, findings from the co-produced research in the WBMF project have directly led to improved performance by small-scale suppliers and improved their relationship with the retailer. The research has also informed the development of marketing support for small-scale suppliers in Northern Ireland.

Impact on Individual Small-Scale Food and Drink Suppliers

The primary beneficiaries are the 113 small-scale food and drink producers engaged in the WBMF project, which has reduced the risk of product de-listing and helped them achieve above-average sales growth during a period of unprecedented uncertainty.

Better resource allocation leading to above-average sales growth [3.1, 3.4, 3.6]: Over the three years following the launch of ‘Project Re-Set’ suppliers reported their total sales had grown by an average of 20% [ 5.1, Q.67], of which they attribute 18% [ 5.1, Q.68] to their involvement in the WBMF project. Moreover, during the twelve months from November 2019 to November 2020, sales grew by an average of 33%, compared to 6% for the relevant product categories [ 5.2]. Much of this success is due to the ability of businesses to adapt to the changing market environment and use their scarce marketing resources more effectively, which the WBMF project has made possible through the provision of customised market intelligence which small-scale suppliers would otherwise be denied:

“Who Buys My Food has become such a valuable resource enabling us to punch above our weight as a small supplier. In this highly dynamic and fast moving market it is essential to have the data and insights you are providing, as we can’t effectively plan and manage what we don’t know.” [Cleone Foods 5.6]

“We can now monitor our performance and see who our customers are and where and when they are purchasing our products. This helps us to tailor our promotions, positioning and sampling events to the needs of our customer – something we would struggle to do otherwise." [Thistly Cross Cider 5.3]

“We had confidence in the strength of our brand but the shopper insight report was a real wake-up call and prompted us to be more much more pro-active in engaging our core shoppers. The in-store sampling not only lifted out penetration rates but brought us into direct contact with our consumers, which generated lots of useful feedback.” [Doherty’s Meats 5.4]

Engagement with the WBMF data visualisation tool leading to improved performance [3.5]: WBMF research informed a re-design of the web data visualisation tool in 2020. This tool provides on-line access from any device at any time and a more differentiated approach to data visualisation, making it efficient for a user to find key data quickly. Metrics from the use of the application show a high frequency of short-period user access with significant benefits at the level of the individual decision-maker. For example, 51% of users reported an increase in their productivity, 57% reported an increase in their effectiveness and 62% reported an increase in their performance [ 5.1, Q.47] and that of their organisation:

“The web-application made it possible for us to access critical pieces of information when we needed it most (during the lockdown) and share it with the buyer with a level of detail that made it easier for us to make our case for launching a bigger bottle in the right stores at the right price, which was beneficial to Tesco as well as our business.” [Nix & Kix 5.6]

Improved functional capability amongst account managers [3.2, 3.4]: When asked how their involvement in the WBMF project had affected their individual (functional) performance, 66% of respondents reported a significant improvement in their sales management, 69% reported a significant improvement in their account management, 57% reported a significant improvement in their marketing management and 57% reported an improvement in their use of market intelligence [ 5.1, Q.10]. In terms of specific marketing areas 74% reported they were “better equipped for meetings with the Tesco buyer”, 67% reported they were “ better able to plan promotions”, 53% found the research findings made a “ significant contribution to new product development” and 42% found them helpful with respect to the “ design of new packaging”. [ 5.1, Q.41]. Moreover, 71% reported the project had enabled them to learn “the value of shopper insight” and (63%) “the value of evidence-based decision-making”, whilst 60% reported that the project had made a “ significant contribution to the development of our relationship with Tesco.” [ 5.1, Q.76]

Impact on the Supplier-Retailer Relationship

Reduced the risk of product de-listing [3.1, 3.3]: During ‘Project Re-Set’, survival was the primary goal for small-scale suppliers. The WBMF project helped suppliers to identify appropriate action to improve two of the key performance metrics, customer penetration and repeat purchase rate, that Tesco used to identify candidates for de-listing.

In February 2020, 86% of respondents to the supplier survey reported a significant increase in their market knowledge, 76% reported a significant improvement in their relationship with the Tesco buyer and 63% had found the project invaluable in helping them to avoid product de-listing [ 5.1, Q.43].

One month later, the Covid-19 pandemic resulted in a national lockdown with devastating impacts on a large number of suppliers. The threat of de-listing was intensified due to the pressure on supermarket supply chains and shelf-space for essential items, and the WBMF project sought to provide further assistance. WBMF demonstrated that suppliers who invested in their relationship with Tesco, by developing mutually beneficial proposals informed by customised market intelligence, were more likely to develop lasting relationships and perform better with Tesco.

“We needed to understand how demand was changing as a result [of] the pandemic and adapt our business accordingly… the insights we gained from the WBMF project were invaluable… you guys absolutely rock and all the tools were super useful!” [Nix&Kix 5.6].

Adoption of regional overlays for more effective ranging of local lines in Tesco [3.6]: A key message of WBMF to suppliers during ‘Project Re-Set’ was to resist the temptation of increased distribution and to focus instead on performing better in fewer stores – those where latent demand was strongest but not yet fully exploited. This would ensure that customer penetration and repeat purchase rate would increase, reducing the threat of de-listing.

WBMF also highlighted to the Tesco Local team a flaw in Tesco’s approach to the ranging of local lines. The existing process for introducing a local line involved the identification of an under-performing product which could be replaced by the local variant. The problem was that the local line was automatically allocated to the stores in which the failing product was listed. In most cases this extended beyond the region in which the product was produced, resulting in a store allocation that was not aligned with consumer demand. The person responsible for ranging local lines in Tesco then set to work on the changes necessary. Six months later regional ‘overlays’ were introduced, enabling buyers to range local lines within the county in which they were produced:

“the messages (you) were giving to suppliers at the regional supplier events regarding the pursuit of increased distribution of their products (‘be careful what you wish for’) contributed towards the changes that Tesco made in the ranging of local lines, to enable a more targeted store allocation with county/regional overlays” [ 5.7].

This change made it possible for suppliers to remove the ‘long tail’ of stores in which their sales were negligible, which were pulling down their (average) performance measures, and focus on their key customer and their core stores, thereby reducing the threat of de-listing and increase the return on their more targeted allocation of marketing resources.

Impact at Sector Level

Transformation of public-funded marketing support for the growth and development of food businesses in Northern Ireland [3.2, 3.4]: Agri-food is the largest industry in Northern Ireland, and in 2015 InvestNI, the business development agency for Northern Ireland, was charged with delivering a strategic action plan with the aim of “g rowing a sustainable, profitable and integrated agri-food supply chain, focused on delivering the needs of the market”. One strategic investment was the creation of the consumer insights team. WBMF was engaged to provide customised market intelligence for small-scale producers supplying Tesco, by far the largest supermarket in Northern Ireland and in the process provided a catalyst for change in InvestNI from “hands off” to “hands on”:

“The research that Professor Fearne and his team have undertaken on the benefits of evidence-based decision making and the work we have done together (webinars, workshops and case studies) to develop the capability within our clients has brought us much closer to the companies with whom we work. This has enabled us to provide more ‘hands on’ support, which Andrew’s research revealed was essential if we wanted to turn insight into action and achieve a greater return on the significant investment we make annually on market intelligence.” [ 5.8].

In 2020, InvestNI undertook an evaluation of the impact of the WBMF project [ 5.9] which:

“revealed that 94% of the businesses involved were “extremely satisfied” with the project. Many of them reported growth in revenue, improved relationships with their retail customers and an acknowledgement that they were better informed and more effective in targeting their marketing resources” [ 5.8]

This resulted in an additional GBP500,000 in sales from new product listings in Northern Ireland and GBP2,300,000 from new product listings in GB [ 5.9 page 4], thereby:

“supporting the business case for investment of public funds into the provision of consumer insight to the agri-food sector in Northern Ireland” [ 5.8].

Competition laws and competition authorities are established with the goal of promoting competition for the benefit of consumers. Having a clear and effective framework for evaluating key public policy interventions in pursuit of well-functioning economies is central to the well-being of society in general. Although UK competition authorities have always been in the vanguard of such evaluations internationally, up until around 10 years ago, these were only done regularly in the largest economies. Moreover, past evaluations sidestepped some important considerations, due to lack of evidence and a clear methodological and analytical framework. This resulted in evaluations that missed potentially the most important function of competition laws and policies: their deterrent effect.

1. Driving the creation of a methodological toolkit to raise the awareness of competition policy evaluations

The research of Ormosi and collaborators has contributed to changing the work priorities of several authorities, as awareness of the importance of competition policy impact evaluation has grown and more competition authorities began to undertake such evaluations.

I.a Driving the creation of the OECD toolkit

The OECD Competition Division, with its 37 member countries, is one of the largest international stakeholders in the area of competition policy. UEA research [3.1] was instrumental in the OECD Competition Division’s decision to direct a quarter of their available resources to start a three-year strategic project to assemble a best practice guide for member and non-member countries. Previously no resources were used for impact evaluation. As [redacted text] (Head of OECD Competition Division at the time of the project [redacted text redacted text]) writes,

“... *I discussed with the Committee a changed approach, under which we would focus our work for the next three years at least on no more than two ‘strategic themes’. To obtain this agreement, I needed to motivate the Committee and to provide some ideas for outputs, and a project plan. One of the areas we thought to develop was measuring the impact of competition policy. I was lucky just at that time to read Davies and Ormosi’s working paper “The Impact of Competition Policy: What are the Known Unknowns?” – which as its title suggests was perfectly suited to our needs. […] As a result of this work, the Committee endorsed our proposal to devote about one quarter of its time to this topic, as one of its two strategic projects over 2012-2015.*” ([5.1], referring to working paper versions of outputs [3.3] and [3.4]).

As the central part of this strategic project, the OECD developed a Reference Guide on evaluating the impact of competition enforcement [5.2]. The Guide references [3.1, 3.5, 3.6] in 20 places. This OECD Reference Guide provides competition authorities around the world with a best practice toolkit to evaluate the impact of competition policy, and many member and non-member countries choose to follow these guides to create their own guidelines. Anticipating this, [redacted text] concludes in his testimonial that:

“*I therefore want to put on the record that I greatly appreciate the work that Drs Davies and Ormosi have done in this area. It has already significantly affected the work of the OECD in the field of competition policy, and through our work will also certainly have an effect on policies and therefore economic outcomes in OECD countries and beyond.*” [5.1].

The influence of this research on the OECD has continued with 7 references in a 2020 publication [5.3] by the OECD Committee on Consumer Policy.

I.b Impacting OECD member states’ impact evaluation guidelines

The inclusion of UEA research in the OECD Reference Guide provided a channel which brought the work to the attention of competition authorities which were at the early stages of setting up their impact evaluation systems. The Spanish competition authority (Comisión Nacional de los Mercados y la Competencia – CNMC) used this to underpin their own impact evaluation guidelines [5.4]. [redacted text redacted text] (Head of Cabinet and Chief Economic Advisor of the President) writes,

“We have found the works of Professor Stephen Davies and Dr Peter Ormosi […] very useful in constructing these guidelines. […] *we have relied on their recommendation to use moving average estimate of consumer benefits to smooth out annual fluctuations between years with exceptionally big and small consumer savings; to justify the need to conduct the evaluations after the activity, but using ex ante information; to discuss the role of selection bias; or to identify the types of conservative benchmarks to use when calculating relevant turnover. […] Having clear guidelines to estimate the consumer benefits of what we do has been particularly important in order to demonstrate that the public resources dedicated to the CNMC lead to consumer benefits that are several times larger...*” [5.5].

Similarly, the Hungarian Competition Authority (HCA) had had limited experience in impact evaluations, having only first designed their impact evaluation guidelines in 2013. Informed by the OECD recommendations incorporating UEA research, they have updated their guidelines, drawing on [3.1, 3.5, and 3.6] (22 references). [redacted text] (Chief Economist of the HCA) wrote that Ormosi and Davies’ contributions ‘ have played a pivotal role in the success of our work in this area in a number of ways’ [5.6].

This line of research has led to conceptual changes in how competition authorities approach evaluation of their policies. Following Ormosi’s presentation at an OECD event in 2015 [3.5], [redacted text] Principal Adviser at the European Commission, noted,

“Dr Ormosi[‘s] *role was to draw lessons from ex-post evaluations presented by national competition authorities. He made a number of interesting suggestions on how to conduct proper ex-post evaluations. This would contribute to promote the work on ex-post economic evaluations to the European competition authorities.*” [5.7]

Following on from the OECD Reference Guide, the World Bank in 2014 invited Ormosi and Davies to develop an impact evaluation framework for the Competition Authority of Kenya. The World Bank wrote,

“As a result of Dr, Ormosi and Dr. Davies work, the CAK [Competition Authority of Kenya] *is on course to adopt and systematically use the designed frameworks.*” [5.8]

Finally, between 2017 and 2020 Ormosi was repeatedly called upon by the Competition and Markets Authority (CMA) (and its predecessor the OFT) to provide academic guidance to constructing their annual impact evaluation reports. In this capacity, Ormosi repeatedly drew on the above listed research. As [redacted text] testifies:

“... *Ormosi has repeatedly reviewed the reports since. I understand that in their detailed reviews they frequently draw on the extensive research they have done in the area of competition policy evaluation. It is worth noting that the original guidelines were used as a model for such guidelines by the OECD and, hence, by a wide variety of national competition authorities.*” [5.9]

1. Providing evidence to include estimates of the deterrent effect in competition policy evaluations

Past evaluations sidestepped some important considerations, due to lack of evidence and a clear methodological and analytical framework. This resulted in evaluations that missed potentially the most important function of competition laws and policies: their deterrent effect. William Kovacic, former Chair of the U.S. Federal Trade Commission highlighted at the October 2013 OECD Conference: “ *No modern development in antitrust law is more striking than the global acceptance of a norm that condemns cartels as the market’s most dangerous competitive vice [but] is modern cartel enforcement attaining its deterrence goals?*”

If competition laws work well, and unlawful conduct is deterred, public resources are saved and harm on the economy and society is reduced. UEA research on cartel detection and deterrence [3.2, 3.3, 3.4] led competition authorities to re-assess the role of deterrence in setting and evaluating policy. As [redacted text] Chief Economist of the CMA, wrote, these works were key in producing a CMA report on deterrence [5.10] and changed CMA priorities, through the understanding of the value of prosecuting relatively small cartel cases, which have small direct effects but potentially large deterrent effects.

“In this respect we have found the recent work by Davies and Ormosi into measuring deterrent effects extremely useful and timely […] it helps remind us not to deprioritise smaller enforcement cases (by demonstrating the important deterrent effect that these cases have, such as our estate agent cartel cases). It substantiates the belief that there is a large deterrent effect of cartel policy and that the positive impact figures for cartels that are detected are a significant 3.3 under-estimate of the overall impact of all our cartel work. In turn, this ensures that we do not concentrate proportionately too many of our resources on merger control and market investigations.” [5.9].

Ormosi’s work helped formulate impact evaluation priorities in the European Commission’s Directorate-General for Competition (DG COMP) as well. Based on UEA’s research on deterrence [3.3, 3.4], the European Commission’s DG COMP and the CMA organised a workshop on how competition authorities could measure deterrence. [redacted text] (DG COMP) wrote,

“Dr Ormosi also highlighted the importance of the deterrent effects of competition policy […] The Competition Directorate General has followed up on these various suggestions by organising in collaboration with the Dutch and UK competition authorities a... conference on “Looking beyond the direct effects of the work of competition authorities: Deterrence and macroeconomic impact” [5.7].

At a time of internal change, the then Secretary General, [redacted text], with the endorsement of Commission President [redacted text], had authorised the research reported in R1, and used the findings to inform her reflections concerning the future development of the administration. As a result, the team gained the recognition and credibility that made possible the subsequent Commission projects and opened the door to the Council Secretariat. Connolly and Kassim co-led the projects, and Kassim often led on liaison. While always maintaining academic independence, the team has worked closely with senior officeholders, to whom it has presented early results, in advance of the wider dissemination of overall findings and invitations to address other departments, groups or networks. This relationship was a key pathway to impact, as it created opportunities to communicate findings and hold exchanges on the research. For instance, in 2014-15 at the invitation of top leaders, Connolly and Kassim made 14 invited presentations, including meetings with the Vice President for Administration, the Secretary General, the Director General for Human Resources and her senior management team to brief them on the findings from the 2014 project. The Vice President for Administration spoke at the launch of the report (R2) which was live streamed across the Commission. In the Council Secretariat in 2016, Connolly and Kassim made presentations to the Secretary General and his private office. They also made a presentation of the findings to a meeting open to all staff which was attended by 300 personnel and viewed by a further 500. In 2018-19, Connolly and Kassim made 22 invited presentations to audiences in the European Commission including the Commissioner for Administration and his private office [S1], the weekly meeting of heads of the private offices of the Commissioners, the Secretary General, the Director General of the HR department, top managers in the Secretariat General and in other departments, the Directors General awayday, and the lunchtime seminar, which is open to all staff. Their work was reported in the staff magazine and the staff intranet carried findings from the project. At the request of the then Secretary General, [redacted text], the team prepared a series of special briefings for President-elect, [redacted text], which were among the materials communicated to her on her nomination [S2, S10].

Research conducted by Connolly and Kassim has had three main impacts.

1. Restructuring of the European Commission, 2014: Analyses of political leadership in the Commission, especially the power of the Commission Presidency, the working of the College, and relations between the College and the Commission departments under the two [redated text] Commissions, influenced the historic reform of the College enacted by President [redacted text], where seven Vice Presidents were given leading roles in upstream coordination. The remodelling was designed to ensure early political policy coordination, strengthen political leadership over the administration, and thereby ensure the effective delivery of the President’s programme. UEA research (R1 and R2), research briefings and slides reporting findings from the 2014 project, was used by [redacted text redacted text redacted text redacted text redacted text redacted text redacted text redacted text redacted text redacted text redacted text].

As President [redacted text] recalls [S3]: “ As I prepared to take office, among the compulsory reading suggested to me by the Secretary-General of the Commission and my Head of Cabinet at the time, were briefings on the European Commission prepared on the basis of the first two research projects [(R1) and (R2)] …. I found their work on leadership and coordination within the Institution particularly useful in my reflections on how to organise and focus the work of the next College of Commissioners”. [redacted text] [S2] commented similarly: “ *I found Professor Kassim’s work on leadership and coordination within the institution particularly useful in my reflections on how to advise President Juncker to organise and focus the work on the College of Commissioners. His research provided both some of the justifications and the guidance needed for the (re-)structuring of the Commission by President Juncker in 2014.*”

2. Modernising the Council Secretariat, 2016-20: UEA research (R3) informed the reform programme initiated by the Secretary General, [redacted text], in 2016 [S4]. In its analysis of the “ understanding of staff's own perceptions of the organisation” [S5, DG for HR], the Kassim report (R3) played an important part in setting the context for the measures. In presentations outlining the reform, the Secretary General cited data from the report (R3) as evidence of where action was necessary and the direction of travel supported by staff. As well as showing that staff in general strongly favoured modernisation, it showed particular support for measures to improve coordination and information flows, delayer the hierarchy, and personnel policy reforms [S4, Secretary General], “ to empower staff more, to use participatory leadership techniques … [T]he study also identified the lack of a common management culture which we have been working to improve since the study” [S5, DG for HR].

This research led specifically to a reform of the staff appraisal system, and a review of competencies and skill for each main staff group that led to new guidance for managers. As the Head of Staff Development notes [S6]: “ *The Kassim Connolly study formed the basis of my team's work to support the review of management role descriptors, staff role descriptors, as well as to work on promoting a common management culture, a new policy on staff development and clearer communication on career paths”.* It also led to changes in the Council Secretariat’s Translation Service (LING). “ Following the insights generated by this research … LING [management] has introduced certain measures to overcome the shortcomings identified by staff by providing opportunities for staff to break silos and reduce LING's isolation (1), to accommodate the need for more and better communication (2) and to provide enhanced mobility options (3)”. [S7, Head of Unit and Director of Translation Services]

More broadly, the research, which was the first project to be conducted by an external team, initiated a cultural change within the Secretariat. As well as encouraging the organisation to be more outward looking, it highlighted the value of listening to the views of staff. Following the 2016 study, an annual staff survey was introduced in the Council Secretariat for the first time “it helped build trust …staff became much more aware of the issues that faced them and the fact that they could influence them” [S8, Office of the Secretary General].

The findings of the research were circulated internally, discussed by senior staff, and presented to a meeting of all staff, attended by 300 and watched by a further 500 via video streaming, in January 2017. This was considered key in developing organisational trust “ Staff confidence in the system was as a result very high, helped by the reputation of the study team for the work they had done previously in the Commission. The openness of the exercise, with the study team presenting their findings in a clear and extensive written report and in an open meeting with staff was an important success factor” [S5, DG for HR].

3. Briefing the von der Leyen European Commission, 2019-20: The 2018 project has influenced the von der Leyen Commission in two ways. First, as the Juncker Commission approached the end of its term in 2019, Connolly and Kassim presented their analyses of the organisational and procedural changes introduced in 2014, and their conclusions concerning possible options for the incoming Commission, to the Secretary General, Directors General, the heads of the Commissioners’ private offices, and the European Political Strategy Centre. Also, as [redacted text], [S2, S10] noted: “ I have included Professor Kassim’s work in the briefing materials delivered to Ms von der Leyen”. Outgoing Commission President [S3, S10] confirmed that: “ As was done for me, I will recommend the briefs from this new project [The European Commission: Where now? Where next?] as compulsory reading to my successor. They offer unique insights on the internal organisation of the Commission and its evolution over time – an indispensable asset when deciding how best to prepare the Institution for the challenges ahead”.

Second, UEA research on management and personnel policy over three Commissions has informed planning for the new HR strategy initiated by the von der Leyen Commission in 2020. Following the announcement by the new Commissioner and Director General for Human Resources of a major consultation exercise to inform HR reforms, Connolly and Kassim were invited to deliver three two-hour Masterclasses to members of Cabinet, HR leadership and managers on: staff recruitment and career pathways; the Commission as a workforce; and organisational change [R1, 2, 4, 5 & 6, S10]. The Masterclasses, which were delivered virtually in October and November 2020 and attended by more than 700 people, presented findings and analysis from all three projects. Thanking Connolly and Kassim “ for the three excellent sessions”, the Principal Advisor, Professionalisation and Orientation, Directorate General Human Resources and Security, noted that: “ These events were very well attended, very much appreciated, and extremely useful in terms of reminding staff of the history and progress on people issues in the Commission. Also very useful for our Cabinet colleagues in getting up to speed on these issues. And very useful for me in our work on the HR Strategy, particularly in terms of the consistency of messages across our own surveys, external benchmarks, and your research” [S9].

As external researchers, Connolly and Kassim have been able to pose questions to staff and investigate issues, such as the views of employees on official policies or their beliefs and values, that management cannot. Because the research draws so extensively on the insights and experience of insiders, the findings have authority and authenticity that senior officeholders recognise and value, and are valued for their quality and detail, as well as for their independence and objectivity. In the words of a former Commission President: “ [t]he insights they provide in their work is unrivalled. Even the most well-informed European think-tanks and top academic literature cannot match the team's depth of understanding about the Commission” [S2]. In the case of the Commission, the longitudinal analysis that repeat studies have made it possible to deliver has been especially important [S9]. The analyses have contributed to organisational development in the two bodies, and helped bring about a cultural change in Council Secretariat. In the words of one official, “ You deserve recognition for the excellent work done. Your study was a game changer in the GSC” [S6, Head of the Staff Development Unit].

After households in Great Britain were given the ability to choose their energy supplier from 1999, there were growing public and political concerns that companies were charging excessive prices because of weak competition, especially to inactive consumers who did not ‘shop around’ for a better deal. From 2009 Ofgem introduced several measures to constrain the prices companies could charge, and to encourage households to identify and switch to cheaper offers. These included non-discrimination clauses which prevented companies from imposing higher mark-ups in their home regions, where they had an inherited base of consumers, than in other regions where they were competing to attract customers from other suppliers. While the non-discrimination clauses were not renewed after their initial three-year term, the regulator introduced further measures in 2013. The ‘four tariff rule’ aimed to simplify price comparisons and stimulate consumer activity by restricting the number of tariffs which each company could offer in each region. However, switching rates remained low and public concern about exploitation, particularly by the large companies with an inherited base of consumers, persisted, and was reflected in party political proposals for intervention in the market. These concerns culminated in a major competition review of the energy market by the Competition and Markets Authority (CMA) between 2014 and 2016. CCP brought its research findings to the attention of the CMA through eight written responses to CMA consultations and an extensive oral hearing with the relevant group and staff.

1. CMA finding of weak consumer response and consumer detriment

Informed by CCP’s research, the CMA developed a new understanding of the important role that the presence of inactive consumers played in the dynamics of the market. The Chair of the CMA Energy Market Investigation group, wrote, “ *The CCP’s work was particularly valuable in two areas. The first was around how consumers chose their energy suppliers. The CMA’s historical view was that consumers reacted quickly to prices and would seek out the cheapest supplier. The CCP’s work both demonstrated that this was not the case in energy markets and also offered key insights on the reasons why it was not.*” (S1). This new conceptual understanding led to one of the main findings of the CMA review: weak consumer response which significantly inhibited competition in household energy markets. Several CMA publications (S2, S3, S4) cite CCP research extensively, and the CMA invited CCP “ to an oral hearing in front of the whole inquiry group - something we did for only one other academic group” (S1). The CMA estimated that energy consumers were each forgoing up to GBP300 a year saving on their energy bills because of their inertia, a total detriment of GBP14,000,000,000 a year. Design of the CMA’s extensive consumer survey, which has become an important source of information for subsequent analysis and policy development, was underpinned by CCP research. The chair of the CMA Energy Market Investigation group reported that the CCP work : “…provided the basis for our own very extensive consumer survey – probably the biggest ever carried out – and CCP’s advice was critical in the design of that survey….” (S1).

1. Repeal of regulatory interventions, in particular the four tariff rule

The CMA used CCP evidence in identifying the harmful effects of two Ofgem interventions. Research by Waddams Price and Zhu (R1) was cited in the CMA updated issues statement (S2) as evidence on the detrimental effects on competition arising from non-discrimination clauses such as the ‘four tariff rule’. The chair of the CMA Energy Market Investigation group wrote, “ The CCP’s second, and less-expected, contribution was around the dangers of ill-judged regulatory interventions. This led us to recommend, and Ofgem to implement, the removal of a number of Ofgem regulations, notably the ‘four tariff rule’.” (S1). The CMA found that this four-tariff rule had led to companies withdrawing low price offers, particularly innovative tariffs which were beneficial to over 200,000 low-income householders over 60 years of age, and to consumers of small amounts of energy who were more likely than average to have low incomes. Removal of the regulatory constraint enabled these households to benefit, as well as strengthening general competition pressures which would benefit consumers overall (S4).

1. Design of the price cap for inactive consumers

Following the CMA report, the government decided to adopt the minority recommendation of capping the prices which energy suppliers could charge to consumers who had not recently engaged in the energy market, even though the main CMA group had concluded against such a move. Imposition of the cap required primary legislation, and CCP was “ involved throughout this debate with Parliament” (Ofgem CEO, S5). CCP submitted an extensive response to the pre-legislative consultation and Waddams was invited to present oral evidence. One apparently attractive proposition was to link the prices charged by companies to loyal consumers to the lower prices which they offered to more active households. However CCP research had demonstrated that the earlier ‘non-discrimination’ clauses, which had similarly linked the prices charged to less active consumers in companies’ home areas to those offered in other regions where they were recruiting new customers, had hampered competition and led to higher prices (R5, R6). A relative cap was rejected and CCP’s evidence on previous regulatory interventions (based on outputs R5 and R6 which had formed the basis of this part of CCP’s consultation response) was widely cited both in the report of the pre-legislative committee (S6) and in a House of Commons briefing on the energy market (S7) (the page/paragraph numbers are given with the sources in section 5). Ofgem’s CEO states that Ofgem estimated that “ …its immediate impact would be to save customers up to £1bn [GBP1,000,000,000] a year, through reducing the degree of market power enjoyed by the bigger firms as well as forcing them to rapidly become more efficient,” (S5).

CCP’s impact continues in the regulator’s considerations of whether and when to lift the price cap for consumers who do not switch suppliers, as required by Parliament. Ofgem’s framework reflects CCP’s contribution to the debate around the imposition of the cap, in particular how effective competition is to be interpreted in a market for an essential service with low consumer engagement (CCP’s definition of effective competition, in particular with respect to price discrimination (R5), is extensively cited in the framework, S8).

1. Wider Impact of the Research

The CEO of Ofgem at the time of the CMA investigation and the introduction of the price cap, noted that “ The CCP research was also influential in bringing the ‘loyalty premium’ into wider public discourse, and was particularly effective in introducing the concept of fairness into economic regulation”, stating that “ Previously, very few if any economic regulators would have used such terminology or studied the distributional effects of regulatory actions... but the patterns of consumer behaviour and market outcomes studied by the CCP aided us in thinking more coherently about these matters” (S5). The fundamental challenge by CCP research to the “ conventional theory of market competition and consumer engagement” (S5) extends both internationally and across sectors. For example, in Australia, the Victorian Essential Services Commission based the review of its own default tariff in 2019 on CCP research (S9). Beyond energy markets, CCP research influenced the super-complaint across markets in the UK by the consumer organisation Citizens Advice (S10). The CEO of Ofgem wrote further, “ The overall impact of the research was considerable, and not just in the energy market. It represented a questioning of the conventional theory of market competition and consumer engagement that is taught to all students of microeconomics… [T]he evidence was… compelling… Moreover the loyalty premium research has now impacted other markets such as telecoms and financial services, particularly insurance, where similar questions…are being asked” (S5). The Chair of the CMA Energy Market Investigation group stated that “ What was once heresy is now close to being dogma, and for that the CCP’s work has been a key driver” (S1).