Results and submissions : REF 2021

A national action plan for tackling the undeclared economy in Greece

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Societal
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

In August 2015, the Greek government agreed with the European Commission five conditions for receiving their bailout money. One of these was to develop a national action plan for tackling their undeclared economy. Based on his policy-focused research on undeclared work in South-East Europe, Professor Williams was appointed to lead the mission to Greece to produce this national action plan, funded by the European Commission. Following extensive consultation by Professor Williams with the Greek Government, Bank of Greece, trade unions and employer organisations, in July 2016, the national action plan was accepted and validated. The Greek authorities have subsequently implemented nearly all its recommendations.

2. Underpinning research

For many decades, undeclared work was considered exploitative waged employment conducted under ‘sweatshop-like’ conditions and governments adopted an eradication approach. Professor Williams has undertaken research that has led to a more balanced view of undeclared work (i.e., paid activities not declared to the authorities for the purpose of evading tax and social security contributions and/or labour laws). He has revealed that much undeclared work is conducted on a self-employed basis, such as by entrepreneurs starting a business who often test-trade its viability in the undeclared economy. Consequently, Williams’ research has shown that a policy shift is required away from eradication and towards transforming undeclared work into declared work.

Conventionally, participation in undeclared work was explained as a rational economic decision taken when the benefits outweigh the expected cost of being caught and punished. The policy solution was therefore to increase the costs of undeclared work by raising the actual or perceived penalties and probability of detection. However, Professor Williams has highlighted that there is also a need to improve the benefits and ease of operating legitimately if undeclared work is to be transformed into declared work rather than eradicated [R1, R2].

Nevertheless, even when the benefits outweigh the costs, and the rational economic decision would be to participate in undeclared work, many do not. To explain this, Professor Williams has developed an alternative social actor approach [R3]. Grounded in institutional theory, this views undeclared work as resulting when norms, values, and beliefs (i.e. civic morale) do not align with laws and regulations (i.e. state morale). The solution is therefore to alter norms, values, and beliefs (informal institutions) and also laws and regulations (formal institutions) so that they align [R4].

His research has revealed that these are complementary rather than competing approaches. Consequently, governments need to change not only the cost/benefit calculation, but also the symmetry between norms, values and beliefs, and laws and regulations, and that this is the most effective means of tackling undeclared work [R4, R5].

Professor Williams has been conducting research on South-East Europe since 2006, publishing 34 journal articles on tackling undeclared work in South-East European countries, including Greece. In addition, Williams was PI, in partnership with consultancy companies in Bulgaria (Vitosha Research) and Croatia (Institute of Public Finance), on a €1.25m Marie Curie Industry-Academia Partnerships and Pathways grant (2013-2017), developing the capacities and capabilities for tackling undeclared work in South-East European countries [R6]. This expertise and experience in South-East Europe, coupled with a keynote speech at the High-level Conference for EU-28 Ministers of Labour in Dubrovnik in 2015, on tackling undeclared work in South-East Europe [R6], resulted in his invitation to lead the mission to Greece.

4. Details of the impact

Tackling undeclared work as a condition for Greece receiving its bailout money

In August 2015, the Greek authorities and European Commission agreed a three-year European Stability Mechanism (ESM) support programme. This set out five conditions for Greece receiving financial assistance, one of which was that “ the authorities will adopt an integrated action plan to fight undeclared and under-declared work” [S1].

To implement this, the European Commission appointed a United Nations (UN) body, the International Labour Organisation and in September 2015, Professor Williams was commissioned by the ILO to lead the mission to Greece to develop this action plan.

Prior to his arrival in Greece, the Greek employer organisations held four national-level workshops, and the trade unions held two workshops to diagnose the problem and recommend policy measures. These reports were presented to Professor Williams on his arrival in April 2016. During April, Professor Williams (accompanied by two national experts and with counsel from a Greek supreme court lawyer), met all relevant Ministers and Ministries of the Greek government, employer organisations, trade unions and the Bank of Greece. During May 2016, he then produced the diagnostic report and national action plan based on his above research [R3, R4, R5] about the most effective policy approach [S2].

In July 2016, at a validation meeting, this action plan was agreed by the Greek government, social partners, and European Commission [S3]. To oversee its implementation, and as recommended by Professor Williams [S2 Recommendation 1], a tripartite body, the Supreme Labour Council (ASE), was established by Law 4468/2017. From September 2017 to December 2018, the Greek authorities then implemented the following 14 recommendations of Professor Williams:

Increasing the costs of undeclared work

Law 4554/2018 introduced a new fines system to transform undeclared work into declared work, reducing the €10,500 fine for an employer detected with an undeclared worker to €3,000 if they declared them employed for one year [S2 Recommendation 13]. This has resulted in 98.4% of undeclared employees detected now being legitimately employed for at least one year compared with only 50.2% before [S4].
A real-time employment database (Ergani) was developed [S2, Recommendation 2]. A pilot project in 2017 using Ergani generated fines totalling €4.2m.
An approach of strategic targeted inspections [S2 Recommendation 3], adopted in 2018, led to 587,383 employees being interviewed, which determined 34,663 were undeclared, leading to fines totalling €364.8m [S5].
To reduce cash usage in the Greek economy [S2 Recommendation 14], it became mandatory from 01/06/2017 for employers to pay employees’ wages into a bank account (Ministerial Decision no. 22528/430/2017).
The maximum cash transaction in Greece was limited to €500 [S2 Recommendation 14].
Under article 9 of Law 4554/2018, contractors became liable for a subcontractor’s employees in relation to tax, social insurance and labour law violations [S2 Recommendation 6].

Improving the benefits of declared work

Under Law 4488/2017, only businesses without labour law violations can bid for public procurement contracts [see S2 Recommendation 7].
To encourage electronic payments [S2 Recommendation 14], citizens spending over 80% of their income by electronic transactions received an additional tax-free income allowance.
Law 4446/2016 made mandatory point-of-sale (POS) terminals in all businesses [see S2 Recommendation 14].
A receipts lottery was introduced in November 2017 [S6], encouraging customers to request receipts [S2 Recommendation 14]. Held every two weeks, 1,000 winners receive €1,000 each (€1m per lottery), with €23m paid out in the first year. 4.5m taxpayers participated in the first lottery and in the first 11 lotteries, 11.3 trillion receipts were submitted totalling €22.4 trillion. In the first 9 months, a 20% increase in VAT revenue resulted (an extra €2 billion) compared with the corresponding period of 2015 [S7].
To make declared work easier, a simple SMS electronic declaration for overtime was implemented by Ministerial Decision (Protocol Number 32143/Δ1.11288 / 11-6-2018) [S2 Recommendation 8].

Aligning norms, values and beliefs with the formal laws and regulations

A pilot campaign on the benefits of declared work, targeting schoolchildren, was implemented in 20 schools in Attica. 1,000 t-shirts, pens and stickers with the logo “Decent Work #Our Right” were distributed [S2 Recommendation 20].
1,000 hard copies of Professor Williams’ report [S2] were distributed to all labour inspectors and officials in other Ministries.
A 90-second youtube video on the benefits of declared work has had over 784,000 views [S2 Recommendation 17].

Further strengthening the impacts

These results were achieved in the 3-year ESM period until December 2018. To implement the remaining policy recommendations in the 2016 report [S2], Professor Williams has since undertaken three additional European Commission missions:

From December 2018 until May 2019, the European Commission Structural Reform Support Services Unit (now DG Reform) commissioned Professor Williams to design and implement a pilot study on training inspectors [S2, Recommendation 4] to use data mining [S2, Recommendation 2] to better target their inspections [S2, Recommendation 3]. The outcome in the four pilot regions was that the total number of registered working hours for employees across these four Greek regions increased by 35.7% and, to take another example of its impact, registered full-time employment in the restaurants and cafes industry increased by 70% [S8].
In December 2019, Professor Williams led a European Commission mutual assistance project to Greece to provide support on implementing the remaining policy recommendations in the 2016 report. All the remaining recommendations were included in the resultant report, and actions were agreed by the Greek authorities [S9].
In September 2020, a follow-up visit was undertaken to assess progress and offer further advice and support [S9]. The finding was that progress had been made on implementing all of the remaining recommendations from the 2016 report.

Achieving ethical human resource and procurement practices and transparency through ...

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Societal
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

The Supply Chain Accounting and Employment Practices (SCA-EMP) study showed that firms in emerging economies knew little about the labour standards or HR practices of their suppliers or how to improve them. The SCA-Emp team at Sheffield developed a diagnostic toolkit, co-produced with practitioners, and implemented by employer bodies and organisations in the UK, Brazil, South Africa, and Ethiopia to self-assess HR, accounting, and procurement practices. The use of the toolkit led to increased transparency and knowledge transfer across supply chains. This knowledge resulted in improvements in competitiveness, productivity, and profits, and most importantly, better labour standards for companies and their suppliers. Since its launch, the toolkit has been used by over 450 companies in 80 countries generating substantial ethical, economic, commercial, and organisational impacts.

2. Underpinning research

The ESRC-funded research investigated supply chain accounting and employment practices in automotive, clothing and textile firms in South Africa and Brazil. It aimed to address gaps in evidence of transparency through the supply chain, knowledge transfer between firms, and whether firms themselves used ethical HR practices and promoted them in their supplier firms. The SCA-Emp research team and advisory board included academics and practitioners from the UK, South Africa, and Brazil. The research involved a detailed survey of over 100 automotive and textile firms and 150 in-depth interviews with organisational stakeholders including senior managers, employer bodies, government officials, NGOs and trade unions. It was necessary as there was a lack of connectivity between the literature on supply chain accounting [R1] and employment relations [R2]. This was particularly important since firms needed a new mechanism for monitoring their suppliers’ HR practices due to commercial and ethical concerns – to improve profits and reduce risk to brands, and prevent harm to workers.

Brazil and South Africa were partnered with due to the challenging contexts of significant development and investment but enduring high levels of poverty. The findings revealed how, against the background of the 2008 economic crisis, there had been reconfiguration and regulation of supply chains [R3], and a weakening of the union role [R4] which had implications for how workers’ rights were protected both within companies and down the supply chain.

Within companies, there were differences between employment policies, and the practices actually used. For example, although companies often had diversity policies, this did not prevent discrimination against disabled workers [R5]. Furthermore, although there were pockets of good practice where firms had embraced job security and diversity while remaining competitive, the findings confirmed that many clothing and textiles jobs were precarious with weak employment conditions and health and safety provision, not unionised and informal in rural areas and that unions struggled to effect change. Across the supply chain, there was a lack of transparency, with limited sharing of accounting information or use of open book accounting, and a lack of knowledge transfer around HR practices [R6]. This was an important gap given lead firms’ potential ability to influence and practically support suppliers’ HR practices. Moreover, awareness of suppliers’ employment policies and practices rapidly diminished down supply chains leaving firms more vulnerable to reputational damage [R5]. The above findings and previous research undertaken by the research team led to the development of the SCA-Emp practitioner toolkit, co-produced with professional bodies in HR and accounting, which aimed to improve practices within companies and increase transparency down the supply chain.

4. Details of the impact

Given the ineffectiveness of legislation around improving labour practices in global supply chains, the SCA-Emp team intervened directly with firms using a diagnostic toolkit to highlight good practice and tackle the lack of transparency. The toolkit, which is directly derived from the research undertaken at Sheffield, contains sections on HR, accounting and supply chain management practices and the monitoring of suppliers. Aware of companies’ overriding commercial concerns, the toolkit was branded as a way to achieve sustainable and ethical practices within organisations and across the supply chain while ‘balancing people with profit’.

Over 450 companies from over 80 countries with a combined workforce of almost 500,000 employees and turnover of £74bn have accessed the toolkit. Companies with a combined turnover of over £31bn have self-assessed their HR and supply chain practices resulting in more ethical HR practices within firms and their suppliers.

Influence on professional practice through professional bodies

Co-production was essential in developing the toolkit since it enhanced relevance and helped to ensure implementation. The Chartered Institute of Management Accountants (CIMA), the Chartered Institute for Personnel and Development (CIPD), Employer Federations and the lead bodies for HR in Brazil (ABRH) and South Africa (SABPP) were involved in the design and evaluation. Subsequently utilising the toolkit to develop products for their >250,000 member companies.

For example, the ABRH (lead HR professional body for Brazil with 9,000 members) requested an ‘Olympics’ (gold, silver, bronze) as well as ‘traffic light’ (red, amber, green) version of the toolkit since self-assessing as red could potentially impact on HR managers’ job security. The ABRH changed their overall strategy and HR and accounting policies and practices, resulting in the creation of seven new HR training products between 2016 and 2018 [S1]. The development of the products enabled them to obtain better relationships with employees, customers, and suppliers, and increase competitiveness, productivity, and profits. The President of ABRH drew lessons from the toolkit and affirmed that they “benefited from engaging with the SCA-Emp project and the University of Sheffield, which was a great inspiration for developing new products with better content which can better serve our associates and HR professionals across all of Brazil” [S1].

Improved competitiveness, productivity, profits, and labour standards

More than 250 people from over 100 large companies, NGOs and employer federations attended workshops in the UK, South Africa, Brazil and Ethiopia, from 2014-19, to develop the toolkit and to promote its use. We also provided tailored interventional guidance to 21 companies. The toolkit is available via the SCA-Emp website ( https://sca-emp.com/) and is freely available for download in English, Portuguese, or Amharic. Our strategy involved promoting use of the toolkit then working with over 70 companies in South Africa, Brazil and Ethiopia to achieve substantial improvements in policies, practices, productivity and competitiveness.

Improved HR, accounting, and supply chain practices within Brazilian SMEs

Musashi Brazil, a foreign-owned automotive subsidiary with approx. 700 employees, used the toolkit to revise HR policies and practices. The changes included reinstating welfare benefits and free training for employees and middle managers. They developed better communication strategies with employees and improved the performance appraisal process. They also extended supply chain management and accounting practices, resulting in more detailed financial information from customers. The toolkit was used to inform how change through automation was carried out and resulted in increased productivity by around 7% from 2016-17, and increasing new business by around 5% from 2016-17 [S2].

Magnetron, a Brazilian owned automotive company with around 120 employees, used the toolkit to establish better HR practices by changing work hours. The HR manager confirmed that ‘as a result of using the toolkit we have improved the motivation of our employees’. They also improved supply chain relationships by increasing training and building partnerships with new suppliers and moved to requiring the employees of new suppliers to be legally registered [S2]. Qualimar, a food manufacturer with around 100 employees increased training by 50%, modernised recruitment practices, and changed accounting, supply chain and employment practices after using the toolkit. This resulted in more integration between functions, more developed logistics indicators and significantly reduced logistics and supply chain management costs, changes in the selection process and improvements in employee motivation and productivity [S2].

Nucleo Gestor - business incubator in the North East of Brazil responsible for 35 SMEs in the textiles industry - benefited from using the toolkit. The NGO President attested that “ that thanks to the use of the toolkit, the business incubator now has better relationships with customers, suppliers and employees, developed its strategy and further training products, set up a collaborative shop in Sao Paulo involving 20 companies, and developed a project to certify companies to be suppliers of ABVTEX (a Brazilian textiles employer body responsible for upholding labour standards)” [S3].

Improved HR, accounting, and supply chain practices in Ethiopian SMEs

After three workshops in Ethiopia 2018-19, the toolkit was translated into Amharic. Companies from a range of sectors used the toolkit to make changes internally and in their supply chains.

Awash Wine, the leading Wine Company in Ethiopia with around 1,000 employees, used the toolkit to identify gaps in their policies, processes and systems and develop action plans. This changed the company strategy, accounting and HR policies and practices. Benefits gained include better relationships with customers, suppliers and employees, and increased competitiveness [S4].

G-Global Gas, a gas and chemical firm used the SCA-Emp toolkit to review their HR practices prior to a successful audit by their largest customer (Coca Cola), resulting in a long-term contract as Coca Cola’s preferred supplier for CO₂ gas in Ethiopia and several neighbouring countries [S4]. Accreditation by Coca Cola led to reputational benefits, subsequently leading to contracts with other major manufacturers including Pepsi and United Breweries and job creation. The Managing Director attested that “I believe that our use of the SCA-Emp toolkit led directly to commercial improvements and to increased employment” [S4].

They also introduced new occupational health and safety and diversity management policies and procedures, including extended rest periods, improved reporting of accidents and appropriate grievance procedures.

Ethical procurement and increased transparency and knowledge transfer across supply chains

From 2017, Sheffield City Council (SCC) worked with the team to develop a bespoke version of the toolkit to monitor HR practices within their supply chain and measure the progress of supplier interventions. The toolkit is used with three of their largest suppliers accounting for over £70m of its total spend, and with 420 smaller suppliers. The toolkit is part of SCC’s Ethical Procurement Policy which was formally launched in October 2018. Additionally, it facilitated accreditation by the Real Living Wage Foundation in 2018 and Stonewall in 2019 [S5].

“We [SCC] are the primary beneficiaries of the toolkit but the secondary and most important beneficiaries are the employees in the supply chain. The toolkit is having genuine impact on individuals’ lives and company culture” [S5].

The toolkit has been used by the suppliers, with a total of 137,000 employees and turnover of £31bn, to make changes. For example, a supplier with over 6,000 employees has now committed to conduct regular supplier checks and audits. The toolkit is to be used by the rest of their 12,000 suppliers, as a contract requirement and contracts can be terminated based on poor practice.

CLEAR IDEAS: improving innovation skills and organisational performance

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Societal
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

To innovate, organisations need to not only be creative in generating solutions but skilled at putting them into practice. Creativity training tends to focus on the former whereas our research focuses on what influences both the generation and implementation of ideas. This knowledge was integrated to create a new innovation training intervention called CLEAR IDEAS (CI) first launched in 2005. Since August 2013, over 450 public, private and third sector organisations have attended CI innovation workshops around the UK and overseas. Their application of CI has led to significant, valuable organisational performance innovations across public and private sectors.

2. Underpinning research

The CLEAR IDEAS (CI) innovation development methodology translates research findings on effective innovation into improved organisational practice by systematically developing the skills of employees and managers to both better generate (I.D.E.A.S. steps) and implement (C.L.E.A.R. steps) new ideas. CI was first launched in 2005, based on insights from research studies undertaken by Dr Kamal Birdi since 2000. It has been continually updated with ongoing research. Birdi’s research has involved: a) investigating the major barriers to and facilitators of organisational innovation; b) identifying the knowledge, skills and other attributes required to overcome these challenges and be a successful innovator; and c) evaluating the effectiveness of training/development activities in this area [R1].

From 2000, Birdi led Sheffield colleagues in conducting multiple studies evaluating the impact of creativity training and other interventions in organisations [R2]. These showed that: multiple factors influence the implementation of ideas and these are not the same as the factors affecting idea generation; different types of courses have different impacts; and there is a need to combine the strength of disparate approaches [R1]. Furthermore, from 2003-4, Birdi contributed to two literature reviews for the Department of Trade and Industry’s 2003 UK Innovation Review. He co-authored the narrative review [R3] on people-related factors influencing organisational innovation, writing the chapter on topics such as employee motivation, skills, creativity training and leadership. He then contributed to the analysis and writing up of a systematic literature review [R4] on the factors influencing the uptake of new practices in organisations. These reviews showed that while innovative ideas can be plentiful, weaknesses in organisations (e.g. lack of stakeholder motivation or engagement, poor alignment of resources, unsupportive leadership, unclear goals) can prevent them being implemented successfully and hence directly influenced the choice of the CLEAR implementation factors in the model.

Creativity training provision focuses predominantly on idea generation. Birdi’s research strongly suggested a need for a new innovation training model which developed the skills of employees and managers to tackle both the creative and implementation aspects involved. Thus, Birdi created the CLEAR IDEAS model in 2005. The IDEAS (Illuminate, Detail, Erupt, Assess, Select) part involves the idea generation phase and builds on creative problem-solving research. The CLEAR aspect (Commit, Lead, Engage, Align, Review) is unique to the literature as it encourages participants to develop a plan for the implementation of their ideas and is based on key lessons evidenced from his own, and others’, research [R1, R2].

Birdi has continued to lead Sheffield research on identifying barriers and influences on employee innovation [R5] and interorganisational collaborative innovation [R6]. These further supported the development of the collaborative aspects of the training (e.g. highlighting the importance of co-creating a shared vision and engaging a diversity of stakeholders) and the new digital toolkits introduced in 2014 and 2015 (which included additional focus on project planning arising from **[R5]**).

4. Details of the impact

REF2014 focused on the impact of a two-day CLEAR IDEAS (CI) workshop designed for public sector leaders in South Yorkshire. CLEAR IDEAS has expanded into new sectors and new CI digital toolkits have been developed by Birdi [ R1, 5 & 6]. Through these pathways, Birdi’s research has led to significant, valuable organisational performance innovations in many sectors.

Since August 2013, more than 1,800 people from over 450 organisations have taken part in CLEAR IDEAS workshops around the UK and overseas (Canada, Hungary) and across the private sector (Corinthia International Hotels, Cisco, Serco Consulting, AWE, Anglian Water, Landsnet, Microsoft, Ferrero, Ocado Technology, TGI Fridays); public sector (NHS Trusts, Ofwat, Department of Work and Pensions, Sheffield City Council, West and South Yorkshire Police, Network Rail); and third sector (VSO, YMCA, WWF, SightSavers International, StopAIDS, Christian Aid, MENCAP and SCOPE) [S1].

The CI training is now available as an iPad app (2014) and online version (2015). This increased accessibility enables users to more easily apply the model in the workplace subsequent to training. Through the online digital toolkit, we have been able to reach hundreds of users nationally and internationally (e.g. Canada, U.S., Australia, Malta, Poland, Iceland, South Africa, and Chile).

The intervention has provided a systematic, generalisable and flexible method for organisations to innovatively address challenges, offering the most value where there is an unmet need to co-produce innovation between users and providers.

Innovation in the water sector, leading to better regulation and a better deal for consumers

Ofwat is the Governmental body responsible for the economic regulation of the water industry in England and Wales. Birdi’s research into the need for greater collaborative innovation in the water sector [R6] was cited by the UK water regulator Ofwat in their ‘Driving transformational innovation in the sector’ Consultation Document in July 2019 to highlight the need for better take up of innovative solutions by water companies.

The report offered a number of strategies for consultation to improve innovation in the industry (e.g. innovation competitions, centre of excellence). Birdi was commissioned by Ofwat to run a consultation workshop using the CI model to increase its engagement with a range of water companies, suppliers, policymakers and academics. By applying the CI model, participants created a shared definition of innovation, identified major barriers to innovation and generated a range of strategies for overcoming those barriers. The workshop increased the volume and quality of feedback to Ofwat on the initial strategies proposed in the consultation [S2]. The results of the CI process influenced Ofwat’s final strategy document published in December 2019.

Ofwat commissioned CI innovation training in February 2020. Attendees included Ofwat staff, members of other regulators such as the Drinking Water Inspectorate, National Resources Wales and the Environment Agency. The workshop aided cross-regulator collaboration and produced useful inputs for new innovation policy development. The Ofwat Principal (Strategy Focus) stated that CI had delivered “ a different approach to stakeholder engagement…and an indirect outcome of that is better regulation and at the end of the day, a better deal for consumers” [S2].

Shaping of new water research agenda

UK Water Industry Research (UKWIR) is an independent body, funded by the major UK water companies, responsible for facilitating the shaping of the water industry's research agenda. It has invested £15m in research over the last 15 years with a focus on answering 12 ‘big questions’ around water delivery and treatment. Collaboration is key to their approach and between 2016 and 2020 UKWIR employed the CI model to engage a cross-section of industry stakeholders to develop their Research Route Maps for three big questions (drinking water quality, zero pipe leakage and environmentally sustainable wastewater systems). A CI co-production workshop increased the zero leakage Route Map from 24 to 33 project areas [ S3], including a new funded project on customer-side leakage approaches delivered in 2020. This engagement allowed UKWIR to create a research agenda more able to represent and meet the needs of water sector stakeholders.

Influencing how the Department of Work and Pensions (DWP) engages with people on benefits.

In 2018, DWP was exploring a digital initiative for 200,000 people on benefits in the UK to receive a virtual consultation rather than a face-to-face meeting to improve access to job advice. DWP employed CI in a workshop with the Labour Market team to determine how the trial could be optimised in terms of design and delivery. However, the Diagnose aspects of the CI model showed participants “there wasn’t the degree of consensus among stakeholders about the purpose and design of the trial that they had initially assumed”. This resulted in DWP stopping the trial and redefining its terms before “ taking the trial forward on a firmer footing” [S4].

Co-producing a new Special Educational Needs and Disabilities (SEND) vision and strategy for Sheffield City Council

In early 2019, OFSTED published an inspection report criticising Sheffield’s SEND strategy, including lack of stakeholder engagement resulting in a number of actions. To address these, the council employed the CI model to run two co-production sessions to reshape their SEND Inclusion vision and strategy. CI allowed active engagement across a diverse range of stakeholders to collaboratively produce solutions. The resulting SEND Consultation Document stated that the CI model “ enabled us to co-produce our priorities and solutions working with over sixty individuals from our key stakeholders. This proved successful in identifying positive solutions and priorities which has formed the basis of the strategy” [S5]. The workshops identified a need to improve communication with parents and young people and this led to a redesign of the local offer website to make it more accessible.

HEALTH AND SOCIAL CARE: Co-design of NHS health technology innovation programme

The £2.5m NHS TestBed programme Perfect Patient Pathway (PEPPA) was set up in 2016 to pilot and evaluate a range of health technologies with patient groups and Primary Care Trusts. A key principle was to co-design the pilot projects. CI workshops enabled a range of stakeholders to identify their needs: “The projects within the programme were designed in response to the perspectives of stakeholders on their priorities for care and the use of technology using the ‘Clear Ideas’ method” [S6]. The report evaluation stated that “Engagement workshops were really helpful for the identification of unmet needs” [ S6].

Enhanced pupil performance of a Primary School

In September 2015, a CI workshop was conducted with staff of [Text removed for publication] a Primary School. Consequently, new focused strategies for maths and literacy improvement were introduced for pupils, including splitting up classes into more age-relevant groups on certain afternoons and introducing bespoke booster classes. The Head Teacher has attributed these strategies to a subsequent annual improvement trend in OFSTED-rated SATS performance for Writing (+3.03% above national average in 2016, +3.48% in 2017 and +4.00% in 2018) and for Reading: (-0.67% in 2016, -0.47% in 2017 and -0.4 in 2018). Further innovations included introducing social media to engage more effectively with parents, creation of new roles and gaining additional funding to open up early years provision. Enhancements such as these facilitated the school being able to join [Text removed for publication] an Academies Trust in 2019. As the Head stated CI app enabled “ how and what different solutions could we come up with, because actually that is the key to improvement in performance in schools” [S7].

LEISURE AND TOURISM SECTOR: Improving the quality of customer service in Corinthia international hotel chain

In November 2017, Birdi ran a two-day CI workshop with managers from Corinthia hotels in Budapest, Malta, London, St Petersburg, and Prague. By March 2018, participants had used the CI digital apps to generate and implement a range of innovative strategies to deal with customer service quality problems that were being faced by them. In Prague, restaurant service was reorganised which led to reduction of customer complaints to almost zero and generation of substantial additional revenue of €72k for the rest of 2018 [S8]. In Budapest, sharing of resources, suppliers and roles with a smaller sister hotel led to notable financial savings for the smaller hotel. In Malta, simplifying the registration card reduced the customer check-in time by 1.5 minutes and English lessons and training in guest names improved employee engagement with guest scores by 10% [S8]. This shows that with the addition of the CLEAR implementation in CI workshops participants were better able to realise the benefits of their creative ideas.

Setting up a technical skills training school inside an energy industry company in Iceland

Since 2014, a major Icelandic energy infrastructure company Landsnet, has used the CI method to design and obtain investment for a major new in-house technical engineering skills training school. The school was initially met with some resistance among employees, but the CI process presented it as mitigating the loss of skills from the impending retirement of many workers. A pilot programme was run in 2019 and the school is now fully operational in 2020. “ The Landsnet Academy has taken training to new heights… and the results can be seen directly in new behavior at work…CLEAR IDEAS helped us arrive at the outcome much quicker than we would have done without it, the reason being that CLEAR IDEAS helped us keep track of everything that we did and acted as a manual for generating ideas and evaluating them, so it helped us prioritise and organise our ideas and also gave us the motivation that we needed” [S9].

Improving measurement of health outcomes for NICE appraisals

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Health
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

NICE makes important cost-effectiveness recommendations on medical and social care practices. Its work requires that health outcomes are valued consistently across conditions and treatments. Health outcomes are typically reported by patients using a simple questionnaire (EQ-5D) which has recently been updated, and NICE now requires an updated method of valuing outcomes from the new measure. The research examined the only valuation system that has so far been developed for new EQ-5D and demonstrated to NICE that (i) its adoption could lead to major changes in recommendations and (ii) that it is based on flawed data and statistical analysis. As a consequence, NICE guidance has been revised, and new research has been launched to put valuation on a sounder basis. This has changed current NICE policy and will have important long-term implications for NHS patients and spending.

2. Underpinning research

The UK National Institute for Health and Care Excellence (NICE) (and similar bodies internationally) makes recommendations on a wide range of medical technologies and practices that affect the health of millions of people. This work is largely based on clinical trials that compare health outcomes of patient groups receiving alternative treatments. Outcome measures are used to generate conclusions about the relative cost-effectiveness of those treatments. The most widely used outcome measure that can directly be used in cost-effectiveness analysis is the EQ-5D, which was originally developed by The EuroQol Group in 1990 to measure health outcomes in five domains (mobility, self-care, usual activities, pain, anxiety/depression). A complication is that EQ-5D has been subsequently updated, to give better sensitivity by measuring each domain on five levels (EQ-5D-5L) rather than the original three levels (EQ-5D-3L). These two alternative versions are both in current use.

For purposes of cost-effectiveness analysis, outcome measures are converted into utility values representing societal views of the relative seriousness of outcomes. The utility values can be used to compare benefits on a common basis (using quality-adjusted life years or QALYs), across alternative treatments and different conditions. Construction of the required utility values is done by means of hypothetical choice experiments, which elicit relative valuations from representative members of the general population. The first valuation system proposed for use of EQ-5D-5L in England was published by Devlin, et al, in Health Economics 2017. NICE were initially minded to adopt this system but, following presentation at the NICE Technical Forum of research underpinning publications [R1-R3], a proposal for quality assurance work was discussed and prioritised by the Policy Research Unit Oversight Group comprising senior representatives from the Department for Health and Social Care (DHSC), NHS England and NICE.

Those discussions led to commissioning of the research described here, which is part of a sustained programme of research on outcome measurement, conducted by Steve Pudney and two colleagues in Sheffield’s School for Health and Related Research (ScHARR) – Mónica Hernández Alava and Allan Wailoo. This research received funding from DHSC (under the auspices of NICE), the ESRC and MRC; and it was carried out within EEPRU (the Policy Research Unit in Economic Methods of Evaluation in Health and Social Care Interventions) established to provide evidence to help the Department of Health and Social Care, and its constituent bodies, make the best use of scarce resources. Pudney’s main contribution was to the overall design of review work, empirical analysis of experimental data, technical analysis of modelling methodology and guidance on software review. The research was prompted by the team’s appreciation of the long-term importance of the choice of valuation system for EQ-5D-5L and our concerns about some aspects of the proposed valuation system.

The research was aimed at improving methodology in two ways:

By re-examining a large number of real-world cost-effectiveness studies to demonstrate that adopting the new valuation system would have potentially important consequences for NICE decisions (R3, R4). This research made clear the need to establish whether or not the system met a reasonable quality threshold.
By conducting an in-depth critical evaluation of the data, statistical methodology and estimation software that had been used to construct the proposed valuation system (R5, R6). Specifically, the evaluation comprised:
Statistical work providing an empirical assessment of the quality of the experimental data on which the value set was based.
Re-estimation and technical evaluation of the econometric specification and methods, and Bayesian software implementation used to construct the value set.

4. Details of the impact

The impact was produced by two complementary strands of research:

Examination of the impact of the proposed value set on existing cost-effectiveness studies (R3, R4) were used to inform NICE of the possible difficulties that might be encountered in using the proposed EQ-5D-5L value set uncritically. This led to the commissioning of an in-depth review. Novel methods that enable this work to be conducted are reported in R2 and have subsequently been adopted by NICE in their new updated Methods Guide (S1).
Review of the proposed EQ-5D-5L value set . The review reached three main conclusions:
There were serious concerns about the quality of data yielded by the choice experiments underpinning the construction of the value set. In particular: the experimental design covered a low proportion of the health states distinguishable by the EQ-5D-5L instrument; many participants gave responses suggesting a lack of engagement with or understanding of the experiments.
There were flaws in the specification of the econometric choice model and the method used to draw inferences about relative values of health states.
The Bayesian estimation procedures were not adequately implemented and did not achieve the required stability properties.

Given these serious concerns about data and methods, our report to DHSC and NICE recommended that “A new programme of further development, including a new data collection initiative, should be considered to put EQ-5D-5L on a sufficiently firm evidential basis” (R5, R6).

The full report (R5) was put out for independent review by NICE to four international reviewers, all of whom accepted the recommendations and one of whom (Prof. Charles F. Manski, Northwestern University and Fellow of the Econometric Society and American Statistical Association) stated: “ The [...] recommendations [...] to NICE and DHSC are exemplary”.

NICE updated its position statement on EQ-5D-5L in November 2018 and October 2019 to reflect the team’s recommendation that the proposed valuation system was not of acceptable quality (S2, S3).

A call for expressions of interest was issued for a new data collection and valuation exercise, to be carried out in conjunction with EuroQol (S4). To avoid any perception of conflict of interest, Pudney, Hernández Alava and Wailoo did not bid for that work. The contract was awarded in June 2020 and the new valuation exercise called for in the report is currently underway.

This decision by NICE, based entirely on the work of Pudney, Hernández Alava and Wailoo, directly affects many of the cost-effectiveness decisions currently made by NICE and will for the foreseeable future (S5, S6). It is impossible to predict with any confidence the number of cost-effectiveness decisions that will be affected by our work in the future. However, between Oct 2019 (when NICE’s updated EQ-5D guidance was published) and July 2020, 29 pieces of Technology Appraisal (TA) Guidance were issued to the NHS, and 27 used EQ-5D. Many of these decisions covered new cancer drugs. For example, Palbociclib (TA619) is a treatment option for around 3,300 people per year with advanced breast cancer in England and costs £2,950 for 21 days treatment. Other decisions concerned treatment for very common conditions. Around 90,100 people with type 1 diabetes are eligible for treatment with dapagliflozin (TA597), which costs around £500 per year per patient, and NICE estimates a budget impact of £3.5m per year with 10% uptake. It is still more difficult to predict the long-term impact on NHS patients or the NHS drugs bill (respectively 16.2m referrals to treatment and £19.8bn at list prices in 2018/19 for England). But this work has already made a significant change to NICE methods guidance (S2, S3) and will in due course influence NICE evaluations further by ensuring that health outcomes valuations based on sound statistical evidence and methods will be adopted. Over the long term, this will have large cumulative impacts on choice of drugs and other medical technologies and consequently on human experience of treatment and value-for-money in NHS spending.

NICE methods in relation to the use of EQ-5D go beyond the highest profile guidance NICE issues on new pharmaceuticals. They also cover guidance on diagnostics and on clinical and public health topics. They are seen as the gold standard for non-NICE assessments such as those undertaken by the Department of Health and Social Care; its arms’ length bodies such as those issuing guidance for vaccinations and screening; and the portfolio of research, including clinical trials, funded via the National Institute for Health Research (NIHR). NICE’s methods, including the use of the UK EQ-5D value set, are also often replicated by countries around the world, meaning that the current change in NICE policy brought about by Pudney’s work will also have implications for health care decision making around the world.

The SF-6D: An internationally adopted measure for assessing the cost-effectiveness ...

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Health
Is this case study continued from a case study submitted in 2014?: Yes

Download case study PDF

1. Summary of the impact

Deciding which health care interventions to fund is a complex and emotive question. Objective analyses of cost-effectiveness are a crucial aid to decision making. The Short Form 6 Dimension (SF-6D) algorithm, developed by University of Sheffield researchers, is used around the world to inform these decisions. It enables the calculation of ‘quality adjusted life years’ (QALYs), which allow calculation of the cost-effectiveness of healthcare interventions given ever increasing demands on scarce resources. The SF-6D is one of only three such algorithms in general use globally, and has major advantages over the other two due to its greater sensitivity to detect changes in health. Public policy and social welfare benefits arise from its use by health reimbursement agencies globally to facilitate more efficient use of funds. Commercial benefits arise from licensing the SF-6D to pharmaceutical companies and other private sector users, to enable them to assess the cost effectiveness of their products. Since August 2013, 1,568 commercial licences have been purchased globally generating more than $2.2 million in royalty income. The SF-6D is also freely available to non-commercial bodies for use in research and policymaking.

2. Underpinning research

Estimating the cost effectiveness of healthcare interventions requires a generic measure of health benefit to allow comparison across treatments for different conditions. The standard accepted measure is the QALY; this combines benefits to both quantity and quality of life (QoL). The SF-6D is used to calculate ‘Q’ - the change in QoL arising from an intervention. For example, if a hip replacement enables patients to move without pain, the SF-6D is used to estimate how much that QoL improvement is worth. In a healthcare system, it is used to compare the benefits arising from say, statins for high cholesterol to different treatments for dementia, or diabetes, or cancer, or a myriad of other conditions.

Roberts was a key member of the 3-person team (with Brazier and Deverill of Sheffield’s School for Health and Related Research - ScHARR) which developed the original SF-6D project funded by GlaxoSmithKline (GSK) in 1999-2001. Roberts’ main contributions included development of the methodology behind the entire project, and the econometric modelling underpinning the SF-6D algorithm. Following data collection in 2000, the main models were developed in 2001, with extensions to research and modelling up to 2008. Impact arises from the whole body of work.

The main research involved three steps:

(i) Deriving the SF-6D from an existing healthcare measure used in clinical studies worldwide, the SF-36, using a pioneering psychometric approach. The SF-36 is not suitable for use in economic evaluation because it is not ‘preference based’; it measures only the amount of limitation, it does not show how people ‘trade’ between different dimensions of health (e.g. pain vs. physical functioning), or between quality and length of life. The research reduced the SF-36 to the simpler SF-6D classification, which defines 18,000 different health states.

(ii) Obtaining population preferences for a sample of the health states from a large sample of the UK population, using the theoretically grounded standard gamble stated preference method.

(iii) Using econometric modelling to estimate preferences (utilities) for all possible SF-6D states, from the sample data. The resulting SF-6D algorithm generates an index, from 0 (equivalent to being dead), to 1 (full-health). It quantifies how much value people place on different health limitations, and how they trade between them; e.g. how much vitality they would sacrifice for a reduction in pain. This index is the ‘Q’ value used to calculate QALYs.

The main SF-6D research output was published in 2002 and has been cited more than 1,800 times [R1]. Four main extensions to the research led to findings that increased the impact in this submission period: (i) in 2005 we developed a second algorithm for the SF-12 [R2] a reduced version of the SF-36, also in common clinical use. (ii) we developed an improved Bayesian algorithm in 2007 [R3], enabling decision makers to take better account of differences across patients. (iii) The Sheffield team pioneered the development of the SF-6D, and since then studies using this method have been replicated in 8 additional countries (Australia, Brazil, China, Indonesia, Japan [R4], Portugal, Singapore and Spain) to obtain local population preferences. (iv) We have extended reach to a broader set of health problems by developing algorithms for some conditions where the SF-6D is not appropriate; for example, in urinary incontinence where the SF-6D may neglect important effects on sleep quality [R5].

The SF-6D has three key advantages over its main competitor (the EQ-5D): (i) It is a richer descriptive system (18,000 states cf. 243), so is more sensitive to changes in health [R6]. (ii) Preferences are obtained using a theoretically superior method. Standard gamble is realistic because it involves making a risky choice; i.e. “would you choose to live the remainder of your life in less than perfect health, or take a risk on a treatment that could restore perfect health but could also leave you in a worse health state”. In contrast, the EQ-5D uses a riskless valuation approach. (iii) The SF-6D can be derived from the SF-36 or SF-12, the most widely used generic outcome measures already included in clinical trials around the world; thus it imposes no additional resource burden on the trial.

4. Details of the impact

Pathways to impact. The original research was funded by the global pharma company GSK, giving the SF-6D a high profile in this important industry. GSK freely uses the algorithm to calculate the cost-effectiveness of their products; other companies have to purchase licences to gain the same capability. Early dissemination was aided by targeted presentations to pharma companies and other key health decision makers. Ongoing dissemination since August 2013 includes: coverage in ScHARR’s annual 2-day course on Utility Data for Health Technology Assessment, as well as in the free Measuring and Valuing Health Massive Open Online Course (MOOC), completed by 3,730 learners globally since it started in 2015; and promotion at the annual International Society for Pharmacoeconomics and Outcomes Research (ISPOR) conference. ISPOR has more than 20,000 members in 110+ countries. To optimise access, the algorithm is supplied via easy-to-use software, including Excel, SPSS, and SAS, and the SF-6D website provides full guidance.

Reach of the SF-6D is illustrated by the range and number of beneficiaries.

International commercial organisations

The main commercial beneficiaries are pharma companies. A standard measure administered in their clinical trial (SF-36/SF-12) can be easily translated into QALYs via the SF-6D, to demonstrate the cost-effectiveness of their products. Most commercial licence sales are via OptumInsight, a US-based global health innovation company who sell the SF-6D under sub-contract, alongside other related products (SF-36 and SF-12). Further sales are via the University’s Licensing Portal.

[Text removed for publication]. Together these companies control over 60% of the $1,200 billion global pharma market. 56% of licence sales are in the US, where other important commercial users include healthcare insurers and providers. Licences have also been purchased in 40 other countries, including Australia, Canada, China, Denmark, France, Germany, India, Japan, Russia, South Korea and the UK [S1]. They cover a broad range of disease areas, including, for example; addiction, asthma, diabetes, heart disease, HIV, multiple sclerosis, rheumatoid arthritis, stroke, and a number of different forms of cancer, including those with the highest population prevalence such as bowel, breast and prostate cancer. Reach is increasing over time. Annual sales have increased from 139 licenses in 21 countries in 2014, to 354 in 31 countries in 2019 [S1]. This reflects the fact that the companies are under increasing pressure to demonstrate the cost-effectiveness of their products in resource pressured healthcare systems; and more health reimbursement agencies (HRAs) around the world are recommending the use of the SF-6D to generate this evidence (see below).

International health and welfare policy

The SF-6D is accepted for use in economic evaluation by HRAs around the world. HRAs publish mandatory guidance on how pharma companies must provide cost-effectiveness evidence, in order for the HRA to decide whether the product provides good value for money and hence will be provided within the healthcare system. Both reach and significance are evident in the fact that the SF-6D is accepted for this analysis in at least 20 of the 42 HRA guidelines available via ISPOR [S2]. 5 countries, including China, were recommending the SF-6D before August 2013 and continue to do so. 11 countries explicitly accept the SF-6D in guidance published since August 2013: Australia (2016), Brazil (2014), Canada (2017) [S2], Egypt (2013), Hungary (2017), Iran (2017), Ireland (2018), Japan (2016), Norway (2017), Spain (2014), United States (2016) [S2]. In Finland (2017) no algorithm is explicitly named but the SF-6D is accepted by implication because it meets the necessary stated criteria. Like NICE in the UK, New Zealand (2015) and the Netherlands (2016) both accept the SF-6D in cases where the alternative EQ-5D measure is not appropriate (e.g. due to ceiling effects **[R3]**).

Significance of the SF-6D is illustrated by the importance of the decisions it enables, the type of benefits it generates and the importance of these benefits for economy and society.

Decisions on the provision of cost-effective health care

HRAs have to decide which health care technologies to provide; these complex and emotive questions are aided by objective SF-6D analysis. In the UK, for example, the SF-6D has been used to calculate QALY benefits in important NICE assessments of treatments for hepatitis C [S3] and gout [S3]. Hepatitis C is a highly contagious disease which can cause liver failure. The drug in question (sofosbuvir) can cure 90% of patients, but it is very expensive; £39,000 for a 12 week course. Over 200,000 people in the UK have hepatitis C so the NHS budget impact is potentially huge. SF-6D analysis demonstrated that the QALY benefits are substantial, so the drug is cost effective despite its large cost. As a result, NICE recommended that sofosbuvir be made available; the NHS’s single biggest new treatment investment in 2017. SF-6D analysis has also been key to drug appraisals in other countries. For example, in Canada, it was used to estimate the cost-effectiveness of drug treatments for hypertension, and epilepsy, as well as hepatitis C.

The use of the SF-6D in these decisions is also significant because users generally have a choice between the 3 existing health valuation algorithms. The SF-6D has key advantages over the EQ-5D, its main competitor (see Research section). In the sofosbuvir decision for example, the HRA states “the SF-6D was used in preference to the EQ-5D because the conversion method is well validated and that EQ-5D utilities are less certain” [S3: p79]. Similar arguments are made to explain the use of the SF-6D in decisions for treatments for gout [S3]. This has also been the case internationally. For example, the appraisal of rifaximin for end-stage liver disease in the Netherlands preferred the SF-6D analysis on the grounds that the direct conversion of the SF-36 to SF-6D was more accurate than the indirect EQ-5D analysis [S3].

As well as single technology decisions, SF-6D evidence is also key to some of NICE’s influential care pathway guidelines. Low back pain (LBP) affects 10.5 million people in the UK and is a major cause of lost workdays. SF-6D evidence is cited throughout the LBP guideline and is key to the recommendations not to recommend acupuncture or epidural steroids as routine treatment, because they are not cost effective [S3]. SF-6D evidence was also used in the US Institute for Clinical and Economic Review assessment of LBP therapies [S3]. This concluded that evidence on acupuncture was inconclusive, hence it is rarely covered by US public health insurance. Other NICE guidelines with key recommendations relying on the SF-6D include: age-related macular degeneration, a deterioration in sight affecting 5% of the over 65s; kidney stones, which affect 9% of the population; and chronic obstructive pulmonary disease which affects 5.5 million people and where SF-6D evidence was key to recommending opportunistic case detection in primary care.

Health outcome in clinical trials

Between them, the two main SF-6D studies (R1 & R2) have been cited over 2,600 times (1,280 since 2014). The vast majority of these report the use of the SF-6D in clinical trials and economic evaluations around the world; providing clear evidence of its significance for healthcare decision-making. For example, [S4] is a large German trial to establish the cost-effectiveness of a program to optimise dementia care via early diagnosis and care in the community. Dementia affects 47 million people worldwide at a cost of around $818 billion; prevalence is increasing over time. SF-6D analysis showed the program to be cost–effective; QoL was improved (and costs reduced) by supporting people to live with dementia at home rather than be hospitalised. [S4] is the cost effectiveness analysis of the large multi-centre US trial of Look AHEAD; an intensive lifestyle intervention for type 2 diabetes (a chronic disease that affects around 10% of the US adult population and accounts for 24% of all US health spending). SF-6D analysis showed that this was not a cost-effective intervention; the QoL gains were not worth the increased costs compared to standard care, so health budgets could be wasted if this program was rolled out.

Measure of patient Quality of Life (QoL)

The SF-6D is also used in medical research to show how health conditions impact patients’ QoL. For example [S5] details how patients undergoing hip replacement in the US experience notable QoL gains up to 5 years post-surgery. The report cites the lead clinician from the Cleveland Clinic who states, “The SF-6D is one of the few straightforward, easily obtainable methods that provide clinicians quantifiable insight into a patient's QoL … widely incorporating the SF-6D into future postoperative assessments is straightforward, and having these values readily available may make prospective cost-effectiveness analyses considerably easier.” In another example [S5] demonstrates the usefulness of the SF-6D in assessing the burden of Atopic Dermatitis (AD). AD is a chronic inflammatory skin disease affecting 15-20% of children and 1-3% of adults. Previously QoL assessments for this disease were not standardised. Clinicians in a 2019 US study of 3,500 people showed that the SF-6D was a reliable measure of QoL in AD meaning that it could be used as a common metric to compare the burden of AD to that of other diseases.

Public sector and international charities

Licenses are free for public sector and charitable organisations; at least 1,340 have been distributed (668 since August 2013) [S1]. This is an underestimate of non-commercial usage, since the software is often shared freely. Non-commercial use demonstrates reach and significance through the wide range of different uses around the world. For example, the UK Department for Education used the SF-6D to demonstrate the value of the health benefits arising from the Youth Contract for unemployed young people (around £1 million/year) [S6]. SF-6D analysis was also central to establishing the cost-effectiveness of the pay-for-performance program for diabetes care under the Taiwan National Health Insurance scheme. [S6]

Summary: The SF-6D contributes to public policy and social welfare and has substantial commercial benefits [S1]. It is accepted by HRAs in at least 20 countries [S2], is a key outcome in many important health technology appraisals in the UK and internationally [S3], and is commonly used to measure cost-effectiveness in clinical trials [S4] and other medical research [S5], as well as more broadly in public service provision [S6]. Ultimately the populations and tax payers of the countries involved are SF-6D’s main beneficiaries, since healthcare authorities are able to extract better value for money from limited resources because of the quantitative comparisons of cost-effectiveness that the SF-6D makes possible.

5. Sources to corroborate the impact

SF-6D license sales [Text removed for publication].

SF-6D use in international Health Reimbursement Agencies (HRA) methods guidance. The International Society for Pharmacoeconomics and Outcomes Research provides summaries of the HRA guidance for 50 countries around the world. Of the 42 available via the website, 17 accept the SF-6D as a means of calculating QALYs. For example: 2017 Guidelines of the Canadian Agency for Drugs and Technologies in Health and 2016 US Academy of Managed Care Pharmacy guidelines for Submission of Clinical

SF-6D use in international health technology appraisals (HTAs) and clinical guidelines. TA430 Sofosbovir for hepatitis C. p79; TA506. Lesinurad for treating chronic hyperuricaemia in people with gout. p18; Pharmaco-economic re-evaluation report for rifaximin in the treatment of recurrent episodes of hepatic encephalopathy manifest in patients aged 18 years (Netherlands HTA). NICE guideline 59: Low back pain and sciatica in over 16s see pp 491, 497, 498. US Institute for Clinical and Economic Review Cognitive and Mind-Body Therapies for Chronic Low Back and Neck Pain: Effectiveness and Value. pp 55, 56.

SF-6D use in important clinical trials. Michalowsky et al (2019) Cost-effectiveness of collaborative dementia care management: results of a cluster-randomized controlled trial. Alzheimer’s & Dementia 15:1296-308. Section 2.4 for use of SF-6D (SF-12 version) to calculate health utilities. Zhang et al (2016) Impact of Intensive Lifestyle Intervention on Preference-based Quality of Life in Type 2 Diabetes: Results from the Look AHEAD Trial Obesity 24(4):856-864. doi:10.1002/oby.21445

SF-6D use in patient quality of life (QoL) studies: Undergoing hip replacement improves five-year quality of life. ScienceDaily (2017). Short-Form 12 and Short-Form 6-Dimensional May Reliably Assess Atopic Dermatitis Burden on QoL. Dermatology Advisor (2019):

Non-commercial use of SF-6D in UK government evaluation. Use by UK Department for Education to evaluate the Youth Contract for 16- and 17-year-olds not in education, employment, or training. Hsieh, H.-M. et. al. (2015). Cost-Effectiveness of a Diabetes Pay-For-Performance Program in Diabetes Patients with Multiple Chronic Conditions. PLOS ONE, 10(7), e0133163. https://doi.org/10.1371/journal.pone.0133163

Transforming policy approaches towards undeclared work in the European Union

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Political
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

Research conducted since 2006 by Professor Colin Williams from Sheffield University Management School (SUMS) has shaped European-wide policy towards undeclared work, challenging the consensus and changing how undeclared work is tackled. His research directly resulted in European Parliament legislation that implements a platform he designed. Following his appointment as lead expert to the ensuing European Platform Tackling Undeclared Work, the widespread implementation of his holistic approach has led to fundamental change in how undeclared work is tackled in European Union member states and beyond. The impacts are maximised by the School’s strategy of working with supra-national bodies.

2. Underpinning research

Since joining Sheffield University Management School (SUMS) in 2006 as Professor of Public Policy, Williams has produced a body of research supporting a new holistic policy approach for tackling undeclared work (which constitutes 9.3% of total labour input in the private sector in the European Union).

Undeclared work involves paid activities not declared to the authorities for the purpose of evading tax and social security contributions and/or labour laws. Different government bodies (i.e., tax authorities, social security agencies and labour inspectorates) are responsible for each component, leading to a fragmented uncoordinated approach. Furthermore, most of these authorities have pursued the objective of eradicating undeclared work and consequently used a limited range of policy measures.

Williams’ body of research has provided the evidence-base for a new holistic policy approach that: (i) has the objective of transforming undeclared work into declared work; (ii) uses a full range of both direct and indirect policy measures, and (iii) joins-up the fields of labour, tax and social security law at the level of strategy and operations.

From eradicating undeclared work to transforming undeclared work into declared work

For many decades, undeclared work was considered exploitative waged employment conducted under ‘sweatshop-like’ conditions resulting in governments adopting a policy objective of eradication. As the only academic on a team that won a European Commission tender in 2006 to design a survey of undeclared work in the EU, Williams ensured that the full range of types of undeclared work was evaluated. The resultant special Eurobarometer surveys conducted in 2007, 2013 and 2019, are the largest of their kind ever undertaken, each involving 27,000 respondents across the EU. These Eurobarometer surveys reveal that much undeclared work is conducted on a self-employed basis, such as by entrepreneurs starting a business who often test-trade its viability in the undeclared economy [R1]. Williams has consequently provided the evidence-base to support a shift in objective from eradicating undeclared work to transforming undeclared work into declared work [R2].

Using a fuller range of direct and indirect policy measures

Conventionally, participation in undeclared work was explained as a rational economic decision taken when the benefits outweigh the expected cost of being caught and punished. To eradicate it, the policy solution was therefore to increase the costs by raising the actual or perceived penalties and probability of detection. Based on the Eurobarometer surveys, however, Williams has highlighted that even when the benefits outweigh the costs, and the rational economic decision would be to participate in undeclared work, many do not. To explain this, Williams developed a social actor approach [R3]. Williams has combined institutional theory and data from these Eurobarometer surveys to provide evidence that undeclared work occurs when norms, values and beliefs (i.e. civic morale) do not adhere to the laws and regulations (i.e. state morale). Williams’ research shows that changing the cost/benefit calculation using direct policy measures at the same as aligning norms, values and beliefs with the laws and regulations using indirect policy measures, is the most effective means of tackling undeclared work [R4].

Joining-up the fields of tax, social security, and labour law

Williams (with colleagues at Regioplan, a Dutch consultancy company) was commissioned by Eurofound to produce reports in 2008, 2009 and 2013 setting out the evidence for joining-up the fields of tax, social security, and labour law in a new coordinated ‘holistic’ policy approach, and to develop a ‘knowledge bank’ of good practice examples [R5]. Williams took sole responsibility for writing the reports, providing the academic basis for this holistic approach, while Regioplan collated the good practices for the on-line knowledge bank. This holistic approach was taken further forward when European Parliament legislation established a European Platform Tackling Undeclared Work to implement this holistic approach [R6].

4. Details of the impact

Stimulating EU legislation

The REF2014 impact case study submitted by the same title revealed how research conducted by Professor Williams had resulted in the European Commission funding him to design an EU-level platform to enhance cooperation when tackling undeclared work. The impact claimed in REF2014 was that his report had been taken forward in the European Commission’s 2013-14 Legislative Work Programme and that legislation was in the pipeline to establish the Platform.

In 2016, the European Parliament and European Council passed the legislation establishing the European Platform Tackling Undeclared Work [S1] designed by Williams. Therefore, in the current REF period, his research has directly led to European Parliament legislation that implements a platform he designed.

Providing the expert advice to the European Platform Tackling Undeclared Work

Immediately following this legislation being passed by the European Parliament, the European Commission tendered for the services contract to support the European Platform Tackling Undeclared Work [S2]. Professor Williams led the successful consortium, composed of the University of Sheffield and three private sector consultancy companies, namely ICF (USA), CSD (Bulgaria) and Adapt (Italy), being awarded the €5.6 million 4-year contract (2016-2020).

As lead expert to the Platform and Deputy Manager of the service contract, Williams’ role has been to implement the mission, objectives and activities of the European Platform, as set out in the legislation [S3].

Implementing the holistic approach across the European Union

Paragraph 12 of the legal decision (EU) 2016/3441 of the European Parliament establishing the European Platform calls for Member States to adopt a holistic approach. Williams in 2016 officially defined this as a whole government approach that joins-up strategy and operations across the fields of labour, tax and social security law and uses the full range of direct and indirect policy approaches to transform undeclared work into declared work [R6].

Two reports detail the impacts that the Platform has achieved after two years [S4] and four years [S5] of operation. These impact reports have been agreed and ratified by both the European Parliament as well as the European Council. They reveal that as a direct result of the Platform, national governments in 16 Member States have moved beyond having different Ministries operating in ‘silos’ each with separate strategies and now have one cross-government national strategy. To take one example, in France, a new cross-government national strategy for tackling undeclared work was adopted in 2019 [which includes 57 citations to Williams’ research on a holistic approach], developed by the Prime Minister’s Employment Advisory Council [S6]. National governments in a further 5 Member States now also have a new cross-government coordinating body (Croatia, Finland, Latvia, Lithuania, Greece).

The ratified impact reports [S4, S5] also explicitly state that as a direct result of the Platform:

National governments in 16 Member States have developed new awareness raising and trust-building initiatives (e.g., to broaden the range of measures used, the State Labour Inspectorate in Croatia in 2017 organised an awareness raising campaign. The evaluation reveals the campaign resulted in 57% of citizens changing their attitude towards undeclared work);
National governments in 12 Member States have developed new sanction and detection initiatives (e.g., the Ministry of Labour in Spain, learning via the Platform about notification letters, decided to send such ‘nudge’ letters to 14,000 companies, resulting in 15% revising their declarations);
National governments in 10 Member States have developed new cross-border initiatives (e.g., a Platform-organised joint inspection by Spanish and Romanian labour inspectorates in the Albacete region of Spain led to four prosecutions for human trafficking), and
National governments in 7 Member States have developed new policy measures to incentivise the transformation from undeclared to declared work (e.g., a 2019 Platform recommendation for the Lithuanian tax authority to experiment with a voluntary disclosure initiative, to allow businesses to legitimise without penalty, resulted in an additional €42m in VAT revenue).

Beyond these impacts on national governments, in 2020, Williams’ consortium designed and implemented an EU-wide marketing campaign, #EU4FairWork, involving as participants the governments of all 27 Member States and all relevant EU- and national-level social partners, on the benefits of declared work. Five million EU citizens were reached in the first three days [S7].

Extending the holistic approach beyond the EU

Since 2018, Williams has been chief adviser on undeclared work to the Regional Cooperation Council who are aiding six Western Balkan countries (Albania, Kosovo, Serbia, Bosnia and Herzegovina, Montenegro, North Macedonia) with the EU accession process. To prepare these Western Balkan countries for EU accession from 2025 onwards, in 2018, Williams produced national action plans for each country [S8], and in 2020, was appointed chief adviser to the newly established Western Balkan Network Tackling Undeclared Work (with a €1.5 million budget), which mirrors the EU Platform [S9]. To aid the EU-accession process, its objectives are to implement a similar whole government holistic approach towards undeclared work in these six countries.

To transfer the holistic approach to the developing world, meanwhile, the International Labour Organisation (ILO) in 2018 commissioned Williams to apply his policy approach to Azerbaijan following an invitation by the President of Azerbaijan. Following this, in 2019, the ILO commissioned Williams to develop guidance for the 193 UN member states on how to develop an integrated strategic approach to implement ILO Recommendation 204 on formalizing the informal economy and the associated Sustainable Development Goal 8 on decent work [S10]. The ILO will now be used as the key mechanism for Williams to extend the reach of his holistic approach across the developing world.

Translating environmental sustainable supply chains into global resource efficient ...

Submitting institution: The University of Sheffield
Unit of assessment: 17 - Business and Management Studies
Summary impact type: Environmental
Is this case study continued from a case study submitted in 2014?: No

Download case study PDF

1. Summary of the impact

Sheffield’s Advanced Resource Efficiency Centre (AREC) uses systems theory to examine how decisions informed by a whole supply chain could maximise resource efficiency and productivity. Through strategic engagement with key bodies in China, this research has helped to embed a technological approach to resource efficiency and environmental sustainability in the country’s main social, economic and development policies: Made in China 2025, the 13th five-year plan, and Belt and Road. By adopting the AREC model, China’s policymakers have made changes to its global supply chains that support economic growth through maximising resource efficiency while minimising negative environmental impacts.

2. Underpinning research

Globally and locally, external pressures on organisations have created an imperative for combined low carbon, low cost operational and organisational solutions. In response, a team from Sheffield University Management School, led by Professor Lenny Koh, has researched and tested theories and methodologies to develop a supply chain resource modelling system to help organisations maximise resource efficiency and reduce carbon footprints, and expand the environmental sustainable supply chains into global resource efficient systems.

In response to user needs and working with public and private sector collaborators, Koh and her team undertook research on green supply chain theory through the study of supply chains in Europe and Asia [R1, R2]. They used systems theory to examine how decisions informed by a whole supply chain or an intra-organisational approach could improve performance across the supply chain. They then focused on low carbon supply chain management, the impact of environmental directives on intra-organisational decisions and establishment of a comprehensive decision support system for carbon management across the supply chain [R1, R2].

The group advanced the theory by advocating a balanced whole supply chain system approach to improve both the understanding of, and decision-making for, carbon accounting across global supply chains [R1, R2]. This applied methodologies to identify areas of greater risk and uncertainty for carbon reduction in the supply chain. Following this, the research advanced to focus on causality between system interventions and supply chains to evaluate and optimise the effectiveness of the tools that the team has developed.

The theoretical and methodological advances and findings, categorised into four strands of intellectual and novel contributions are:

F1) Koh’s circular framework for supply chain resource sustainability (SCRS) provided a robust and novel set of decision rules in which to assess the interconnectedness of environmental, economic, and social capital of supply chain resources from pre-production to post-production [R1].

F2) At a global scale, Koh’s research on electronic waste recycling and recovery, the circular economy and toxicology assessments, have found sustainable investment opportunities and new supply and demand governance structures [R3-R5]. Koh found that the toxicology footprint will increase if the economy is thriving and decrease if it is in recession [R3]. Her research recommends that policymakers and governments look at their toxicological footprint carefully and, instead of exporting their emissions, make changes to their current environment that reduce their emissions [R3]. Furthermore, Koh’s research found that the current resource efficiency system is ineffective and unsustainable due to the lack of a systematic framework [R5]. To address these weaknesses, there needs to be a specific structure that combines financial and governance frameworks. In-depth analysis by Koh [R4] revealed a huge economic incentive for this kind of market, which also leads to significant CO2 savings, contributing to the circular economy in a way that leads to a wider economic impact.

F3) Koh’s combined life cycle approach and sustainability measurement using the integrated resource efficiency (IRE) index of 40 countries including 34 OECD nations. It found China, Australia, Greece, and UK, had varying levels of urgent needs to address their economic, environmental, and social resource efficiencies in order to increase their integrated resource efficiency index [R1, R2]. The life cycle approach recognises selections as one part of a whole system of events. It identifies both opportunities and risks of a product or technology, all the way from raw materials to disposal. By integrating Koh’s life cycle methodology into a resource efficient model Koh’s research suggests that these economic systems and their industries will gain significant improvement in their sustainability performance.

F4) Koh’s research found that digital technological approaches such as blockchain, big data, artificial intelligence (AI) and Industry 4.0 have the potential to enable improved resource efficiency and productivity while reducing waste. Blockchain implementation in the supply chain, for example, eliminates the need for an intermediary by building trust directly between stakeholders [R1, R6].

These findings have been incorporated into a suite of software tools developed by Koh with Microsoft called the Supply Chain Environmental Analysis Tools (SCEnAT). SCEnAT, and its cloud-based versions, are an important tool for policymakers and industry to apply green supply chain theory.

4. Details of the impact

The AREC model identifies how to reduce environmental impacts and maximise productivity and resource efficiency within the supply chain. It does this by making use of the software tool based on Koh’s research [R6, F1-4], the Supply Chain Environmental Analysis Tool (SCEnAT), which uses data from the supply chain to reveal the points of highest carbon consumption and recommends interventions that have the most cost-effective impact on the carbon footprint.

Koh’s research has influenced the implementation of three of the highest-profile social, economic and development policies that China has launched: The 13th five-year plan; The Belt and Road initiative; and Made in China 2025. Since 2018, Koh has collaborated with Chinese organisations working on these three policies. By making use of her research, policymakers in China have strengthened the sustainability and resource efficiency of current supply chains. The AREC model provided a framework of recommendations targeting the maximum benefit for users instead of narrow changes without regard for the larger context, which was common practice prior to implementing AREC. Having a set of actions that are connected to each other enables a higher level of systems achievement overall throughout the supply chain. Implementing the AREC model enabled the impacts described below.

A. Embedding technological approaches to supply chain sustainability in the implementation of China’s Made in China 2025 and other key national policies.

The Made in China 2025 policy aims to upgrade the manufacturing capabilities of Chinese industries from labour-intensive factories into a more technology-intensive powerhouse. This transition required a technological approach to supply chain sustainability and resource efficient systems strategy globally. [Text removed for publication].

Koh recommended that blockchain, big data, AI and industry 4.0 technology should be embedded in China’s policies to enable the improvement of resource efficiency and productivity while at the same time reducing waste [F4]. [Text removed for publication], this recommendation was fed into the economic, industrial, science and innovation policies as well as the Made in China 2025 policy [S1]. [Text removed for publication] implemented Koh’s recommendation of introducing blockchain in its development and industrial policies [S1], to reduce reliance on intermediaries and build trust directly between stakeholders [F4]. This has resulted in direct impacts on companies and workers.

For example, in Shanxi, China, using Koh’s proposals, blockchain was embedded in seven key companies in the agri-food supply chains, turning them into resource efficient systems. All farmers in that area can now directly access the market without reliance on an intermediary. This improved their economic status in the value chain by 31% [S1], and developed more sustainable agricultural practices [F3]. These led to a 26% reduction in CO2 emissions and a 27% improvement in the utilisation of their natural resources [S1].

DRTT’s technological approach accomplished with blockchain, Cloud technology and Koh’s SCEnAT platforms [F4, R6] has improved connectivity between rural and urban operations. Koh recommended this technology to be more widely offered to the poorer population on a regional level, such as farmers. The farmers are now able to share these resources amongst themselves and through them earn more money. This led to significantly improved quality of life for the 37% of the 900,000 people living in poverty in Shanxi, China [S1].

B. Improving the resource efficiency of China’s USD 670 billion Belt and Road initiative.

China’s Belt and Road initiative (BRI) policy connects global supply chains for global economic and social development through investment, trade, and cooperation in major infrastructure such as transport and energy. In 2013, [Text removed for publication] the AREC model was adopted to maximise the resource efficiency in BRI and optimise sustainable development. Since then, China's trade with countries along the Belt and Road expanded 11.3% year on year to 4.57 trillion yuan in the first seven months in 2018 [S2]. [Text removed for publication].

The adoption of Koh’s findings [F1-4] resulted in a reduced carbon footprint in the Belt and Road supply chains and improved resource efficiency in the BRI. For example, AREC was considered to have made a meaningful contribution to the Belt and Road’s expansion of 2.7% above China’s overall trade increase, achieving USD 672.1 billion in trade in the first seven months in 2018 [S3].

C. Influencing sustainability strategy in the implementation of China’s national development and reform policy and in support of China’s 13^th five-year plan.

[Text removed for publication]. Koh prepared a confidential report [Text removed for publication] which provided short-, medium- and long-term actions. Koh’s research is credited with improving its development and reform policy planning, which is a key pillar in delivering China's 13th five-year plan (2016-2020) [S2]. They also designed the actions into their overall system, so the wider supply chains follow these requirements as well [S2].

As a direct result, Koh’s research was used to help transform social, economic, and environmental development in Ningbo, China. The environmental sustainability of Ningbo’s transport infrastructure, also connected to the Belt and Road initiative, has been improved by including environmental criteria as a requirement in public procurement contracts [S2].

[Text removed for publication]. Through workshops and sustained engagement with key stakeholders and senior policy makers, Koh has successfully incorporated the AREC model into economic, social, and environmental strategies and policies. By implementing her model and using the tools in it (such as SCEnAT), [Text removed for publication] reported that they have achieved “energy consumption per unit of GDP falling by 3.1% and decreasing the rural poor population by 13.86 million people” [S4].

Impact case study database

Search and filter

Filter by

A national action plan for tackling the undeclared economy in Greece

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Achieving ethical human resource and procurement practices and transparency through ...

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

CLEAR IDEAS: improving innovation skills and organisational performance

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Improving measurement of health outcomes for NICE appraisals

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

The SF-6D: An internationally adopted measure for assessing the cost-effectiveness ...

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Transforming policy approaches towards undeclared work in the European Union

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Translating environmental sustainable supply chains into global resource efficient ...

1. Summary of the impact

2. Underpinning research

3. References to the research

4. Details of the impact

5. Sources to corroborate the impact

Filter by higher education institution

Filter by unit of assessment

Filter by continued case study

Filter by summary impact type

Filter by impact UK location

Filter by impact global location

Filter by underpinning research subject