Impact case study database

Creating resources for changing practice and understanding

By bringing forth an awareness of the, often hidden, power and centrality of food practices within care contexts and relationships, FFT emphasised how significant improvements in care experiences could be achieved through easily implemented changes ( R4). It added to the knowledge base and skill set of residential and foster carers, as well as their managers and supervising social workers in the UK, Finland, and Australia ( R2). This was achieved primarily through five freely available resources at http://www.foodforthoughtproject.info:

• Online Interactive Introduction (for initial awareness raising),

• Reflective Workshop Materials (Facilitators' Pack/Handouts – initially delivered by the project team),

• Reflective Tool and Guidance (for use with specific children/young people),

• JOTIT Notebook (an easy-to-use way of tracking food practice and reflecting on its use),

• Peer Support Guidance (for group learning and sharing around food practices).

To maximise the impact on practice, the team ran 9 ‘train the trainer’ events in Scotland. From these, locally based practitioners delivered the Reflective Workshops and promoted the FFT resources within their own geographical areas (across Scotland and England). The FFT team was invited to deliver the same workshops in Australia, Ireland, and USA. The team also facilitated 25 peer-support groups in Scotland and Australia which again have been embraced by organisations (e.g., SWiSS foster care, Wesley Mission) as a means of supporting staff and carers. One of the original FaCS resources, the Staff Handbook, was translated into Finnish for use by residential and foster carers. To date, impact has taken place across five key areas:

Reconceptualising food and care

A) FFT has been instrumental in transforming thinking in relation to food and care and decisively shifting the practice focus from simply nutrition ( S1, S6) to broader concerns including:

The relationship between food and power/control: *‘It [FFT] made me think of all the times that we’d been in places where food is locked away … I did that, I was part of the power and control and feel terrible about it.*’ (Josef, Residential Worker, Australia, questionnaire)

Personal histories and meanings of food: ‘I need to think more about how the experiences she has had in the past have affected her eating habits and how she is now.’ (Jess, Foster Carer, Scotland, questionnaire)

**The role of food in making and testing relationships: ‘it [now] makes sense of why she won’t eat with us. It’s all so new for her.’ (John, Residential Worker,* Australia , questionnaire )

Food as a means of resisting and expressing care: ‘ Knowing what he likes for tea or in his packed lunch and making sure that’s what I give him can be a small but massive way to show I care about him. It’s so obvious but I hadn’t thought about it before [FFT]’. (Allie, Foster Carer, Scotland, questionnaire)

B) From the evaluation data, individuals reported that by embracing the FFT philosophy and training, staff and carers moved from fixed food routines and procedures to more flexible, child-centred approaches. They reported staff and carers’ increasing awareness of messages communicated via food, shifting concerns about healthy eating and eating disorders to a broader awareness of the relational and emotional aspects of food practices ( S1, S8, S9). How easily the FFT recommendations could be implemented, and how they result in significant positive changes, came as a revelation to practitioners:

‘ … it [FFT] made me think about food differently ... when their (children’s) food habits change just even slightly it can tell you a lot. I wouldn’t have picked up on that before.’ (Evaluation interview - Foster Carer, Scotland, S2)

C) FFT has impacted the monitoring and evaluation of care more broadly. By encouraging professional and regulatory attention on food, far greater insight into the overall care experience has been gained. One of the key findings from the FaCS study showed that knowing about food practices says a great deal about care practices. This has influenced the content of assessments and recommendations made about children’s lives:

‘[as a result of FFT] I find myself picking up on observations made about food practices when I’m in the child’s place of residence. I find myself having regular conversations with children/young people about food routines where they live which leads to me building up a picture of their world.’

(Testimonial - Children’s Safeguarder *similar to the role of Guardian ad litum, Scotland, S8)

In summary, bringing to light the power of everyday practices around food was central to the impact of FFT in changing the food practices of carers. FFT resources have been adopted by key national foster and residential providers in the UK and Australia (e.g. Aberlour Child Care Trust who support 7,500 children in Scotland, Berry Street, with 1,212 staff and 28,284 service users across Australia and Foster Care Associates who care for 1,853 children across the UK) where food was rarely considered as having such symbolic and transformative power ( S7, S8): ‘It [FFT] really did open up my eyes to how we use food in residential care, and it made me think … why do we keep the biscuits in the school office locked?’ (Stage 2 Evaluation Interview - Service Manager, Scotland, S2)

Transforming practice and care

To date, approximately 450 practitioners and carers have taken part in either a FFT reflective workshop or a peer support group (primarily in Scotland and Australia), which aimed to raise awareness of using food as a way of connecting to others and building relationships with them:

‘After attending the FFT workshop, we recognised as a care team that we were missing opportunities to connect with the young people using food. … We had some progress with one of our young people … This particular young person also joined us around the table for dinner which has not happened for quite some time. The FFT training has really opened our eyes to seeking opportunities that we can use food to build relationships and connect.’ (Evaluation Interview - Residential Worker, Australia, S2)

Evaluation data gathered after such training ( S1), 6 months and 3 years later, identified concrete areas of practice that have altered as a consequence of FFT. Analysis of data from the self-complete evaluation with practitioners and carers in Scotland, England, Ireland and Australia (n= 318), identified a range of instrumental, behavioural and conceptual changes produced by FFT, enabling respondents to build stronger emotional and relational connections with the children they care for. We clustered these into key themes including reported shifts in understanding of what was ‘behind’ or causing children’s food behaviour (65 respondents); changes to how they talked with children about food (60 respondents); changes to routines around food (53 respondents); changes to how they use food to communicate more consciously (28 respondents); confidence in tackling food issues (58 respondents) and change to how they supervise or support staff/carers (30 respondents). Significantly, 94% of respondents ranked the FFT materials as ‘Highly’ or ‘Extremely relevant’ to their practice ( S1).

Changing how carers are trained and supported

By creating interactive and reflective work and personal based resources, FFT has innovated the ways in which residential and foster carers are trained and supported. Much of existing training has focused on general principles where FFT encourages participants to not only bring the experiences of their children to the sessions but also their own personal experiences of food and food practices. This model has been picked up by others working in this area. For example, colleagues at Monash University in Melbourne run an online module based on FFT as part of a training programme rolled out across Australia ( S7). Similarly, the Health Inequalities, Food In Care project (which aims to offer support and practical tools to direct carers of children in looked after care across England - www.foodincare.org.uk) has embedded FFT within national training and support materials ( S4) in the ‘food as a symbol section’. They conducted an evaluation project ( S4) which identified that carers had found FFT more helpful than training around nutrition. As one participant said: ‘ If I knew earlier what I know now after completing the (FFT) course, the foster placement that I had last year and another which had broken down after six years would probably still last’ ( S4).

More locally, four local authorities in Scotland have reported ways in which they have included FFT in their preparation for adoption training (e.g. S6) and support and training for kinship and foster care (e.g. S5, S9).

Impacting policy and practice standards: embedding FFT into inspections

The Scottish Care Inspectorate (SCI), responsible for the monitoring and evaluation of all residential services for children in Scotland, inspect all children’s homes (308) and residential schools (62) (totalling 1,477 ‘looked after and accommodated’ children and approximately 8,503 staff). FFT highlighted the lack of consideration of food practices in the quality indicators used by SCI national inspectors. Following invited presentations and discussions of FFT with the SCI, they included food practices in the monitoring and evaluation tools for residential services for children ( S3). FFT also influenced SCI policy on the need for inspectors and care homes to consider the unconscious, symbolic meanings of food (SCI Quality Framework June 2019: 23):

‘Using the food practices resources can contribute to a better understanding of the more intangible aspects of care and the ethos of residential services for children and young people. These resources have now been placed on the Care Inspectorate’s Hub, which provides access to a range of resources aimed at supporting improvement in the social care and social work sectors through the use of research skills and evidence-led practice. This information will therefore be available both to the inspection team and to others with an interest in providing the highest quality care to Scotland’s children. References to food practices and a link to the research and associated tools is also contained in the Care Inspectorate’s ‘Quality Framework’, which forms part of a new approach to scrutiny, self-evaluation and improvement support.’

(Testimonial - Inspector/Business Analyst, Children & Young People Team, SCI, S3)

The FFT website and resources appear in the ‘Scrutiny and Improvement Toolboxes’, part of the SCI national policy document, as well as in the HUB on the SCI website as part of Best Practice Guidance ( S3).

Impact in the wider caring community

Initially FFT was designed to target foster and residential carers in Scotland. Reports of change in practice ( S1) have extended to a far more interdisciplinary group than anticipated, including adoptive parents, social workers, service managers, nutritionists, nurses, cooks, policy makers, academics, and adult care support staff. Metrics data show FFT materials downloaded across the globe including UK (4,120) USA (1,192), Australia (647) Brazil (396) and India (199). By October 2020 the site has been visited by 9,367 people (Nb. no data was recorded between May-Sept 2020 due to technical problem with IRISS platform) ( S10).

Scotland and other European jurisdictions face acute issues of rising prison numbers and mass incarceration, often accompanied by overcrowded and inhumane prison conditions, high levels of staff stress and absence, and spiralling costs. Scotland has one of the highest incarceration rates in Europe. Authorities are looking to understand better how people can be effectively monitored and supervised in the community, rather than sentenced to prison or remanded in custody. Towards this purpose, our research has helped to inform and change tagging laws and practices.

From 2015-2020, our research has informed and influenced key decision-making and reform of legislation, policy and practice on EM in Scotland – and, significantly, was among the few pieces of academic empirical evidence on EM used by the Scottish Parliament in this law reform process. Our research ( R1, R2, R3, R4) offered the Scottish Government and Scottish Parliament empirical evidence of how EM can be used in community sentences (diversion from prison) and as a form of early release from prison (decarceration), to inform law reform and practice development.

Management of Offenders (Scotland) Act 2019

For two decades, use of electronic monitoring had not substantively changed in Scotland until the Management of Offenders (Scotland) Act 2019 passed in July 2019. The EM components of this Act are informed by our research findings and recommendations, which are reflected in those of the EM Expert Working Group, and our research is named and repeatedly cited ( S1, S2, S7). For the first time in Scotland, the Act enables the introduction of new EM technologies (GPS location monitoring, alcohol monitoring), and the prospect of EM integrated with supervision and rehabilitative activities for people on a Community Payback Order. By December 2019, the Scottish Government reported ‘significant increases’ in courts imposing EM and ‘the numbers of electronically monitored community sentences are now at the highest level ever seen in Scotland’. The Scottish Government procured a GBP40,000,000 new contract with an EM service to introduce the new tagging technologies required by this Act and expand the use of EM as an alternative to custody. In October-November 2020, the Government laid two new pieces of secondary legislation (regulations, i.e. The Electronic Monitoring (Relevant Disposals) (Modification) (Scotland) Regulations 2020 ) before the Scottish Parliament to activate parts of the 2019 Act. These Scottish Statutory Instruments were approved by the Parliament enabling the re-introduction of EM as an option with bail, along with updated practice guidance. In 2020, the longstanding issue of remand in Scotland was exacerbated by the pandemic and, by October, people on remand accounted for approximately 27% of the Scottish prison population due to a COVID-19-related court backlog. EM Bail was presented as an option to try to reduce such a high use of remand in custody: this only became an option because of the passing of the 2019 Act and our research that informed it.

The Act introduced the following EM measures, informed by our research recommendations from R2, R3, and R4, which were embedded in the final report of the EM Expert Working Group ( S2), which formed the basis for the Act. The measures were further informed by written and oral evidence provided by Dr Graham to Scottish Parliament Justice Committee on the Bill ( S4, S6).

• It enables GPS tagging to be used in ways which simultaneously prioritise proportionality alongside victim and public protection in particular cases and types of risk (e.g., domestic abuse; sex offending; or restrictions away from places and at times associated with crime).

• The Act adds the option of an electronic monitoring requirement to the Community Payback Order (the most common form of social work order), meaning that EM can be better integrated with supervision and support and, potentially (if assessed as appropriate), can be used in hundreds or thousands more cases as an alternative to custody.

• The Act also enables EM to be added as an option to bail to try to reduce Scotland’s high use of remand in custody.

• Breach criteria and responses to (non)compliance have also been clarified and improved by this new law and practice guidance.

In summary, the new Act has permanently changed electronic monitoring in Scotland.

When the Act was successfully passed into law in June 2019, the Scottish Government issued a press release (S8) in which Dr Graham was quoted alongside Cabinet Secretary for Justice Humza Yousaf, with the Government describing her as ‘a criminologist at Stirling University and leading electronic monitoring researcher whose work informed the Management of Offenders Bill’, now an Act. Dr Graham was the only named researcher in the press release, underlining her contribution to the process of reforming EM in Scots law.

Impact on public policy prior to the Act

HM Inspectorate of Prisons for Scotland and HM Inspectorate of Constabulary for Scotland name the 2011 EM evaluation research ( R5) as having informed their 2018 joint Inspectorate review report ( S5) on using EM as a form of early release from prison on licence (Home Detention Curfew). The joint review and practice recommendations were requested by the Cabinet Secretary for Justice and scrutinised by the Parliament Justice Committee in deliberating the Management of Offenders (Scotland) Bill, alongside a Government review ( S7) extensively citing our research.

The contribution of our research to the Scottish EM law reform process began with the Scottish Government’s Expert Working Group on Electronic Monitoring (2014-2016). The Expert Working Group on EM commissioned and published our report ( R4). In launching our report at a national conference on EM in August 2015, then Cabinet Secretary for Justice Michael Matheson featured it in a keynote speech and Scottish Government press release ( S1) saying: ‘I welcome these findings which is an important step forward in shaping the future of electronic monitoring in Scotland… This latest research and today’s conference is another step forward in evidence gathering.’ In 2016, the Expert Working Group Final Report ( S2, 2016: 4) to Scottish Ministers states: ‘the Working Group sought to envision a new strategy for EM use in Scotland… this envisioning was guided by a review of international research evidence on EM specially commissioned by the Scottish Government, and by the findings of an EU funded comparative research project on EM in which Scotland took part’ (that is, R4, R2 and R3), with our research cited extensively throughout their report (pages 3, 4, 13, 22, 23, 25, 26, 29, 32, 49, 53, 58, 60, 63). Their recommendations included and affirmed our research recommendations: introducing new technologies (e.g., GPS), proportionate and tailored uses that better integrate EM with supervision and support, clarifying and improving responses to breach and compliance with EM, and awareness raising about EM. In a 2016 press release about advancing EM in Scotland, the Cabinet Secretary for Justice Michael Matheson acknowledged our research evidence and accepted the Expert Working Group’s report recommendations and new strategy in full ( S9), with Dr Graham quoted alongside the Cabinet Secretary, with the Government describing her as the ‘electronic monitoring researcher who carried out the studies’ with Prof McIvor. These formed the basis of the relevant section of the Government’s Management of Offenders (Scotland) Bill/Act.

In March 2018, Dr Graham submitted written evidence based on our research, and in April 2018, she was invited to give oral evidence as an expert witness to the Scottish Parliament Justice Committee about the Management of Offenders (Scotland) Bill ( S4). Our EM research was also cited in evidence submissions by other agencies, for example, Community Justice Scotland and the Criminal Justice Voluntary Sector Forum. In 2019, our evidence on EM was cited several times in the Justice Committee Stage 1 Report on the Bill ( S6) published in 2019, and in a speech by an MSP in Parliamentary debate on the Bill in February 2019. In April 2019, a Scottish Government Justice report ( S7) reviewing the evidence on EM and home detention was published, to inform discussions of the Bill and practice reforms at the Justice Committee; it cited our Scottish and European EM research extensively.

Our Ongoing Impact Development

Our EU comparative and Scottish research ( R1, R2, R3) has been recognised by the European Commission Directorate-General of Justice ( S10) as ‘the first comparative empirical study’ of its kind in Europe. They said this ‘useful’ and ‘highly relevant’ ‘project has improved knowledge and understanding of electronic monitoring in Europe, and has certainly identified best practices which can easily be transferred to other Member States.’ Our research evidence and expertise has helped frame media and public debates (with 30+ news media reports in Scotland) and inform major European events and expert groups on uses of EM in criminal justice.

We have achieved impact on changing practice in professions, business, and building design through training and education, consultancy, accreditation of designs and products, business development and activities in outside spaces. Through these actions, we ensure our inclusionary approach is embedded in design thinking for dementia. Our national and international reputation as the ‘go-to’ place for advice on design for dementia provides an important pathway for our research impact to achieve its reach. Media reports of newly opened and refurbished care facilities for people with dementia regularly cite the University of Stirling’s guidance in their design (Nexis search to 31^st July 2020 identifies 333 verified results, see S1). Research by the University of York (Buildings in the Making study 2019, see S2b) found architects widely knowledgeable about and using our guidance ( S2a). The Iris Murdoch Building (IMB - completed 2003), to our knowledge the first public building worldwide to be built on dementia-friendly design principles, houses the Dementia Services Development Centre (DSDC), which exists to translate research into impactful activities, and is staffed by architects, designers, and trainers.

Improving professional practice through training and education

Our research (e.g. R1) has been used extensively in training qualified architects in dementia-friendly design, mainly of care environments. Training is led by a professional architect, Lesley Palmer, who holds a knowledge exchange post in DSDC. During 2014-2020, 446 ( S3) professionals have attended the training, including architects, interior designers, construction companies, care home companies, project managers and others. This includes 116 internationally based professionals from 21 countries. They report changes to their practice in a range of contexts as a result of our training ( S4). A company team leader states they are ‘using the knowledge to better inform the architects and designers I work with about the best practice when designing for dementia’ ( S4). A museum developer reports ‘I was able to look at every aspect of the design and fit-out plans in detail and make confident decisions on how to put theory into practice in our unique space’, affecting 2,500 visitors (people living with dementia and their carers) per year ( S4). A planner in a large Northern UK local authority explained they were now making ‘towns and developments more accessible and more attractive to those living with dementia, ensuring that pedestrianisation schemes are more inclusive’ ( S4).

Our research is regularly integrated into practitioner education through our Dementia Studies postgraduate programmes (291 graduates 2014-2020) ( S5a). Students taking these programmes generally work in dementia care, and cover a range of professions including medicine, nursing, allied health professions, care work and social work. Professionals who have completed their studies report how their practice has been enhanced: ‘(I am) able to effect policy matters at government level’ (India); ‘I was able to discover a whole new world of knowledge’ (Canada) ( S5b).

Embedding dementia friendly design through consultancy

We provide consultancy nationally and internationally ( S6a): Since January 2014, we have provided input to design work for 312 clients (including 28 outside the EU: in Australia, Cambodia, India, Japan, New Zealand, Singapore, and the USA). We calculate from our records of clients that the resulting designs and redesigns have benefited at least 8,000 people through design modifications to the environments in which they live, as clients have implemented the advice ( S6a). Examples include working with Mediva, Japan to design the Grancreer Setagaya Nakamachi (Tokyo) and the Yokohama Tokaichiba Creer Residence. Both these developments include senior housing and care facilities for 75 and 91 people respectively. In Australia, we worked with Life Care Australia on its Gaynes Park Manor ( S6b) development in Joslin, South Australia, which includes a 96-bed nursing care home. This facility won a prestigious award for its architecture at the 6th Asia Pacific Eldercare Innovation Awards 2018.

Driving good practice through accreditation

We provide care facilities and other buildings with research-informed assessments of their dementia-friendliness using our Design Audit Tool ( S7a & S7b). Clients modify their buildings using our guidance and can receive our ‘Gold Award’ once they meet all the guidelines: 27 have been awarded 2014-2020, including 3 in Japan and 1 in Australia ( S7c). Our DSDC consultants were involved throughout the design and building process of the Great Sankey Neighbourhood Hub in Warrington, a community facility incorporating sports pitches, fitness studios, a swimming pool, a spa and a library. In January 2020, we accredited the Hub with a Gold Award, with the Council Leader reporting ‘Our plans for Great Sankey… place dementia-friendliness at the heart of the facility… receiving this recognition from the University of Stirling is great news’ ( S7d).

Using our evidence regarding the effectiveness of design features e.g. R3, DSDC operates a product accreditation scheme ( S8a), whereby companies can have their products assessed for dementia friendliness. Through this process, our research evidence is directly informing the fixtures, fittings and furnishings which are used in care facilities, nationally and internationally, through its influence on commercial operations. The accreditation is done by a team in Stirling, including our architects and designers and Gibson provides embedded research expertise, drawing on his work on user choices ( R4). In the REF period, 929 products and product ranges were accredited ( S8b). Clients include Arjo (a major supplier of medical equipment), Altro, Forbo and Polyflor (flooring), Gradus (interior design), Wallgate (washrooms and sanitaryware) and Grayson (clocks), all of which are suppliers to care environments ( S8b). Companies report that the accreditation supports them to advise clients on the best products to use in environments where people with dementia will be living, enhancing their commercial offering, market status and bottom line. For example, Polyflor state ‘DSDC accreditation is a further example of Polyflor as a manufacturer highlighting its commitment to the health and care sector’; and Gradus explains ‘The knowledge we have gained from DSDC has helped improve our service, recommendations and new product development process’ ( S8b).

Improving design through business development

The design principles, closely informed by our evidence appraisal ( R3), have been incorporated into an app, IRIDIS (see S9) , developed in partnership with a national construction technologies innovation company, Space Group Ltd. This collaboration has generated (2020) a spin-out Joint Venture business development, showing commercial impact. IRIDIS enables existing buildings and buildings in the process of design to be assessed for dementia-friendly design, and suggests what modifications may be needed. Launched in 2017, IRIDIS is designed to be used, free of charge, by carers, occupational therapists and people living with dementia to optimise homes, for example by making changes to decoration, soft furnishings or floor coverings. IRIDIS has been installed 2,608 times since launch ( S9a & S9b), including international users in Australia, Canada, New Zealand, and USA. In 2018, IRIDIS won the 'Best Collaborative' category at the Blackwood Design Awards - an international contest held to ‘discover and champion brilliant concepts, designs and products that have the potential to help people live independently’ ( S9c). IRIDIS is now developing more sophisticated applications for care professionals and architects.

Supporting the use of outside spaces by setting standards

Our work on outside spaces with Paths for All (PfA) has delivered tangible impact for people with dementia across PfA’s whole area of operation ( S10). Throughout Scotland, PfA are using a Dementia Friendly Walking Accreditation directly derived from our research as a tool to ensure good practice. To achieve accreditation, walks must demonstrate they meet criteria derived from our research including involvement of people with dementia in developing the walks, training walk leaders in dementia friendly practice, and partnering with organisations to ensure knowledge of the walks for people living with dementia. Since 2015, 31 Walking for Health projects have achieved the accreditation and are delivering over 145 Dementia Health Walks every week, supported by 579 specially trained Volunteer Walk leaders. The PfA outcome has been identified as an exemplar case study to support promotion of international standards in support for people with dementia by the ISO (International Standards Organization) Technical Committee for Ageing Societies, of which Robertson is a member ( S10).

Our work “ contributed to [OSCR’s] strategic objectives of ‘helping the public to have more confidence in charities’ and ‘ensuring that registration and reporting is straightforward and proportionate’” [S1, Head of Casework, OSCR].

Our research underpinned the Scottish Charity Regulator’s (OSCR) regulatory process improvements; increased their regulatory efficiency; reduced the administrative burden on 24,500 Scottish charities; and developed their conceptualisation and operationalisation of risk from 2014 to the present (figure 1).

Our work made a significant contribution, as “ without the work carried out [by] the University of Stirling, [OSCR] would not have had the skills or capacity to carry out such in-depth analysis to further our understanding of risk in charities” [S1].

Figure 1: Impact Model, showing how the research informed the linked impacts

Impact: Moving to risk-based processes has streamlined the regulator’s work, allowing more efficient use of the regulator’s limited resources.

Based on our work, OSCR has created a dedicated Risk Team [S1] with staff from across the organisation, reflecting the importance of combining data across the organisation [R2, R3] in evaluating risk. Using our research data and indicators, this team have developed and piloted a Risk Assessment Tool [S5].

“Most significantly, the collaborative work contributed to the development of a dedicated Risk Assessment Pilot Team and Risk Assessment Tool. The tool features key fields identified in the research programme, such as source of evidence, as relevant to predicting case outcomes.” [S1]

Figure 2: OSCR Risk-based case classification[S7]

Our insights [R1, R2] allowed OSCR to use their limited resources more efficiently to focus on risk in the charitable sector. Following the introduction of risk-based triaging in 2017, the percentage of investigations resulting in a “Not Appropriate” outcome fell from 50% to less than 20% (Figure 2, [S7]), allowing OSCR to focus their resources on the riskiest concerns within their remit.

“[A] real change that resulted from [the research] has been the redeployment of resources and the creation of ‘Not Appropriate’ case files when the subject of a reported concern is not within our regulatory remit. This is a much more streamlined area of work which allowed more focus on the complex cases.” [S4, Engagement Manager, OSCR]

Impact: The Scottish Charity Regulator has changed their complaint-handling processes to be more targeted, based on our models of risk.

Our research [R1, R2] changed OSCR’s complaint prioritisation and the decision-making about which complaints should lead to full inquiries and regulatory action. We showed that some types of charity were much more likely to receive complaints that were not substantiated, and that the source of the complaint was a good predictor for the likelihood of regulatory action. This allowed OSCR to streamline their processes and better prioritise their responses to the complaints about charities that they received.

The effect of targeted regulation can be seen in the investigations and enforcement actions that OSCR undertakes. Measuring the direct contribution of our research on these changes is challenging. However, using our insights, OSCR now takes more consideration of the source of a complaint in assessing the appropriate action. Table One, below, shows the percentage of complaints that resulted in some form of regulatory action by the source of that complaint, comparing the period 2008-2014 to 2015-2018. It shows that following our research [R1], complaints about charities from professionals (e.g. auditors, accountants, etc.) increased in their likelihood of resulting in regulatory action, while the regulatory response to anonymous reports fell, and public reports was stable.

Table One: Proportion of Complaints Leading to Regulatory Action by Source [S6]

Source: Internal OSCR data on complaints and investigations (2019)

Compared to the proportion of complaints acted on pre-2015, this would suggest that post-2015 an additional 14 regulatory actions were undertaken based on Professional complaints, and 14 fewer regulatory actions on the basis of Anonymous complaints. This is not a causal analysis of a change in the regulator’s behaviour, but reflects the process of moving to a data-driven risk-based approach to regulation that was strongly informed by our research. This is confirmed by the Head of Casework for OSCR:

“That [R1, R2] has been really quite influential over the past 18 months where we have been trying to operationalise that targeted-regulation, in terms of how we go about inquiries and the risk assessment phase. It’s influential in two ways. One is what kind of weight we give different sources of intelligence, […]. That’s then coalesced into thinking about how we deal with the journey for people who raise concerns with us, and about the purpose of our inquiries.” [S3, Head of Casework, OSCR]

Impact: The Scottish Charity Regulator has developed their conceptualisation and operationalisation of risk, connected to the academic literature.

Our research [R1, R2, R3, R4] and the collaborative working supported OSCR in developing their conceptualisation of risk, and how it could be operationalised in their work. In particular, in setting up a cross-department risk team and tool [S5], they moved from a focus on financial measures of risk to measuring risk more broadly with the data they have available.

“Robust analysis and insight from an academic perspective helped us identify and evidence priorities for information gathering and targeted monitoring that gave us confidence to move to targeted regulation.” [S1, Head of Casework, OSCR]

Our research has underpinned the building of evidence-led practice within OSCR, with strong participation in knowledge exchange and data sharing across the organisation. We developed this through regular meetings, presentations, workshop participation and practitioner reports.

“[University of Stirling] kept up regular visits to talk to SMT members and business managers about the research findings. The practitioner reports are incredibly helpful. I look at [the research] sometimes so that we can pull out directions of what we’ve learned, feed that into things when we are reviewing the risk framework, say ‘what have we got on this’.” [S4, Engagement Manager, OSCR]

As well as specific examples of modelling risk, our work [R1, R2, R4] allowed OSCR to critically reflect and refine their rationale for collecting and linking data about charities, and consider an evidence-based approach to assessing the impact of regulatory actions.

“It’s been really useful to have external perspectives drawing on our information, acting as a critical friend, doing a ‘deep dive’ into the data. It’s really enriched the scope we’ve had to think about the practical application of our risk-led approaches, and it’s allowed us to test some thoughts and hunches that we had with more rigour behind it.” [S2, Chief Executive, OSCR]

Impact: By prioritising the data to be collected, the annual reporting burden on charities has been reduced.

Charities (n=24,500) must submit Annual Returns to OSCR. There is a difficult balance between collecting the data needed to measure risk, and over-burdening charities with information requests. Using our research [R1, R3], OSCR now reviews the data they collect, and the way they use that data to measure risk. Collecting the right data improves the efficiency of the regulator and reduces the reporting burden on charities.

“We’ve had to consider are we collecting the right information […] in our Annual Returns and other sources. In some cases we realised that some of the data we were collecting was of very little value, so we’ve reduced that reporting burden on charities. Sometimes we’ve started collecting new data.” [S2, Chief Executive, OSCR]

We [R1] considered the key performance indicators that OSCR were using to measure risk, and how predictive these were of the negative outcomes that the regulator was concerned about. We identified which indicators were the more targeted measures; this underpinned the review of the risk measures used by the regulator and the data which was to be collected from charities in the Annual Return, described by OSCR’s Engagement Manager:

“On the financial vulnerability and indicators, there were things we had a hunch were not working well […]. Now we have made changes to the Annual Return, we have the benefit of the completed research [R1, R2, R3], it has really helped us think about how we use information, how we keep data, and how we structure the Annual Return.” [S4, Engagement Manager, OSCR]

Using data from across the organisation, including financial information; risk indicators; complaints and investigations; and serious incident reporting, we explored how these disparate data sources could be combined to better understand different forms of risk in the charitable sector, and OSCR’s engagement with charities. Our research [R1, R2, R3] allowed OSCR to understand the relative value of different data sources, and informed their decisions about how to collect the right data more efficiently, in order to inform their casework:

“That aspect of the research [R1, R2, R3] has been really useful in getting us thinking about how we treat these things, how do we weigh the concerns that come through the door. The essence of being risk led is having the confidence in your perception of risk and you can’t do this unless you have the data and a useful analysis of that to back it up, otherwise you’re just going on instinct. We were collecting a lot of data, but it wasn’t giving us what we needed in terms of analysable data. We’ve streamlined the data collection, we can get a snapshot of what’s going on at any time.” [S3, Head of Casework, OSCR]

Finally, the overall value that OSCR has placed on our work is further evidenced by the fact that “ the findings have been discussed with other UK and international regulators to share our [OSCR’s] experience on the subject matter of risk” [S1, Head of Casework, OSCR] .