Impact case study database

The background: Exit polls at UK General Elections since 2005

The major UK broadcasters fund and organise an election-day exit poll, in order to inform their coverage (on TV, radio and internet) on the night of the UK General Election, a time when current-affairs broadcasting attracts large audiences.  For the election in 2005, David Firth as consultant for BBC developed (with John Curtice) a completely new method [3.1] for the exit poll, to improve the accuracy of the headline prediction made at 10pm on election night.  The method proved so successful in 2005 — with an exact prediction of the 66-seat Labour majority — that it was adopted jointly by all of BBC, ITV and Sky for the following election in 2010.  After a similar success in 2010, the method has been used jointly by BBC, ITV and Sky at all subsequent general elections (2015, 2017 and 2019).  Corroborative statement [5.1], written after the 2010 election by the then Editor of BBC Political Research, confirms the pre-2015 background and emphasises the importance of the new method to the broadcasters. By 2010 Firth had moved on to other projects, and he passed the operation of the exit-poll method and associated software to Jouni Kuha (LSE), who replaced Firth as the statistician in John Curtice's election-day team at every General Election from 2010 to date.

Fresh impact in 2015–2019

The impacts described below are of two main kinds:

1. Impact of the underpinning research in conveying timely and accurate information to a large worldwide audience, via conventional media (TV, radio, press) and the internet. While this kind of impact was previously generated in connection with the General Election of 2010; the impact between 2015 and 2019 is around three times as big since it relates to three further elections.

2. Additional impact via the public track-record of accuracy of the new exit-poll method since 2005, in the form of substantially increased trust in the exit poll and the 10pm election-night prediction that comes from it. By 2019 the 10pm exit-poll prediction was presented (by broadcasters and other media outlets) without the scepticism that had routinely accompanied such predictions previously (in 2005 and 2010 for example); and even politicians now typically accept the exit-poll prediction as a reliable indication of the election outcome.   Moreover, the world's financial markets also now exhibit essentially 100% trust in the accuracy of the exit poll (whereas previously, even as recently as 2010, the markets' response at 10pm to a surprising but ultimately accurate exit poll was rather more muted because exit polls back then were not trusted to be accurate).

The Curtice-Firth methodology developed in the underpinning research [3.1] was used unchanged, by a team of academics working jointly for the three major broadcasters (BBC, ITV, Sky), to design and analyse the exit polls for the three General Elections in 2015, 2017 and 2019. The corroborative statement in [5.2], written in 2020 by Professor Jouni Kuha who is the statistician in the exit-poll team led by Sir John Curtice, fully confirms this. Kuha states straightforwardly: "The methodology of the exit poll in all of these years has been the one which was developed by David Firth and John Curtice for the 2005 election, as described in Curtice and Firth (JRSSA, 2008)." Kuha's statement letter also comments on the remarkable accuracy of exit-poll predictions in the five elections between 2005 and 2019, and states that "This success would … have been impossible without the current exit poll methodology."  The article in [5.2] by Curtice, Kuha and other exit-poll team members, published after the 2017 election, gives a more detailed account.

The broadcasters (BBC, ITV and Sky) disseminate their exit-poll prediction to the whole world at 10pm on election day via TV, radio and the internet.  In the UK alone, the number of BBC1 viewers at 10pm on each election night exceeds 6,000,000, according to reports [5.3] on the 2015, 2017 and 2019 elections from the BBC and Press Gazette; the corresponding number of viewers on ITV and Sky is reported to be 1–2 million.  The exit poll's global reach via the internet is not measured but is very substantial: the exit-poll prediction is picked up immediately by news media across the world, and by global financial markets.  The background letter [5.1] indicates the high value that the BBC and other broadcasters attach to the exit poll.  According to paragraph 29 of the 2018 report of the House of Lords Select Committee on Political Polling and Digital Media (in **[5.3]**), the cost of running the exit poll at a General Election is around GBP300,000 — a figure which further corroborates the high value that the broadcasters attach to the exit poll.

Interest in the exit poll from the general public is strong (as indicated by the quoted audience figures, for example), and there is evidence that the public at large now appreciates that modern-day ("John Curtice") exit polls are much more reliable than pre-election polls.  Evidence [5.4] collects some representative social-media posts from members of the public in December 2019, e.g., " There is only one poll to trust and it's the exit poll".  From these social media posts, it seems also that the exit poll influences individual behaviour, for example the decision to stay up on election night: typical tweets include " No point - exit poll is always right", " I didn't sleep a wink on referendum night. Wouldn't have missed it for the world. I don't bother on General Election nights though. I trust John Curtice enough to watch the exit poll and get off to bed" and " JC's exit poll is normally bang on, at 10pm tomorrow I'll know whether to go to bed early".

To support wider understanding of the exit poll and its methods, David Firth created (in 2010) the public-facing "Exit Poll Explainer" page at warwick.ac.uk/exitpolling — which sees thousands of visits around the time of an election (access statistics [5.5] show more than 15,000 distinct visitors in the week of the 2017 election, for example).  Sir David Spiegelhalter, then President of the Royal Statistical Society, highlighted the Explainer page along with this public comment [5.6] immediately after the 2017 election: " Clear outright winners of this election (and others): statisticians who designed the exit poll and its model".  In his 2019 popular book The Art of Statistics, Spiegelhalter even highlighted Firth's work on exit-poll methodology as the book's final chapter " An Example of Good Statistical Science".

In all three of the most recent elections (2015, 2017, 2019) the exit poll's prediction was markedly different from what had been expected from pre-election polls (as indeed it had been also in 2010 and 2005).  Whereas in 2010 the surprising exit poll was downplayed by TV broadcasters and on-air pundits (as noted also in **[5.1]**), by 2017 its track-record of accuracy had forced the commentators and politicians to accept the 10pm exit poll as a reliable prediction of the election outcome.  Evidence [5.7] collects some media items which show this.  Even in 2015, former LibDem leader Paddy Ashdown rubbished the exit poll in his interview on BBC1 and famously said he would eat his hat on TV if his party lost the large number of seats that the exit poll had predicted (their actual loss was two seats more than the prediction).  By 2017, though, the tone had changed markedly towards acceptance of the exit poll's accuracy.  In her BBC interview a few weeks after the 2017 election, PM Theresa May admits to having been "shocked...devastated" upon hearing the exit poll that signalled the loss of her House of Commons majority.  By 2019 we even see John McDonnell, Labour's shadow chancellor, conceding the election on BBC1 just 15 minutes after the 10pm exit poll has been announced.  Whereas in 2015 The Independent still asks " Just how did the exit poll manage to get it so right?", in 2019 The Economist reports simply that " Britain's exit poll has an exceptional record".

In the peer-reviewed paper [5.8], written after the 2017 election and published in one of the most prominent international journals of political science, the evidence just described is solidified through formal text-analysis of election-night broadcasts across UK general elections since 1955.  Paper [5.8] provides many new insights, and a key conclusion is that " The remarkable accuracy of the 2010 exit poll, and the subsequent success of the 2015 and 2017 exit polls, have been associated with a growing focus on them in the early part of the broadcasts."

Evidence [5.9] collects clippings about the exit poll in relation to financial markets, clippings which show the markets' increased trust in the 10pm exit poll prediction. In 2017 a CNN Money article stated: "The exit poll has typically been much more accurate than Britain's notoriously shoddy opinion polls".  A 2019 pre-election article from Pound Sterling Live quotes Samuel Tombs (Chief U.K. Economist at Pantheon Macroeconomics): "Abandon all your priors when the UK exit poll is released at 22:00 — it has a very good track record since its current methodology was introduced in 2005".  A Financial Times online article published shortly before close of polls in 2019 assesses the increase in trust directly, and states: "The exit poll's recent run of prescient results has meant that traders are putting increased stock in its predictive power, with the pound fluctuating more aggressively following the poll's release from one election to the next."  The sequence of graphs shown below amplifies the message of that Financial Times article; it shows the GBP-USD exchange rate, through election nights 2010, 2015, 2017 and 2019 (using data from forexite.com).  In each panel the red line is the average price in the half-hour before the 10pm close of polls, while the green line is the average for the half-hour after 10pm.

In 2010 the 10pm fall in Sterling was only a relatively small fraction of what the currency would ultimately lose by the next morning (even though the surprising exit poll was highly accurate): just like the political pundits, the market did not really trust the exit poll by 2010.  As the accuracy of exit polls became more firmly established, though, the market's trust increased.  By 2017 the market's 10pm reaction to a surprising (but ultimately accurate) exit poll was as big as its full response to the actual result of the election the next morning.

John Curtice — who is by now very much the enduring public face of election polls in the UK — received a knighthood in 2018.  David Firth's underpinning technical contribution to the accuracy of modern exit polls has been recognised in public less formally, through mentions in media articles and occasional interviews for BBC TV and radio.  Ahead of the election in 2017 the BBC acknowledged David Firth's exit-poll work through a 1-minute feature video [5.10] which was broadcast via BBC TV news bulletins in the UK and syndicated to global media via the internet.