Impact case study database

Impact on Great Britain Capacity Market

A reliable electric power supply underpins almost all of modern life (see https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/48129/2176-emr-white-paper.pdf) – thus ensuring an appropriate level of security of supply is critical for society. This societal significance is confirmed by testimonial letters from National Grid, and from a former board member at Ofgem (the GB energy regulator) who was responsible for this area [5.1, 5.2]. The direct cost of capacity procurement (i.e. the cost feeding to consumer bills) in the main 4 year ahead capacity auctions has ranged from GBP290,000,000 (for delivery in 2022/23) to GBP1,180,000,000 (delivery 2020/21) [5.3].

The Maxwell Institute team of Dent, Wilson and Zachary have been the main academic consultants to National Grid (NG) on risk modelling and decision analysis methodology since 2011, and thus for the entire REF period. This is evidenced by the specific points of impact below. The ongoing broad significance of the decisions based on our research is confirmed by NG’s 2019 Capacity Assessment report [5.4a]: on pages 22-25, this references in broad terms the significance of the Maxwell Institute’s research-based advice (references to unnamed ‘academic consultants’ refer to us).

The reach of the impact is GB-wide. The Maxwell Institute work underpins the decision support approach of NG by the Department of Business, Energy and Industrial Strategy (BEIS) in this nationally vital area of procuring capacity to ensure electricity security of supply.

The specific items of GB impact in this study fall into two strands:

1. Choice of scenarios considered and decision analysis approach used. Our decision analysis work [3.4] was used as described in National Grid’s 2017 Capacity Assessment Report [5.4b] to confirm the choice of methodology for supporting the decision on volume of capacity to procure in 2017/18 auction, for delivery in 2021/22, an auction worth GBP423,000,000 [5.3]. The 2019/20 auction for delivery in 2023/24, with design based similarly on our work, was worth GBP699,000,000. In particular, pages 17, 39 and 87-97 of the 2017 report [5.4b] describe how our work guided the choice of scenarios (particularly extreme optimistic and pessimistic scenarios) used in decision support for capacity procurement, through the demonstration in [3.4] of how the LWR analysis outcome depends on these extreme scenarios.

BEIS had been concerned about whether sufficient account was being taken of very pessimistic scenarios of planning background by the LWR approach, and we were asked to investigate whether including more extreme scenarios with an appropriate weighting scheme could improve decision making. We showed that the outcome was not very sensitive to addition of further extreme scenarios, and thus BEIS decided to go forward with their incumbent analytical approach given appropriate choice of scenarios. The significance to BEIS of this confirmation of performance of their approach is confirmed by BEIS’s Panel of Technical Experts [5.5, page 46] who note that ‘ The analysis for National Grid by Wilson and Zachary formalises the concerns we have previous expressed about the LWR methodology’ and ‘ we were grateful that National Grid did conduct a trial run to assess the impact of adopting such a hybrid approach’.

National Grid’s 2016 Electricity Capacity report [5.4c] confirms (page 30) that our work [3.1] on the statistical relationship between electricity demand and available wind capacity formed the basis for inclusion of a ‘low wind sensitivity’ in their decision analysis, due to historic data suggesting the possibility of wind power availability typically being slightly poorer at times of highest demand. This resulted in a need for an additional 0.8 GW of other capacity to give the appropriate level of security of supply, equivalent to an extra large gas-fired power station, and at a clearing price of GBP22.50/kW giving a direct additional expenditure of about GBP18M. This is an example of where improved analysis can demonstrate the need for increased up-front expenditure in order to give an appropriate level of security of supply.

The scale of this impact on decision support is the whole monetary value of the capacity market, and the nationally significant issue of procurement of generating capacity for security of supply, in that the methodology directly supports the top level decision of volume of capacity to procure.

2. Determining how to treat energy storage in the capacity market. In 2016 it was necessary to update at short notice the approach to crediting battery storage in capacity auctions. Under the existing rules, batteries that would only discharge at maximum output for up to 1 hour had been credited with 95% of their capacity, in analogy to the nearest existing technology of pumped storage hydro (which can discharge at maximum capacity for much longer, and cover the whole duration of the early evening demand peak). Based on our work in [3.2] and [3.3], National Grid and the government changed the risk index used for valuing storage from duration-based to energy-based, so as to make a fair comparison between storage and other resources as described above. They also made the specific decision to use the marginal value of the resource, when added to the rest of the portfolio procured, to credit emerging technologies such as storage and wind energy within the calculations.

The executive summary of [5.6] states “ We also acknowledge the benefits of discussions with academic experts from The University of Edinburgh on a number of high level issues, notably in relation to the choice of risk metric and the definition and calculation of the storage EFC”, with the Maxwell Institute report based on research in [3.2] and [3.3] included as Appendix 3. Following implementation of the recommendations in this report, battery units have been credited with as little as 10% of their capacity, versus the previous 96% [5.6, page 29]. The capacity of battery and similar storage awarded contracts in the 2019/20 auction for delivery in 2023/24 is 0.18 GW [5.3, page 3 of that year’s report]. Without the revised capacity credit factors based on our research this would have been over-credited by more than 0.1 GW, a direct overpayment to storage of approx. GBP2,000,000 at the market price of GBP16/kW/year. The supply capacity available to the system to support reliability would effectively have been lowered by this amount.

More importantly than these figures for the 2019 auction is the basis our work provides in supporting inclusion of storage in the capacity market on an ongoing basis, which in turn supports national plans for increased use of storage to the benefit of consumers while maintaining security of supply. Thus the annual capacity and monetary figures will grow considerably as storage becomes more significant on the scale of the whole system.

Verification of GB impact through testimonial letters. The EMR Modelling Manager from National Grid [5.1] commented “ *What the research has enabled is the ability to quantify the statistical relationship between wind and demand at times of high demand, improving the robustness of our modelling, which had previously used an assumption of independence.*” A former Board Member of Ofgem [5.2] said “ The work of Dent, Wilson and Zachary in this field has been significant in promoting … statistical techniques…. and has informed the assessment of capacity requirements in the capacity market and the assessment of capacity adequacy”. The letters also contain their own statements of the importance of reliable electricity supply and the significance of our research in underpinning analysis supporting the capacity market.

Impact on development of IEEE Standard 859

Dent was chosen to chair the recent revision of IEEE Standard 859 on “Standard Terms for Reporting and Analyzing Outage Occurrences and Outage States of Electrical Transmission Facilities”, published as IEEE 859-2018 [5.7]. IEEE 859 is widely used in the N American industry, in particular underpinning the continent-wide Transmission Availability Data System run by the North American Electricity Reliability Corporation (NERC), which records reliability data from the majority of utilities, and by other industry organisations internationally.

According to the letter [5.8], based on the data collected under guidance of IEEE 859, “ Industry and regulators use the resulting transmission outage statistics to determine best investments, and societal impacts resulting from transmission performance”. In NERC’s own geographical scope, this supports reliable supply to nearly 400 million people. In the USA alone, the significance of the use of data collected based on this standard is shown by annual investment in electricity networks at USD45,000,000,000 annually in 2016 and on an increasing trajectory [5.9]; and the US Government estimating annual costs of weather-related outages (see point (ii) below) standing at between USD18,000,000,000 and UDS33,000,000,000 annually in the decade to 2012 [5.10]. The US Energy Information Agency explicitly credits IEEE standards including 859 as underpinning relevant statistics [5.9] (“Many of the standards in reporting these metrics were initially developed by the IEEE” – IEEE standards 762 and 859).

As well as his general leadership based on research and industry experience, Dent made a number of specific contributions based on the above research [5.7]:

1. Note after 4.2.2.2.2 “Multiple Independent Outages”: clarifies, based on insights from [3.1-3.3] above, the concept of statistical (in)dependence in circumstances where failure rates are elevated (e.g. due to bad weather). This has been a point of confusion in the power system research and industry literature for many years.

2. Note in 6.3 “Weather” and Note 1 in 6.3.1 “Adverse Weather”: clarifies how the classification of different weather states (normal, adverse etc) in the standard should be used in data recording, and how care should be taken using indirectly relevant data (again based on insights from [3.1-3.3]).

3. Note in 7.3 “State Probability Indices” clarifies the important conceptual point that the indices defined are empirical probabilities, i.e. “the proportion of a set of historic trials in which a particular event occurred”. This is closely related to the use of historic wind resource and demand data described in [3.1].

This section of the impact study is verified by the testimonial letter of the Chief Reliability Officer at the North American Electricity Reliability Corporation [5.8]. The letter further describes the international significance of IEEE 859, and the contribution of our research in underpinning the revisions in IEEE-859-2018 described above.