Impact case study database
Improved accuracy of weather forecasting internationally through advanced use of observational data
1. Summary of the impact
Weather forecasts play a vital part in our lives, with major impacts on society and the economy. Forecasts are obtained by combining observations of the weather with computational predictions using a data assimilation process. Forecast accuracy relies on accurate estimates of the uncertainty in these weather observations. Remotely sensed observations, for example from satellites and ground-based instruments such as radar, provide the most benefit to forecast accuracy, but these are the most expensive data to acquire and their errors are not well understood. Research at Reading led by Nichols and Dance has developed new methodology for estimating observation error statistics and has contributed important theoretical advances in understanding the role of observation uncertainty in weather forecasting. The Reading methods are now being used in operational systems worldwide by the UK Met Office, European Centre for Medium-Range Weather Forecasts, US Naval Research Laboratory and NASA, amongst others. Thanks to these advances, there have been significant improvements in forecast accuracy and cost-effectiveness of observations for forecasting agencies, without loss of computational efficiency. The research has also influenced strategies for system enhancement, diagnostic activities and training at these centres and has raised the quality of weather information provided to government, business and the public for decision-making.
2. Underpinning research
Weather forecasts are critical for transport, emergency services, agriculture, energy and defence sectors. National meteorological services typically use around ten million observations each day to improve global and regional predictions of weather on timescales from hours to weeks. Overall, improvement in operational weather forecast accuracy internationally increases value for money for forecasting agencies, who fund the data. Moreover, better forecast accuracy benefits society through protecting lives and mitigating economic losses as a result of adverse weather.
In weather forecasting systems, data assimilation is routinely used to improve predictions by combining forecasts from numerical model simulations with earth system observations of the atmosphere, oceans, land surface and cryosphere, in order to provide information on a wide range of variables such as temperature, pressures, winds and humidity. The assimilation problem is an enormous nonlinear Bayesian statistical estimation problem, where the model and observational data are weighted by their respective uncertainties. Forecast accuracy depends upon correctly specified uncertainties, as measured by error statistics. Better estimates of uncertainty result in better forecasts. A key research question is therefore how to characterise and treat observation uncertainty in an assimilation system.
Historically, the statistical relationships, or correlations, between errors in the observations have been difficult to estimate due to the number of variables involved and so have been neglected in data assimilation. The operational norm in the past for the UK Met Office (UKMO) and other agencies around the world was to assume that the errors were uncorrelated. Research undertaken at the University of Reading explored, for the first time, the implications of the omission of observational error correlations in weather forecasts [R1]. Defining ‘information content’ to be a measure of how much the use of observations reduces overall uncertainty in the assimilation system, the research indicated that neglecting the correlations between observations reduced the information content provided by these observations by 75%. The team then showed that inclusion in the assimilation of even a rough approximation to the true correlations can substantially increase information content and hence improve the weather forecast accuracy [R2]. In tests with a shallow water model, in which observations of fluid velocity and geopotential were assimilated, the forecast errors could be reduced by a factor of three with little additional computation time.
To further increase information content and reduce uncertainty in forecasts, the Reading team worked with the UKMO to develop efficient mathematical methods for estimating observation error correlations. PhD work in 2009 (published in 2014, [R3]) showed for the first time that diagnostic methods that were originally designed as a consistency check could successfully be used to estimate the full observation error covariance matrix, including correlation terms, for operational data. The UKMO successfully applied these methods to estimate inter-channel observation error correlations for satellite radiance data in its system. Reading researchers also made important theoretical advances in understanding observational errors and the effects of including correlated observation errors in the assimilation. Using spectral analysis, they investigated the sensitivity of the assimilation system and bounded differences in the estimated error variances and correlations due to changes in the system [R4]. High sensitivity, measured by the condition number of the problem, was found by the UKMO to cause computational failures in their assimilation system. In [R5] Reading researchers mathematically analysed the problem, deriving upper and lower bounds on the condition number and showing that high sensitivity is related to complicated interactions between the error covariances of the observational and model data.
Subsequent research at Reading focussed on methods for improving the condition number of the system and reducing the computational time required for the assimilation. Following the early work in [R3], the team adapted their methods to quantify (i) inter-channel error correlations for a range of instruments, including various satellite infra-red sounders that monitor water vapour, temperature and humidity, and (ii) spatial error correlations, for example from satellites tracking clouds and from Doppler radar tracking precipitation and winds. The application to Doppler radar winds [R6] by the Reading team led to international inter-comparisons between the UKMO and the German Weather Service (Deutscher Wetterdienst) assimilation systems. Through this research, issues affecting the assimilation of the observational data were identified and treated. These included biases in the data, inconsistencies in boundary treatment, and quality control issues in handling the data.
Research at Reading on the characterisation and treatment of observation uncertainty in assimilation systems for weather prediction has made significant contributions to the field, both theoretically and in practice. It has resulted in: international recognition of the value of accounting fully for observational errors [R1, R2]; the development and refinement of new methodology for estimating observation error correlations [R3, R6]; and important mathematical advances in understanding the role of correlated observation errors in assimilation systems [R4, R5].
3. References to the research
[R1] Stewart, L.M., Dance, S.L., Nichols, N.K. (2008). ‘Correlated observation errors in data assimilation’. International Journal for Numerical Methods in Fluids, 56, 1521–1527. DOI: https://doi.org/10.1002/fld.1636
[R2] Stewart, L.M., Dance, S.L. and Nichols, N.K. (2013). ‘Data assimilation with correlated observation errors: experiments with a 1-D shallow water model’. Tellus A, 65, 19546. DOI: https://doi.org/10.3402/tellusa.v65i0.19546
[R3] Stewart, L.M., Dance, S.L., Nichols, N.K., Eyre, J.R. and Cameron, J. (2014). ‘Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system’. Quarterly Journal of the Royal Meteorological Society, 140 (681), 1236-1244. DOI: https://doi.org/10.1002/qj.2211
[R4] Waller, J.A., Dance, S.L. and N.K. Nichols (2016). ‘Theoretical insight into diagnosing observation error correlations using background and analysis innovation statistics’. Quarterly Journal of the Royal Meteorological Society, 142, 418–431. DOI: https://doi.org/10.1002/qj.2661
[R5] Tabeart, J.M., Dance, S.L., Haben, S.A., Lawless, A.S., Nichols, N.K. and Waller, J.A. (2018). ‘The conditioning of least squares problems in variational data assimilation’. Numerical Linear Algebra and Applications, e2165, 22. DOI: https://doi.org/10.1002/nla.2165
[R6] Waller, J.A., Simonin, D., Dance, S.L., Nichols, N.K. and Ballard, S.P. (2016). ‘Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minus-background and observation-minus-analysis statistics’. Monthly Weather Review, 144, 3533–3551. DOI: https://doi.org/10.1175/MWR-D-15-0340.1
Evidence for the body of work meeting minimum 2* threshold: The research has resulted from competitive, peer-reviewed funding applications and has been published within the main peer-reviewed, international journals of the field. Outputs that develop methodology contribute new techniques and results to the field. Outputs that evaluate the methods in the setting of operational weather forecasts advance the understanding and use of the mathematical techniques in practice.
4. Details of the impact
For the European Union alone, the economic benefits of weather forecasts are estimated at between EUR15bn and EUR61bn per year [S1]. Because of the unknown error characteristics of the observational data, weather centres have previously found it difficult to achieve full value from expensively acquired satellite and radar observations, which has motivated research at Reading. The methods that Reading has developed for estimating observation error statistics have enabled significant improvements in forecast accuracy and have been incorporated into operational weather prediction systems at major international weather centres and other institutions throughout the world, as described here. The research has also influenced strategic developments in system enhancement and diagnostic activities, as well as training at these centres.
UK Met Office (UKMO): The Met Office is the national meteorological service for the UK. It generates detailed regional forecasts every hour, and global forecasts every six hours, 365 days a year. Other national weather services use UKMO operational weather forecasting software through the UKMO Unified Model Partnership, the reach of which is global [S2], with core partners in Australia, India, New Zealand and South Korea and associate partners in Poland, Singapore, South Africa and the United States (with the US Air Force). Researchers at Reading have been closely collaborating with the UKMO on observational uncertainty since 2006, initially through CASE sponsorship and co-supervision for a PhD studentship. The evidence from the PhD work [R1, R2, R3] “convinced us [the UKMO] that we should be taking better account of observation error correlations in our data assimilation processes” [S3]. From 2009, the UKMO invested the research time of two UKMO staff over four years to develop the method further for use in their operational weather forecasting system [S3].
The new method for dealing with correlated inter-channel observation errors for IASI (Infrared Atmospheric Sounding Interferometer) became part of the UKMO operational global data assimilation system in January 2013 and has remained active throughout the REF period (from August 2013 onwards) [S3]. This resulted in a general improvement in forecast accuracy, with the most significant improvements being between 0.3% and 2.5% reductions in forecast root mean square error in key atmospheric variables. To put this in perspective, an improvement of around 3% is expected from a full year’s worth of research and development improvements by the whole UKMO Weather Science Team of approximately 200 scientists [S3]. The technique, proved in the global model, was then adopted in the UK regional forecasting system in February 2015. Between 2014 and 2016 staff at the UKMO also implemented the new methods for two additional satellite instruments, AIRS and CrlS, in both the global and the regional UK operational systems. The time spent by UKMO staff in the implementation of the method was seen as a worthwhile investment as “ each year the Met Office spends around GBP30m on satellites and their data and work of this type is important to allow us [the UKMO] to exploit our investment in these systems as fully as possible” [S3].
The research in [R4, R6] led the UKMO to “ use the Reading methods more broadly as a tool for technical analysis” [S3]. This has been so successful that the UKMO wrote general software to apply the uncertainty diagnostics to any type of observation in their system and the method has been incorporated into the UKMO’s standard practice for observation monitoring [S3]. The research [R4, R6] further prompted a decision within the UKMO to re-parallelise its supercomputing software to enable spatial error correlations also to be handled. This has made it feasible, for example, to increase the number of weather radar wind observations assimilated by a factor of four, with no significant impacts on computation time. The new software has been used in the UKMO’s operational system since November 2019. Forecasts made with this system feed directly into weather warnings issued by the UKMO’s Public Weather Service and hence it is essential that the computations be fast enough to ensure there is no delay in the provision of forecast products to customers and the public [S3].
European Centre for Medium-Range Weather Forecasts (ECMWF): The ECMWF is an independent intergovernmental organisation which is a 24/7 operational service producing global numerical weather predictions and monitoring climate change for 22 member and 12 cooperating states. The Reading method of accounting for satellite interchannel observation error correlations has been implemented operationally at ECMWF for various satellite instruments, including: IASI in 2016; CrlS in 2019; geostationary satellites in 2019; and ATMS in 2020. The Head of the Earth System Assimilation Section at ECMWF notes [S4]: “ In all cases, significant benefits for forecast skill were obtained from this enhanced exploitation of satellite observations. Treatment of inter-channel error correlations is now considered standard practice at ECMWF and other NWP [Numerical Weather Prediction] *centres for many satellite sensors, and it is recognised that it critically underpins the positive forecast impacts for many observations.*” Furthermore, since 2016, the role of observation error correlations has become an integral topic in ECMWF’s annual training course activities [S4, S5] that are aimed at international, early career scientists (approximately 30 students per year) working in operational centres and sponsored by EUMETSAT – a European intergovernmental organisation supplying satellite data, images and products to national meteorological services.
US Naval Research Laboratory (NRL): The NRL carries out development and upgrades of numerical weather prediction systems and products for US Department of Defense users. The NRL data assimilation system uses a different (dual) approach from the UKMO and the ECMWF. Regardless, it has been possible for scientists at the NRL to adapt the Reading methods [specifically R2, R3 and R4] into their forecasting system for the IASI, CrlS and ATMS instruments. This “ led to substantial gains in forecast skill, verified both against independent ECMWF analyses and against high-quality radiosondes” [S6]. The approach was subsequently transitioned into US Navy operations in January 2018 and the Head of the Atmospheric Dynamics and Prediction Branch at the NRL states “ the improved use of data has been shown to have critical importance in the assimilation of high-resolution, high-density data and the accurate prediction of high-impact weather” [S6].
US National Centers for Environmental Prediction (NCEP) and US National Aeronautics and Space Administration (NASA): NCEP and NASA have jointly developed a global atmospheric data assimilation system (GSI) that supports both NCEP’s day-to-day weather prediction obligations for the US National Weather Service and several NASA Earth observing missions. The team has developed their own implementation of satellite inter-channel correlated observation errors. Reading’s research [specifically R1, R2, R3, R4, R5] “ served as direct guidance” for how NASA and NCEP implemented their approach for handling error correlations [S7, S8]. For NASA, the “ GSI implementation resulted in clear improvement in the use of AIRS and IASI observations” [S7]. Furthermore, NCEP trials have shown that “for IASI, incorporating the error correlations reduced the forecast root-mean square error (RMSE), especially for temperature… For CrIS, the impact from correlated error was also positive” [S8]. The method will become part of the NCEP operational system at their next upgrade in early 2021.
Inspired by Reading work [R6], the NCEP regional data assimilation system teams have made further significant modifications to their methods for utilising radial wind data from radar. As of September 2020, the developments are in the NCEP code base and ready for operational implementation in the NCEP future regional system, called the ‘Rapid Refresh Forecast System.’ The Chief of the NCEP Data Assimilation and Quality Control Group said: “ In both cases, University of Reading research has had a direct influence on our development activities, improving how we use observations to initialise numerical weather prediction models. This will lead to improved forecasts, better lead times, and improved communication of high impact hazards” [S8].
Deutscher Wetterdienst (DWD): DWD is responsible for meeting the meteorological requirements arising from all areas of economy and society in Germany. A joint project between Reading and DWD has led to the use of the Reading methods [R1, R2, R3, R4, R6] as “ an important guiding tool” that has “ influenced our [DWD’s] operational implementation choices” for the assimilation of radar radial wind observations [S9]. In March 2020, radar wind data became operational in the COSMO-D2-KENDA (Kilometer Scale Ensemble Data Assimilation) system that runs with an hourly cycle for a domain covering Germany, Austria and Switzerland. DWD’s trials showed that the Doppler radar data improved weather forecast accuracy, particularly for convective events associated with hazards such as wind gusts, hail and heavy precipitation. DWD have now developed their own version of the diagnostics used in the joint work with Reading and have been applying them to other observation types. In particular, they have applied a similar approach to another type of weather radar data, 3D radar reflectivity measurements, the use of which became operational in June 2020. Both radar reflectivity and radar winds have also been integrated, “ with clear improvements in forecast accuracy” [S9] into the pre-operational ICON-D2-KENDA system, which is scheduled to become operational on 20 January 2021. The planned system has a focus on providing improved forecasts for summer severe weather. The radar observations are key for rapid update cycling, so that DWD can “ provide better forecasts of flash flood events, potentially mitigating against flood damage and loss of life” [S9].
RIKEN Centre for Computational Science (R-CCS): R-CCS is a Japanese national laboratory, the leading centre in high performance computing and computational science in Japan and home to the world’s leading supercomputer, ‘Fugaku’. R-CCS contributes to operational development for the Japan Aerospace Exploration Agency, the Japan Agency for Marine-Earth Science and Technology and the Japanese Meteorological Agency [S10]. Reading and R-CCS have agreed a Memorandum of Understanding for collaboration and Reading’s techniques have been incorporated into R-CCS’s cutting-edge research and development work. As with other users of the underpinning research [R1, R3], the centre’s scientists have established considerable improvements in the accuracy of forecasts by employing Reading’s methods [S10]. With ‘Fugaku’ now in operation, the R-CCS team are able to explore super high-resolution global data assimilation, using a greater density of observations than any other centre, where they say the treatment of observation error correlations is “ crucial” [S10].
Summary: Weather services are of benefit across all sectors of society - from farming to flying. The value of a weather forecasting system depends on its accuracy, which is limited by the uncertainties in observations. Research at Reading has investigated the effects of observational errors on weather prediction, contributed to theoretical advances in the assimilation of the data and provided new methodology for better estimation of observation error correlations. The application of this research in major meteorological centres worldwide has proven that incorporating the observation error estimates in weather prediction systems increases forecasting accuracy and improves the cost-effectiveness of expensive data. Research at Reading has influenced strategic developments, diagnostic activities, and capacity building through training. With the rapid increase in the volume and types of satellite and radar data now available, the research continues to have a far-reaching influence on the best use of weather observations across the globe.
5. Sources to corroborate the impact
The case for EPS/Metop Second-Generation: cost benefit analysis (2014) EUMETSAT Report.
UK Met Office Unified Model Partnership website – last accessed October 2019:
Letter from the UK Met Office, July 2020
Letter from the ECMWF, August 2020
ECMWF Training Material. Observation Errors Lecture by Niels Bormann > see ‘Satellite Data Assimilation (EUMETSAT/ECMWF)),’ Thursday @ 9:30-10:45, ObsErrors_2016.pptx
Letter from the NRL, May 2020
Letter from NASA, August 2020
Letter from NCEP, September 2020
Letter from DWD, September 2020
Letter from RIKEN – CCS, August 2020
Additional contextual information
Grant funding
| Grant number | Value of grant |
|---|---|
| EP/P503434/1 | £17,325 |
| EP/P002331/1 | £1,706,722 |
| ENV13-2 | £5,476,370 |
| ESRIN/RFQ/3-13355/11/I-LG | £989,000 |
| NE/N006682/1 | £31,771 |
| NE/K00896X/1 | £968,334 |
| NE/K008900/1 | £1,320,803 |
| NE/H524514/1 | £21,000 |
| NER/S/C/2006/14271 | £16,950 |
| ENV13-1 | £1,200 |