Catalogue Search | MBRL
Search Results Heading
Explore the vast range of titles available.
MBRLSearchResults
-
DisciplineDiscipline
-
Is Peer ReviewedIs Peer Reviewed
-
Item TypeItem Type
-
SubjectSubject
-
YearFrom:-To:
-
More FiltersMore FiltersSourceLanguage
Done
Filters
Reset
2,186
result(s) for
"Forecasting skill"
Sort by:
On Some Limitations of Current Machine Learning Weather Prediction Models
2024
Machine Learning (ML) is having a profound impact in the domain of Weather and Climate Prediction. A recent development in this area has been the emergence of fully data‐driven ML prediction models which routinely claim superior performance to that of traditional physics‐based models. We examine some aspects of the forecasts produced by three of the leading current ML models, Pangu‐Weather, FourCastNet and GraphCast, with a focus on their fidelity and physical consistency. The main conclusion is that these ML models are not able to properly reproduce sub‐synoptic and mesoscale weather phenomena and lack the fidelity and physical consistency of physics‐based models and this has impacts on the interpretation of their forecasts and their perceived skill. Balancing forecast skill and physical realism will be an important consideration for future ML models.
Plain Language Summary
The last few years have seen the emergence of a new type of weather forecasting models completely based on ML technologies. These models do not codify the physical laws governing atmospheric dynamics but learn to produce forecasts from historical reanalysis data sets of the Earth system like the ECMWF ERA5. In this work we show that the forecasts produced by some of the leading ML models are physically inconsistent and should be better considered as post‐processing algorithms rather than realistic simulators of the atmosphere. The challenge for next generation of ML models for weather forecasting will be to improve their fidelity while maintaining forecast skill.
Key Points
Forecasts from Machine Learning (ML) models have energy spectra notably different from those of their training reanalysis fields and Numerical Weather Prediction models
This results in overly smooth predictions and weather phenomena at spatial scales shorter than 300–400 km are not properly represented
Fundamental physical balances and derived quantities are not realistically represented in the forecasts of the ML models
Journal Article
Explainable El Niño predictability from climate mode interactions
2024
The El Niño–Southern Oscillation (ENSO) provides most of the global seasonal climate forecast skill
1
–
3
, yet, quantifying the sources of skilful predictions is a long-standing challenge
4
–
7
. Different sources of predictability affect ENSO evolution, leading to distinct global effects. Artificial intelligence forecasts offer promising advancements but linking their skill to specific physical processes is not yet possible
8
–
10
, limiting our understanding of the dynamics underpinning the advancements. Here we show that an extended nonlinear recharge oscillator (XRO) model shows skilful ENSO forecasts at lead times up to 16–18 months, better than global climate models and comparable to the most skilful artificial intelligence forecasts. The XRO parsimoniously incorporates the core ENSO dynamics and ENSO’s seasonally modulated interactions with other modes of variability in the global oceans. The intrinsic enhancement of ENSO’s long-range forecast skill is traceable to the initial conditions of other climate modes by means of their memory and interactions with ENSO and is quantifiable in terms of these modes’ contributions to ENSO amplitude. Reforecasts using the XRO trained on climate model output show that reduced biases in both model ENSO dynamics and in climate mode interactions can lead to more skilful ENSO forecasts. The XRO framework’s holistic treatment of ENSO’s global multi-timescale interactions highlights promising targets for improving ENSO simulations and forecasts.
An extended nonlinear recharge oscillator model shows skilful and explainable El Niño–Southern Oscillation forecasts at lead times up to 16–18 months, better than global climate models and comparable to the most skilful artificial intelligence forecasts.
Journal Article
Improving Subseasonal‐To‐Seasonal Prediction of Summer Extreme Precipitation Over Southern China Based on a Deep Learning Method
2023
The reliable Subseasonal‐to‐Seasonal (S2S) forecast of precipitation, particularly extreme precipitation, is critical for disaster prevention and mitigation, which however remains a great challenge for mission agencies and research communities. In this study, a deep learning method based on U‐Net with additional atmospheric factor forecasts included is proposed to improve S2S quantitative forecasts of summer precipitation over Southern China. The weighted loss function integrated by mean square error and threat score is introduced to capture extreme precipitation more precisely. Generally, the U‐Net model shows promising results in both general statistics and extreme events. Predictor importance analyses show that the U‐Net forecast skills at the 1‐week lead time mainly arise from synchronous precipitation forecasts, but the contributions made by atmospheric factor forecasts rise rapidly with increasing lead times. Therefore, the channel combining numerical weather prediction model and deep learning framework is demonstrated promising in S2S precipitation forecasts.
Plain Language Summary
The Subseasonal‐to‐Seasonal (S2S) forecast of precipitation, in particular of the extreme precipitation events, from 2 weeks to a season in advance is challenging despite increasing social demand and scientific interest for accurate and dependable predictions. In this study, the U‐Net based deep learning method is employed with additional atmospheric variable forecasts (e.g., wind and specific humidity at multiple levels) included to correct the S2S forecasts of summer precipitation derived from a numerical weather prediction model over Southern China. It is demonstrated that the U‐Net improves the forecast performance in both general statistics and extreme events and shows a pronounced superiority to the traditional statistical postprocessing method. Thus, combining numerical models and deep learning is very promising in subseasonal precipitation forecasts and can also be applied to the routine forecast of other atmospheric and ocean phenomena in the future.
Key Points
The Subseasonal‐to‐Seasonal prediction of summer precipitation over southern China is improved with a U‐Net based deep learning method
The U‐Net demonstrated promising performance in both general statistics and extreme events and shows superiority to the quantile mapping benchmark
The model skills arise from precipitation itself at the early stage, while atmospheric factors play important roles at longer lead times
Journal Article
Seasonal Soil Moisture Drought Prediction over Europe Using the North American Multi-Model Ensemble (NMME)
by
Schäfer, David
,
Mai, Juliane
,
Samaniego, Luis
in
Climate models
,
Computer applications
,
Crop yield
2015
Droughts diminish crop yields and can lead to severe socioeconomic damages and humanitarian crises (e.g., famine). Hydrologic predictions of soil moisture droughts several months in advance are needed to mitigate the impact of these extreme events. In this study, the performance of a seasonal hydrologic prediction system for soil moisture drought forecasting over Europe is investigated. The prediction system is based on meteorological forecasts of the North American Multi-Model Ensemble (NMME) that are used to drive the mesoscale hydrologic model (mHM). The skill of the NMME-based forecasts is compared against those based on the ensemble streamflow prediction (ESP) approach for the hindcast period of 1983–2009. The NMME-based forecasts exhibit an equitable threat score that is, on average, 69% higher than the ESP-based ones at 6-month lead time. Among the NMME-based forecasts, the full ensemble outperforms the single best-performingmodel CFSv2, as well as all subensembles. Subensembles, however, could be useful for operational forecasting because they are showing only minor performance losses (less than 1%), but at substantially reduced computational costs (up to 60%). Regardless of the employed forecasting approach, there is considerable variability in the forecasting skill ranging up to 40% in space and time. High skill is observed when forecasts are mainly determined by initial hydrologic conditions. In general, the NMME-based seasonal forecasting system is well suited for a seamless drought prediction system as it outperforms ESP-based forecasts consistently over the entire study domain at all lead times.
Journal Article
The National Hurricane Center Tropical Cyclone Model Guidance Suite
by
Kaplan, John
,
Sampson, Charles R.
,
Knaff, John A.
in
Advection
,
Climate models
,
Climate science
2022
The National Hurricane Center (NHC) uses a variety of guidance models for its operational tropical cyclone track, intensity, and wind structure forecasts, and as baselines for the evaluation of forecast skill. A set of the simpler models, collectively known as the NHC guidance suite, is maintained by NHC. The models comprising the guidance suite are briefly described and evaluated, with details provided for those that have not been documented previously. Decay-SHIFOR is a modified version of the Statistical Hurricane Intensity Forecast (SHIFOR) model that includes decay over land; this modification improves the SHIFOR forecasts through about 96 h. T-CLIPER, a climatology and persistence model that predicts track and intensity using a trajectory approach, has error characteristics similar to those of CLIPER and D-SHIFOR but can be run to any forecast length. The Trajectory and Beta model (TAB), another trajectory track model, applies a gridpoint spatial filter to smooth winds from the National Centers for Environmental Prediction (NCEP) Global Forecast System (GFS) model. TAB model errors were 10%–15% lower than those of the Beta and Advection model (BAM), the model it replaced in 2017. Optimizing TAB’s vertical weights shows that the lower troposphere’s environmental flow provides a better match to observed tropical cyclone motion than does the upper troposphere’s, and that the optimal steering layer is shallower for higher-latitude and weaker tropical cyclones. The advantages and disadvantages of the D-SHIFOR, T-CLIPER, and TAB models relative to their earlier counterparts are discussed.
Journal Article
To Identify the Forecast Skill Windows of MJO Based on the S2S Database
by
Liu, Xinli
,
Peng, Yihao
,
Liu, Xiaolei
in
Economic forecasting
,
forecast skill windows
,
Forecasting skill
2024
As a practical reflection of the opportunity window of Madden‐Julian Oscillation (MJO), there are intermittent periods of relatively high forecasting skills, namely the forecast skill windows. Robust forecast skill windows are identified based on the subseasonal‐seasonal reforecast database, during which the majority of models show high forecast skills. A total of 15 MJO forecast skill windows during 1993–2020 have been identified. Most of the forecast skill windows are closely associated with active MJO events with high amplitude. Whether a high‐skill forecast window appears significantly depends on the magnitude of MJO intensity during the same period. The maintenance of active strong MJO events is potentially related with the warmer surface sea temperature anomalies in the western Pacific. Further research into such processes may unveil the MJO development mechanism and improve the MJO forecast skill.
Plain Language Summary
With the continuous development of the economy and society, there is an increasing demand for subseasonal‐seasonal forecasts. The forecast skills of Madden‐Julian Oscillation (MJO) show significant high and low variabilities, associated with specific conditions. MJO forecast skill windows are confirmed based on subseasonal‐seasonal (S2S) forecast products in this study. These windows represent the periods when most models in the S2S database can accurately capture the MJO signal, indicating the MJO forecast skill experiences intermittent enhancements. When the MJO intensity is high in the same period, the high‐skill forecast window is more likely to appear. One potential reason for the maintenance of strong MJO is the anomalously warmer sea surface temperature in the western Pacific, along with other potential factors, providing the opportunity to forecast MJO with high skill. When the opportunity arises, the forecast skill of MJO improves, highlighting the significance of exploring the processes leading to the occurrence of these skill windows. This provides a pathway for more accurate MJO forecasting and improves the predictability of S2S.
Key Points
Robust forecast skill windows of Madden‐Julian Oscillation (MJO) are identified based on the subseasonal‐seasonal reforecast database
The forecast skill windows are mostly related to strong MJO amplitude
Warmer sea surface temperature anomalies in the western Pacific potentially favors the maintenance of active strong MJO events
Journal Article
Potential Predictability of the Madden‐Julian Oscillation in a Superparameterized Model
2023
The Madden‐Julian Oscillation (MJO) is a promising target for improving sub‐seasonal weather forecasts. Current forecast models struggle to simulate the MJO due to imperfect convective parameterizations and mean state biases, degrading their forecast skill. Previous studies have estimated a potential MJO predictability 5–15 days higher than current forecast skill, but these estimates also use models with parameterized convection. We perform a perfect‐model predictability experiment using a superparameterized global model in which the convective parameterization is replaced by a cloud resolving model. We add a second “silent” cloud resolving component to the control simulation that independently calculates convective‐scale processes using the same large‐scale forcings. The second set of convective states are used to initialize forecasts, representing uncertainty on the convective scale. We find a potential predictability of the MJO of 35–40 days in boreal winter using a single‐member ensemble forecast.
Plain Language Summary
The Madden‐Julian Oscillation is a convective signal in the tropics that has the potential to improve 10‐40‐day weather forecasts. Current weather forecast models struggle to simulate the MJO, leading to a lower forecast skill than many studies estimate could be possible. We use a model with a comparatively good representation of the MJO that calculates convection information with a cloud permitting model. We modify this multiscale model's structure to generate MJO forecasts for its own MJO. Results from these forecasts suggest that the MJO in this model could be predictable up to 35–40 days using a single‐member ensemble forecast, which is 5–10 days longer than current state‐of‐the‐art forecasts.
Key Points
A novel approach is used to conduct perfect‐model predictability experiments using a superparameterized global model
A single ensemble member model with superparameterized convection finds a potential Madden‐Julian Oscillation predictability of 35–40 days
Resulting predictability estimates are comparable to those from current state‐of‐the‐art multiple ensemble member forecasting models
Journal Article
Subseasonal Ensemble Prediction of Flash Droughts over China
2023
Flash droughts have been occurring frequently worldwide, which has a serious impact on food and water security. The rapid onset of flash droughts presents a challenge to the subseasonal forecast, but there is limited knowledge about their forecast skills due to the lack of appropriate identification and assessment procedures. Here, we investigate the forecast skill of flash droughts over China with lead times up to 3 weeks by using hindcast datasets from the Subseasonal-to-Seasonal Prediction (S2S) project. The flash droughts are identified by using weekly soil moisture percentiles from two S2S forecast models (ECMWF and NCEP). The comparison with reanalysis shows that ECMWF and NCEP forecast models underestimate flash drought occurrence by 5% and 19% for lead 1 week. The national mean hit rates for flash droughts are 0.22 and 0.16 for ECMWF and NCEP models for lead 1 week, and they can reach 0.29 and 0.18 over South China. The ensemble of the two models increases equitable threat score (ETS) from ECMWF and NCEP models by 8% and 40% for lead 1 week. In terms of probabilistic forecast, ECMWF has a higher Brier skill score than NCEP, especially over eastern China, which is consistent with higher temperature and precipitation forecast skill. The multimodel ensemble has the highest Brier skill score. This study suggests the importance of multimodel ensemble flash drought forecasting.
Journal Article
Evaluation of the skill of North-American Multi-Model Ensemble (NMME) Global Climate Models in predicting average and extreme precipitation and temperature over the continental USA
by
Slater, Louise J
,
Villarini, Gabriele
,
Bradley, Allen A
in
Climate
,
Climate models
,
Climatic extremes
2019
This paper examines the forecasting skill of eight Global Climate Models from the North-American Multi-Model Ensemble project (CCSM3, CCSM4, CanCM3, CanCM4, GFDL2.1, FLORb01, GEOS5, and CFSv2) over seven major regions of the continental United States. The skill of the monthly forecasts is quantified using the mean square error skill score. This score is decomposed to assess the accuracy of the forecast in the absence of biases (potential skill) and in the presence of conditional (slope reliability) and unconditional (standardized mean error) biases. We summarize the forecasting skill of each model according to the initialization month of the forecast and lead time, and test the models’ ability to predict extended periods of extreme climate conducive to eight ‘billion-dollar’ historical flood and drought events. Results indicate that the most skillful predictions occur at the shortest lead times and decline rapidly thereafter. Spatially, potential skill varies little, while actual model skill scores exhibit strong spatial and seasonal patterns primarily due to the unconditional biases in the models. The conditional biases vary little by model, lead time, month, or region. Overall, we find that the skill of the ensemble mean is equal to or greater than that of any of the individual models. At the seasonal scale, the drought events are better forecast than the flood events, and are predicted equally well in terms of high temperature and low precipitation. Overall, our findings provide a systematic diagnosis of the strengths and weaknesses of the eight models over a wide range of temporal and spatial scales.
Journal Article
Composite Environments of Severe and Nonsevere High-Shear, Low-CAPE Convective Events
by
Sherburn, Keith D.
,
King, Jessica R.
,
Parker, Matthew D.
in
Alarm systems
,
Ascent
,
Atmospheric sciences
2016
Severe convection occurring in environments characterized by large amounts of vertical wind shear and limited instability (high-shear, low-CAPE, or “HSLC,” environments) represents a considerable forecasting and nowcasting challenge. Of particular concern, NWS products associated with HSLC convection have low probability of detection and high false alarm rates. Past studies of HSLC convection have examined features associated with single cases; the present work, through composites of numerous cases, illustrates the attributes of “typical” HSLC severe and nonsevere events and identifies features that discriminate between the two. HSLC severe events across the eastern United States typically occur in moist boundary layers within the warm sector or along the cold front of a strong surface cyclone, while those in the western United States have drier boundary layers and more typically occur in the vicinity of a surface triple point or in an upslope regime. The mean HSLC severe event is shown to exhibit stronger forcing for ascent at all levels than its nonsevere counterpart. The majority of EF1 or greater HSLC tornadoes are shown to occur in the southeastern United States, so this region is subjected to the most detailed statistical analysis. Beyond the documented forecasting skill of environmental lapse rates and low-level shear vector magnitude, it is shown that a proxy for the release of potential instability further enhances skill when attempting to identify potentially severe HSLC events. This enhancement is likely associated with the local, in situ CAPE generation provided by this mechanism. Modified forecast parameters including this proxy show considerably improved spatial focusing of the forecast severe threat when compared to existing metrics.
Journal Article