## Physics-Still Reigns Supreme: Why AI Stumbles When Nature Roars
In a fascinating development for meteorological science, new research from the University of Geneva and the Karlsruhe Institute of Technology has thrown a spotlight on the capabilities, and indeed limitations, of artificial intelligence in predicting the most volatile of weather phenomena.
This study directly pit leading AI forecasting systems against a well-established physics-based model, revealing a critical gap in how these different approaches handle extreme weather events.
### The AI vs. Physics Showdown: A Test of Forecasting Prowess
The scientific community has been abuzz with the rapid advancements in AI for weather prediction.
Systems like GraphCast, Pangu-Weather, and Fuxi have demonstrated remarkable proficiency in forecasting “ordinary” weather conditions, often surpassing traditional methods.
However, this latest research delves into a more challenging and crucial aspect of meteorology: the prediction of extreme events – the heatwaves, blizzards, and storms that pose the greatest risks to human life and infrastructure.
This investigation, detailed in *Science Advances*, involved a rigorous comparison between three prominent AI forecasting systems and the European Centre for Medium-Range Weather Forecasts’ High RESolution (HRES) numerical weather prediction (NWP) model.
The results paint a clear picture, highlighting a fundamental difference in how these models grapple with the unexpected.
#### AI’s Tendency Towards the Mean: A Flaw in Forecasting Extremes
While the AI models showcased impressive accuracy for typical weather patterns, their performance faltered when confronted with extreme conditions.
The study’s findings indicate that these AI systems consistently underestimated the magnitude of extreme events.
Instead of predicting the record-breaking temperatures or intense storm surges, they tended to forecast values closer to the statistical average.
This suggests a subtle yet significant bias towards the mean, a phenomenon that could have serious consequences.
* The AI models frequently predicted less intense heatwaves than actually occurred.
* Similarly, storm intensity was often underpredicted by these AI systems.
* Conversely, cold extremes were sometimes overestimated, further demonstrating this push towards the average.
The Root of the Problem: A Reliance on the Past
The researchers posit that the primary reason for these shortcomings lies in the very foundation of AI learning: historical training data.
These models are trained on vast archives of past weather events.
While this allows them to learn patterns and correlations, it also limits their ability to extrapolate beyond the conditions they have already “seen.”
Extreme events, by their very definition, represent departures from historical norms, and AI models trained solely on past data may struggle to conceptualize or predict these unprecedented occurrences.
To put this hypothesis to the test, the research team meticulously assembled a comprehensive dataset of record-breaking events spanning from 2018 to 2020.
This included high-profile events such as the Siberian and US heatwaves of 2020, alongside numerous other smaller but still significant extreme occurrences.
Analyzing tens to hundreds of thousands of individual records – for instance, 162,751 heat records recorded in 2020 alone – the study found that the physics-based HRES model consistently demonstrated superior performance in predicting temperature and wind extremes compared to the AI systems.
The Path Forward: Towards More Robust Extreme Event Prediction
The computational demands of analyzing such extensive datasets, encompassing both AI and NWP outputs, are substantial. The study also acknowledges the challenge of accessing some state-of-the-art AI models from commercial entities, which can hinder comprehensive external validation.
To address these issues, the authors propose a structured protocol for the rigorous evaluation of extreme-event forecasts. This framework aims to provide a standardized and reliable method for assessing the accuracy and reliability of different forecasting approaches.
The researchers are actively extending their investigations to encompass probabilistic AI models. These models aim to provide a range of possible outcomes rather than a single deterministic forecast.
It is hypothesized that these models might also encounter similar limitations in predicting extreme departures from the norm. However, their probabilistic nature could offer valuable insights.
Looking ahead, the consensus among these seasoned researchers is that hybrid models, which blend the predictive power of physics-based approaches with the pattern-recognition capabilities of AI, hold the most promise for significantly improving our ability to forecast extreme weather events. This integration could potentially bridge the gap, allowing us to better anticipate and prepare for the most challenging meteorological events nature has to offer.
Here is the source article for this story: Physics-based models still beat AI for predicting extreme weather events