AI & the Atmosphere
The ECMWF continues to push boundaries
Artificial Intelligence & The Atmosphere
The AI-Weather/Climate space has witnessed numerous announcements over the last several months highlighting developments in the application of compute to forecasting. Google, NVIDIA, Huawei, IBM and many others are all contributing to the discussion. Improving forecasting capabilities, and perhaps more importantly, the applications of the forecasts themselves, do not constitute a zero-sum game, so given the susceptibility of nearly all aspects of modern society to weather and climate, more is better. Different models exhibit strengths and weaknesses, so safety in model selection and deployment resides in numbers and an ensemble approach is most certainly preferable than reliance on one source, particularly when decisions around human health, economic activity and natural hazards/extreme events are involved.
Perhaps under the radar to many outside of the weather/climate/water enterprise, but more important in terms of a step change in the AI-Atmosphere arena, was news from the European Centre for Medium Range Weather Forecasts (ECMWF). The ECMWF took another step in widening the gap between what many consider to be the most accurate global model, and everyone else, by launching the operational version of their AI-driven model, the Artificial Intelligence Forecasting System (AIFS), to run in parallel with their standard physics-based Integrated Forecasting System (IFS). The new AIFS model is expected to result in more accurate forecasts with longer lead times. The need for higher accuracy with respect to forecasting capabilities does not require explanation. The ECMWF has published that their new platform outperforms standard physics-based model forecasts by up to 20%. To be clear, defining ‘accuracy’ does not come down to a single metric. Different physical parameters across a variety of timescales need to be taken into account to develop a score. But a 20% improvement in accuracy for any parameter translates to a real number, whether it is measured in lives saved, number of structures protected, or dollars deployed. Better lead times allows for decision makers to be more proactive, rather than reactive. This applies to everything from logistics to emergency planning.
Physics is Still Important
Traditional physics-based models generate forecasts by solving differential equations for a variety of atmospheric and oceanic attributes, utilizing empirical data from millions of observation points and guided by the laws of physics. Due to the size and complexity of these parameters, the process can be tedious, at times slow, reliant on many, many potential points of failure, and computationally intensive. AI driven models in contrast, are nimble, and much more energy efficient. Further, AI-derived algorithms can spot and correct for missing data in a much more robust manner. Also, since the AI based models are built upon an architecture that is less computationally intensive, and most importantly open, running models and optimizing parameters can move from being an enterprise level activity to one that can can be run from an individual workstation - a data scientist’s dream (albeit, data scientist who understands physics, a topic to be addressed separately). There is still much to be done with respect to verification, accuracy, and generation of insights that abide by these laws, but early indications are pointing in a favorable direction.
Looking Ahead
While short to medium range forecasts are considered to be a commodity, things (in my opinion) will really begin to get exciting when we center on the seasonal to subseasonal (S2S) domain. There will always be inherent uncertainties embedded in a forecast, but when we extend the boundaries with respect to the time frames for decisions, the greatest potential value resides in the S2S space. This goes well beyond operational activities around seasonal demand expectations for energy supply and demand planning. For example, think about the farmer who can use extended range forecast outputs to help determine not only which variety of seed to plant, but which diseases the crop may be susceptible to, and can then more effectively select the appropriate crop protection and product management portfolio. Commercial and residential real estate can consider where and how to build (and rebuild) as a function of weather and climate risk. Or consider data centers which require fresh water for cooling that need to plan operations months in advance. It is easy to make the connection between climate, commerce and society; in fact, it is difficult to find an area of society not impacted by the atmosphere.
The efforts of the ECMWF should be applauded. They continue to raise the bar for everyone around how multi-country science and cooperation is carried out for the benefit of the public.