N241-054 TITLE: Probabilistic Forecasts of High Impact Weather on Medium Range to Subseasonal Timescales using Artificial Intelligence

OUSD (R&E) CRITICAL TECHNOLOGY AREA(S): Trusted AI and Autonomy

OBJECTIVE: Develop a robust artificial intelligence/machine learning (AI/ML) based numerical weather prediction (NWP) system capable of generating skillful high resolution forecasts globally at the medium range (5-14 days) and subseasonal (14-90 days) timescales. System should meet or exceed conventional NWP metrics for skill and computational expense, especially for high impact weather events and high error scenarios.

DESCRIPTION: Rapid progress has been made in the past few years in the development and application of AI/ML meteorological models, especially with respect to mirroring capabilities of state of the art dynamical NWP models. There is recent evidence that AI/ML models can be as skillful as state-of-the-art NWP using standard metrics (e.g., anomaly correlations, root-mean-square errors, etc. – see Weyn et al. 2020, 2021; Lam et al. 2022; Bi et al. 2022, 2023; Zhang et al. 2023). However, it remains unclear how well these models perform using more complete metrics that incorporate high-impact weather events (e.g., rainfall, peak winds, etc.), smaller scale atmospheric features, and high sigma (i.e., uncommon) weather pattern scenarios. Furthermore, many of these AI/ML models are trained with relatively coarse resolution reanalysis data. A challenge remains how to better use volume of data (including from commercial sources) for better initial conditions and forecasts. While certain aspects of traditional NWP infrastructure may be difficult to change (such as data assimilation and model analysis states), other aspects such as parameterized physics, construction and diversity of ensembles toward probabilistic forecasts, and identification of forecast uncertainty are ripe for improvement by AI/ML methods. This topic aims to bolster strong development of AI/ML NWP techniques by soliciting targeted research and development to robustly improve skill and resolution of weather forecasts. Specifically, we seek to improve scientific understanding and forecast capability at the medium range (5-14 days) and subseasonal (14-90 days) timescales for both state variables and derived metrics (e.g., clouds, precipitation, etc.). While it is anticipated that weather feature fidelity will be achieved via large ensemble development, use of downscaling towards high resolution techniques would also be considered. Robust methods to resolve high impact cases and development of verification statistics that do not smooth/average out those signals will be necessary. Finally, a significant component of this effort involves development of understandable (explainable) AI/ML infrastructure and techniques that support refined physics improvements and the ability to substitute newer methods as routines evolve.

PHASE I: Perform a comprehensive feasibility study on the proposed end-to-end software architecture as well as demonstrations of AI/ML technique effectiveness to address this problem. Study should include a comprehensive literature review including the state of the science and trajectory, an analysis of alternatives on different AI/ML methods and their strengths and weaknesses, and a discussion on the most challenging research and development parts of the problem (focusing on physics representation, skill challenges with lead-time, and representativeness of weather features at different resolutions). Analysis must include potential for large ensemble and probabilistic approaches and/or high resolution and downscaling techniques and the potential need to use coupled earth system models (that include atmosphere, ocean, land and/or ice) to foster longer range skill. Requirements on computational architecture/software/data should be outlined, as well as proposed metrics to improve beyond point-based averaged metrics towards emphasizing high impact weather events, skill dropouts, and bifurcating scenarios.

PHASE II: A prototype system capable of running real-time forecasts comparable with state of the art dynamical weather prediction systems shall be delivered at the end of Phase II. Development of this capability includes building out the appropriate data ingest, physics representation, forecast propagation, and data output. Architecture components must conform to leveraging processes available to traditional dynamical modeling platforms. Throughout this development, computational efficiency benchmarks should be provided to assess acquisition hardware needs. For more mature efforts, there should be a strong focus on developing very large ensembles and/or high resolution and/or other proposed novel/innovative use of NWP, data assimilation and initial condition sensitivity to demonstrate fidelity for high impact events on 5-90 day time scales. Software must incorporate potential to train/retrain model or model elements. Verification and validation (V&V) will be emphasized throughout the Phase II, with multiple retrospective and real-time V&V development check points. System should be flexible enough to have iterations of skill improvement, validated using impact-based probabilistic metrics with concurrent ONR field campaigns (that may include one or more of analysis of tropical cyclones, atmospheric rivers, boundary layer, and air-sea interaction). There will be particular interest in comparison of skill with other NWP models and analysis of high error/dropout events when AI/ML technique is superior to the traditional NWP model (and vice versa). Sensitivity to grid resolution, initial conditions, and training versus validation datasets (seasons, years, models) will need to be tested and reported.

PHASE III DUAL USE APPLICATIONS: Primary Naval transition opportunity would be to the Fleet Numerical Meteorology and Oceanography Center as an operational weather forecast model run in real-time production. Phase III efforts toward this goal would entail building out the software infrastructure needed to run on local compute, dedicated HPC, and/or government cloud compute solutions. Careful consideration would be needed to develop an environment conducive to upgrades, such as from new data sources, retrained AI/ML equations, and application to varied use cases (such as regional domains, output variables, verification statistics, etc.). Phase III, or alternate projects, would also coordinate with partner Air Force and Army numerical weather prediction programs for other DoD use cases and spin-off efforts. There will also be opportunity and potential to partner with other government agencies outside of the DoD, such as NOAA, NASA, and DoE for their weather modeling use cases. Commercialization beyond government services would support a growing industry with varied needs for computationally efficient and highly specialized meteorological forecasts, including demonstration of commercial weather data services, tailored platforms spanning multiple industries (e.g., agriculture, insurance, aviation, etc.), and general public interests.

REFERENCES:

1. Weyn, J.A., Durran, D.R. and Caruana, R., 2020. Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere. Journal of Advances in Modeling Earth Systems, 12(9), p.e2020MS002109
2. Weyn, J.A., Durran, D.R., Caruana, R. and Cresswell-Clay, N., 2021. Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models. Journal of Advances in Modeling Earth Systems, 13(7), p.e2021MS002502
3. Lam, R., Sanchez-Gonzalez, A., Willson, M., Wirnsberger, P., Fortunato, M., Pritzel, A., Ravuri, S., Ewalds, T., Alet, F., Eaton-Rosen, Z. and Hu, W., 2022. GraphCast: Learning skillful medium-range global weather forecasting. arXiv preprint arXiv:2212.12794
4. Bi, K., Xie, L., Zhang, H., Chen, X., Gu, X. and Tian, Q., 2022. Pangu-weather: A 3d high-resolution model for fast and accurate global weather forecast. arXiv preprint arXiv:2211.02556
5. Bi, K., Xie, L., Zhang, H. et al. Accurate medium-range global weather forecasting with 3D neural networks., 2023. Nature. https://doi.org/10.1038/s41586-023-06185-3
6. Zhang, Y., Long, M., Chen, K. et al. Skilful nowcasting of extreme precipitation with NowcastNet, 2023. Nature. https://doi.org/10.1038/s41586-023-06184-4

KEYWORDS: Numerical Weather Prediction; Machine Learning; Artificial Intelligence; Subseasonal to Seasonal; Forecasting; Reanalysis; Mesoscale; Meteorology; Oceanography; Earth System Modeling

---