title: SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models
summary: Probabilistic forecasting is crucial to decision-making under uncertainty
  about future weather. The dominant approach is to use an ensemble of forecasts
  to represent and quantify uncertainty in operational numerical weather
  prediction. However, generating ensembles is computationally costly. In this
  paper, we propose to generate ensemble forecasts at scale by leveraging recent
  advances in generative artificial intelligence. Our approach learns a
  data-driven probabilistic diffusion model from the 5-member ensemble GEFS
  reforecast dataset. The model can then be sampled efficiently to produce
  realistic weather forecasts, conditioned on a few members of the operational
  GEFS forecasting system. The generated ensembles have similar predictive skill
  as the full GEFS 31-member ensemble, evaluated against ERA5 reanalysis, and
  emulate well the statistics of large physics-based ensembles. We also apply the
  same methodology to developing a diffusion model for generative
  post-processing: the model directly learns to correct biases present in the
  emulated forecasting system by leveraging reanalysis data as labels during
  training. Ensembles from this generative post-processing model show greater
  reliability and accuracy, particularly in extreme event classification. In
  general, they are more reliable and forecast the probability of extreme weather
  more accurately than the GEFS operational ensemble. Our models achieve these
  results at less than 1/10th of the computational cost incurred by the
  operational GEFS system.
paper_authors._id: 649a61e02b53f415912419b5
paper_authors.name: Lizao Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-29T12:00:52.965Z
paper_authors._id: 63da20313b8591bd11f2f948
paper_authors.avatarUrl: /avatars/cbcdb73cfa72ed6672fe60b96a7fdd6f.svg
paper_authors.fullname: Lizao Li
paper_authors.type: user
paper_authors.user: lzlarryli
paper_authors._id: 649a61e02b53f415912419b6
paper_authors.name: Rob Carver
paper_authors._id: 649a61e02b53f415912419b7
paper_authors.name: Ignacio Lopez-Gomez
paper_authors._id: 649a61e02b53f415912419b8
paper_authors.name: Fei Sha
paper_authors._id: 649a61e02b53f415912419b9
paper_authors.name: John Anderson
paper_title: SEEDS: Emulation of Weather Forecast Ensembles with Diffusion Models
paper_summary: Probabilistic forecasting is crucial to decision-making under uncertainty
  about future weather. The dominant approach is to use an ensemble of forecasts
  to represent and quantify uncertainty in operational numerical weather
  prediction. However, generating ensembles is computationally costly. In this
  paper, we propose to generate ensemble forecasts at scale by leveraging recent
  advances in generative artificial intelligence. Our approach learns a
  data-driven probabilistic diffusion model from the 5-member ensemble GEFS
  reforecast dataset. The model can then be sampled efficiently to produce
  realistic weather forecasts, conditioned on a few members of the operational
  GEFS forecasting system. The generated ensembles have similar predictive skill
  as the full GEFS 31-member ensemble, evaluated against ERA5 reanalysis, and
  emulate well the statistics of large physics-based ensembles. We also apply the
  same methodology to developing a diffusion model for generative
  post-processing: the model directly learns to correct biases present in the
  emulated forecasting system by leveraging reanalysis data as labels during
  training. Ensembles from this generative post-processing model show greater
  reliability and accuracy, particularly in extreme event classification. In
  general, they are more reliable and forecast the probability of extreme weather
  more accurately than the GEFS operational ensemble. Our models achieve these
  results at less than 1/10th of the computational cost incurred by the
  operational GEFS system.
paper_ai_keywords: probabilistic diffusion model ensemble forecasts generative artificial intelligence generative post-processing reanalysis data extreme event classification