title: PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
summary: Time-dependent partial differential equations (PDEs) are ubiquitous in
  science and engineering. Recently, mostly due to the high computational cost of
  traditional solution techniques, deep neural network based surrogates have
  gained increased interest. The practical utility of such neural PDE solvers
  relies on their ability to provide accurate, stable predictions over long time
  horizons, which is a notoriously hard problem. In this work, we present a
  large-scale analysis of common temporal rollout strategies, identifying the
  neglect of non-dominant spatial frequency information, often associated with
  high frequencies in PDE solutions, as the primary pitfall limiting stable,
  accurate rollout performance. Based on these insights, we draw inspiration from
  recent advances in diffusion models to introduce PDE-Refiner; a novel model
  class that enables more accurate modeling of all frequency components via a
  multistep refinement process. We validate PDE-Refiner on challenging benchmarks
  of complex fluid dynamics, demonstrating stable and accurate rollouts that
  consistently outperform state-of-the-art models, including neural, numerical,
  and hybrid neural-numerical architectures. We further demonstrate that
  PDE-Refiner greatly enhances data efficiency, since the denoising objective
  implicitly induces a novel form of spectral data augmentation. Finally,
  PDE-Refiner's connection to diffusion models enables an accurate and efficient
  assessment of the model's predictive uncertainty, allowing us to estimate when
  the surrogate becomes inaccurate.
paper_authors._id: 64d59d6c887f55fb6e055a63
paper_authors.name: Phillip Lippe
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-11T14:37:49.983Z
paper_authors._id: 63087808f48eff2e8eb64522
paper_authors.avatarUrl: /avatars/e6fb4143f4d2a1fe721a50167717803a.svg
paper_authors.fullname: Phillip Lippe
paper_authors.type: user
paper_authors.user: phlippe
paper_authors._id: 64d59d6c887f55fb6e055a64
paper_authors.name: Bastiaan S. Veeling
paper_authors._id: 64d59d6c887f55fb6e055a65
paper_authors.name: Paris Perdikaris
paper_authors._id: 64d59d6c887f55fb6e055a66
paper_authors.name: Richard E. Turner
paper_authors._id: 64d59d6c887f55fb6e055a67
paper_authors.name: Johannes Brandstetter
paper_title: PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers
paper_summary: Time-dependent partial differential equations (PDEs) are ubiquitous in
  science and engineering. Recently, mostly due to the high computational cost of
  traditional solution techniques, deep neural network based surrogates have
  gained increased interest. The practical utility of such neural PDE solvers
  relies on their ability to provide accurate, stable predictions over long time
  horizons, which is a notoriously hard problem. In this work, we present a
  large-scale analysis of common temporal rollout strategies, identifying the
  neglect of non-dominant spatial frequency information, often associated with
  high frequencies in PDE solutions, as the primary pitfall limiting stable,
  accurate rollout performance. Based on these insights, we draw inspiration from
  recent advances in diffusion models to introduce PDE-Refiner; a novel model
  class that enables more accurate modeling of all frequency components via a
  multistep refinement process. We validate PDE-Refiner on challenging benchmarks
  of complex fluid dynamics, demonstrating stable and accurate rollouts that
  consistently outperform state-of-the-art models, including neural, numerical,
  and hybrid neural-numerical architectures. We further demonstrate that
  PDE-Refiner greatly enhances data efficiency, since the denoising objective
  implicitly induces a novel form of spectral data augmentation. Finally,
  PDE-Refiner's connection to diffusion models enables an accurate and efficient
  assessment of the model's predictive uncertainty, allowing us to estimate when
  the surrogate becomes inaccurate.
paper_ai_summary: PDE-Refiner, inspired by diffusion models, provides accurate, stable, and data-efficient solutions for time-dependent PDEs by refining all frequency components.
paper_ai_keywords: partial differential equations deep neural networks neural PDE solvers temporal rollout strategies diffusion models PDE-Refiner multistep refinement fluid dynamics spectral data augmentation predictive uncertainty