title: Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs
summary: Solving time-dependent parametric partial differential equations (PDEs) is
  challenging, as models must adapt to variations in parameters such as
  coefficients, forcing terms, and boundary conditions. Data-driven neural
  solvers either train on data sampled from the PDE parameters distribution in
  the hope that the model generalizes to new instances or rely on gradient-based
  adaptation and meta-learning to implicitly encode the dynamics from
  observations. This often comes with increased inference complexity. Inspired by
  the in-context learning capabilities of large language models (LLMs), we
  introduce Zebra, a novel generative auto-regressive transformer designed to
  solve parametric PDEs without requiring gradient adaptation at inference. By
  leveraging in-context information during both pre-training and inference, Zebra
  dynamically adapts to new tasks by conditioning on input sequences that
  incorporate context trajectories or preceding states. This approach enables
  Zebra to flexibly handle arbitrarily sized context inputs and supports
  uncertainty quantification through the sampling of multiple solution
  trajectories. We evaluate Zebra across a variety of challenging PDE scenarios,
  demonstrating its adaptability, robustness, and superior performance compared
  to existing approaches.
paper_authors._id: 67094cf078efff62b1b492d1
paper_authors.name: Louis Serrano
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-11T16:33:15.712Z
paper_authors._id: 6565d4d4d4ef4fe85fb50388
paper_authors.avatarUrl: /avatars/7b8752b30aac8346ade2165a0ba09ca2.svg
paper_authors.fullname: Louis Serrano
paper_authors.type: user
paper_authors.user: sogeeking
paper_authors._id: 67094cf078efff62b1b492d2
paper_authors.name: Armand Kassaï Koupaï
paper_authors._id: 67094cf078efff62b1b492d3
paper_authors.name: Thomas X Wang
paper_authors._id: 67094cf078efff62b1b492d4
paper_authors.name: Pierre Erbacher
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-14T12:41:47.367Z
paper_authors._id: 642a94c892094a1b74dbcfe8
paper_authors.avatarUrl: /avatars/155bc725b20f1f5f6b538dc30116c010.svg
paper_authors.fullname: Pierre Erbacher
paper_authors.type: user
paper_authors.user: erbacher
paper_authors._id: 67094cf078efff62b1b492d5
paper_authors.name: Patrick Gallinari
paper_title: Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs
paper_summary: Solving time-dependent parametric partial differential equations (PDEs) is
  challenging, as models must adapt to variations in parameters such as
  coefficients, forcing terms, and boundary conditions. Data-driven neural
  solvers either train on data sampled from the PDE parameters distribution in
  the hope that the model generalizes to new instances or rely on gradient-based
  adaptation and meta-learning to implicitly encode the dynamics from
  observations. This often comes with increased inference complexity. Inspired by
  the in-context learning capabilities of large language models (LLMs), we
  introduce Zebra, a novel generative auto-regressive transformer designed to
  solve parametric PDEs without requiring gradient adaptation at inference. By
  leveraging in-context information during both pre-training and inference, Zebra
  dynamically adapts to new tasks by conditioning on input sequences that
  incorporate context trajectories or preceding states. This approach enables
  Zebra to flexibly handle arbitrarily sized context inputs and supports
  uncertainty quantification through the sampling of multiple solution
  trajectories. We evaluate Zebra across a variety of challenging PDE scenarios,
  demonstrating its adaptability, robustness, and superior performance compared
  to existing approaches.
paper_ai_summary: Zebra, a generative auto-regressive transformer, solves parametric PDEs without gradient adaptation by leveraging in-context learning.
paper_ai_keywords: generative auto-regressive transformer parametric PDEs data-driven neural solvers gradient-based adaptation meta-learning in-context learning large language models (LLMs) uncertainty quantification