Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs
Solving time-dependent parametric partial differential equations (PDEs) is
challenging, as models must adapt to variations in parameters such as
coefficients, forcing terms, and boundary conditions. Data-driven neural
solvers either train on data sampled from the PDE parameters distribution in
the hope that the model generalizes to new instances or rely on gradient-based
adaptation and meta-learning to implicitly encode the dynamics from
observations. This often comes with increased inference complexity. Inspired by
the in-context learning capabilities of large language models (LLMs), we
introduce Zebra, a novel generative auto-regressive transformer designed to
solve parametric PDEs without requiring gradient adaptation at inference. By
leveraging in-context information during both pre-training and inference, Zebra
dynamically adapts to new tasks by conditioning on input sequences that
incorporate context trajectories or preceding states. This approach enables
Zebra to flexibly handle arbitrarily sized context inputs and supports
uncertainty quantification through the sampling of multiple solution
trajectories. We evaluate Zebra across a variety of challenging PDE scenarios,
demonstrating its adaptability, robustness, and superior performance compared
to existing approaches.
67094cf078efff62b1b492d1
Louis Serrano
claimed_verified
2024-10-11T16:33:15.712Z
6565d4d4d4ef4fe85fb50388
/avatars/7b8752b30aac8346ade2165a0ba09ca2.svg
Louis Serrano
user
sogeeking
67094cf078efff62b1b492d2
Armand Kassaï Koupaï
67094cf078efff62b1b492d3
Thomas X Wang
67094cf078efff62b1b492d4
Pierre Erbacher
admin_assigned
2024-10-14T12:41:47.367Z
642a94c892094a1b74dbcfe8
/avatars/155bc725b20f1f5f6b538dc30116c010.svg
Pierre Erbacher
user
erbacher
67094cf078efff62b1b492d5
Patrick Gallinari
Zebra: In-Context and Generative Pretraining for Solving Parametric PDEs
Solving time-dependent parametric partial differential equations (PDEs) is
challenging, as models must adapt to variations in parameters such as
coefficients, forcing terms, and boundary conditions. Data-driven neural
solvers either train on data sampled from the PDE parameters distribution in
the hope that the model generalizes to new instances or rely on gradient-based
adaptation and meta-learning to implicitly encode the dynamics from
observations. This often comes with increased inference complexity. Inspired by
the in-context learning capabilities of large language models (LLMs), we
introduce Zebra, a novel generative auto-regressive transformer designed to
solve parametric PDEs without requiring gradient adaptation at inference. By
leveraging in-context information during both pre-training and inference, Zebra
dynamically adapts to new tasks by conditioning on input sequences that
incorporate context trajectories or preceding states. This approach enables
Zebra to flexibly handle arbitrarily sized context inputs and supports
uncertainty quantification through the sampling of multiple solution
trajectories. We evaluate Zebra across a variety of challenging PDE scenarios,
demonstrating its adaptability, robustness, and superior performance compared
to existing approaches.
Zebra, a generative auto-regressive transformer, solves parametric PDEs without gradient adaptation by leveraging in-context learning.
generative auto-regressive transformer parametric PDEs data-driven neural solvers gradient-based adaptation meta-learning in-context learning large language models (LLMs) uncertainty quantification