title: Self-Supervised Learning with Lie Symmetries for Partial Differential
    Equations
summary: Machine learning for differential equations paves the way for computationally
  efficient alternatives to numerical solvers, with potentially broad impacts in
  science and engineering. Though current algorithms typically require simulated
  training data tailored to a given setting, one may instead wish to learn useful
  information from heterogeneous sources, or from real dynamical systems
  observations that are messy or incomplete. In this work, we learn
  general-purpose representations of PDEs from heterogeneous data by implementing
  joint embedding methods for self-supervised learning (SSL), a framework for
  unsupervised representation learning that has had notable success in computer
  vision. Our representation outperforms baseline approaches to invariant tasks,
  such as regressing the coefficients of a PDE, while also improving the
  time-stepping performance of neural solvers. We hope that our proposed
  methodology will prove useful in the eventual development of general-purpose
  foundation models for PDEs.
paper_authors._id: 64ae1e6eb575c5e2722159c1
paper_authors.name: Grégoire Mialon
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-12T13:28:39.442Z
paper_authors._id: 6458c976f1f5263c2ef0514c
paper_authors.avatarUrl: /avatars/0cab7449293be91a4fa08c23533b1b1a.svg
paper_authors.fullname: Grégoire Mialon
paper_authors.type: user
paper_authors.user: gregmialz
paper_authors._id: 64ae1e6eb575c5e2722159c2
paper_authors.name: Quentin Garrido
paper_authors._id: 64ae1e6eb575c5e2722159c3
paper_authors.name: Hannah Lawrence
paper_authors._id: 64ae1e6eb575c5e2722159c4
paper_authors.name: Danyal Rehman
paper_authors._id: 64ae1e6eb575c5e2722159c5
paper_authors.name: Yann LeCun
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-23T11:52:22.496Z
paper_authors._id: 64ed0b8c2203a126eb1a5b9a
paper_authors.avatarUrl: /avatars/9156dc406ed3f9ee62b73657ac20f5ed.svg
paper_authors.fullname: Yann LeCun
paper_authors.type: user
paper_authors.user: ylecun
paper_authors._id: 64ae1e6eb575c5e2722159c6
paper_authors.name: Bobak T. Kiani
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-07-13T08:32:27.702Z
paper_authors._id: 64aec711749f04f4f4c7cb4b
paper_authors.avatarUrl: /avatars/0449729bdbae87f27ae6663a36430bf3.svg
paper_authors.fullname: Kiani
paper_authors.type: user
paper_authors.user: Bobak
paper_title: Self-Supervised Learning with Lie Symmetries for Partial Differential
    Equations
paper_summary: Machine learning for differential equations paves the way for computationally
  efficient alternatives to numerical solvers, with potentially broad impacts in
  science and engineering. Though current algorithms typically require simulated
  training data tailored to a given setting, one may instead wish to learn useful
  information from heterogeneous sources, or from real dynamical systems
  observations that are messy or incomplete. In this work, we learn
  general-purpose representations of PDEs from heterogeneous data by implementing
  joint embedding methods for self-supervised learning (SSL), a framework for
  unsupervised representation learning that has had notable success in computer
  vision. Our representation outperforms baseline approaches to invariant tasks,
  such as regressing the coefficients of a PDE, while also improving the
  time-stepping performance of neural solvers. We hope that our proposed
  methodology will prove useful in the eventual development of general-purpose
  foundation models for PDEs.
paper_ai_keywords: joint embedding methods self-supervised learning (SSL) unsupervised representation learning regressing the coefficients of a PDE time-stepping performance neural solvers) general-purpose foundation models for PDEs