title: A Generalist Dynamics Model for Control
summary: We investigate the use of transformer sequence models as dynamics models
  (TDMs) for control. In a number of experiments in the DeepMind control suite,
  we find that first, TDMs perform well in a single-environment learning setting
  when compared to baseline models. Second, TDMs exhibit strong generalization
  capabilities to unseen environments, both in a few-shot setting, where a
  generalist model is fine-tuned with small amounts of data from the target
  environment, and in a zero-shot setting, where a generalist model is applied to
  an unseen environment without any further training. We further demonstrate that
  generalizing system dynamics can work much better than generalizing optimal
  behavior directly as a policy. This makes TDMs a promising ingredient for a
  foundation model of control.
paper_authors._id: 6466fe5f14e059dde8bc6bb1
paper_authors.name: Ingmar Schubert
paper_authors._id: 6466fe5f14e059dde8bc6bb2
paper_authors.name: Jingwei Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-05-26T09:06:32.879Z
paper_authors._id: 63ff47cc12c51862e5d9e2ce
paper_authors.avatarUrl: /avatars/fd08feaee58cebb3c4541eac8fe6bed6.svg
paper_authors.fullname: Jingwei Zhang
paper_authors.type: user
paper_authors.user: allanz
paper_authors._id: 6466fe5f14e059dde8bc6bb3
paper_authors.name: Jake Bruce
paper_authors._id: 6466fe5f14e059dde8bc6bb4
paper_authors.name: Sarah Bechtle
paper_authors._id: 6466fe5f14e059dde8bc6bb5
paper_authors.name: Emilio Parisotto
paper_authors._id: 6466fe5f14e059dde8bc6bb6
paper_authors.name: Martin Riedmiller
paper_authors._id: 6466fe5f14e059dde8bc6bb7
paper_authors.name: Jost Tobias Springenberg
paper_authors._id: 6466fe5f14e059dde8bc6bb8
paper_authors.name: Arunkumar Byravan
paper_authors._id: 6466fe5f14e059dde8bc6bb9
paper_authors.name: Leonard Hasenclever
paper_authors._id: 6466fe5f14e059dde8bc6bba
paper_authors.name: Nicolas Heess
paper_title: A Generalist Dynamics Model for Control
paper_summary: We investigate the use of transformer sequence models as dynamics models
  (TDMs) for control. In a number of experiments in the DeepMind control suite,
  we find that first, TDMs perform well in a single-environment learning setting
  when compared to baseline models. Second, TDMs exhibit strong generalization
  capabilities to unseen environments, both in a few-shot setting, where a
  generalist model is fine-tuned with small amounts of data from the target
  environment, and in a zero-shot setting, where a generalist model is applied to
  an unseen environment without any further training. We further demonstrate that
  generalizing system dynamics can work much better than generalizing optimal
  behavior directly as a policy. This makes TDMs a promising ingredient for a
  foundation model of control.
paper_ai_summary: Transformer sequence models as dynamics models (TDMs) excel in single-environment learning, generalize well to unseen environments, and outperform policy generalization in control tasks.
paper_ai_keywords: transformer sequence models dynamics models DeepMind control suite few-shot learning zero-shot learning generalist model policy generalization foundation model of control