A Generalist Dynamics Model for Control
We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.
6466fe5f14e059dde8bc6bb1
Ingmar Schubert
6466fe5f14e059dde8bc6bb2
Jingwei Zhang
admin_assigned
2023-05-26T09:06:32.879Z
63ff47cc12c51862e5d9e2ce
/avatars/fd08feaee58cebb3c4541eac8fe6bed6.svg
Jingwei Zhang
user
allanz
6466fe5f14e059dde8bc6bb3
Jake Bruce
6466fe5f14e059dde8bc6bb4
Sarah Bechtle
6466fe5f14e059dde8bc6bb5
Emilio Parisotto
6466fe5f14e059dde8bc6bb6
Martin Riedmiller
6466fe5f14e059dde8bc6bb7
Jost Tobias Springenberg
6466fe5f14e059dde8bc6bb8
Arunkumar Byravan
6466fe5f14e059dde8bc6bb9
Leonard Hasenclever
6466fe5f14e059dde8bc6bba
Nicolas Heess
A Generalist Dynamics Model for Control
We investigate the use of transformer sequence models as dynamics models
(TDMs) for control. In a number of experiments in the DeepMind control suite,
we find that first, TDMs perform well in a single-environment learning setting
when compared to baseline models. Second, TDMs exhibit strong generalization
capabilities to unseen environments, both in a few-shot setting, where a
generalist model is fine-tuned with small amounts of data from the target
environment, and in a zero-shot setting, where a generalist model is applied to
an unseen environment without any further training. We further demonstrate that
generalizing system dynamics can work much better than generalizing optimal
behavior directly as a policy. This makes TDMs a promising ingredient for a
foundation model of control.
Transformer sequence models as dynamics models (TDMs) excel in single-environment learning, generalize well to unseen environments, and outperform policy generalization in control tasks.
transformer sequence models dynamics models DeepMind control suite few-shot learning zero-shot learning generalist model policy generalization foundation model of control