Masked Trajectory Models for Prediction, Representation, and Control
We introduce Masked Trajectory Models (MTM) as a generic abstraction for
sequential decision making. MTM takes a trajectory, such as a state-action
sequence, and aims to reconstruct the trajectory conditioned on random subsets
of the same trajectory. By training with a highly randomized masking pattern,
MTM learns versatile networks that can take on different roles or capabilities,
by simply choosing appropriate masks at inference time. For example, the same
MTM network can be used as a forward dynamics model, inverse dynamics model, or
even an offline RL agent. Through extensive experiments in several continuous
control tasks, we show that the same MTM network -- i.e. same weights -- can
match or outperform specialized networks trained for the aforementioned
capabilities. Additionally, we find that state representations learned by MTM
can significantly accelerate the learning speed of traditional RL algorithms.
Finally, in offline RL benchmarks, we find that MTM is competitive with
specialized offline RL algorithms, despite MTM being a generic self-supervised
learning method without any explicit RL components. Code is available at
https://github.com/facebookresearch/mtm
64548367ec40bbbd012552d0
Philipp Wu
claimed_verified
2023-05-26T08:07:36.297Z
62f6adc3d3bdacb7eec0f549
/avatars/309609c044689b6770f14bcd5b9bbb68.svg
Philipp
user
phil80301
64548367ec40bbbd012552d1
Arjun Majumdar
claimed_verified
2023-05-26T17:14:34.440Z
631bd5f95b628ccf3ced4fc7
/avatars/5e8c4c73b53754008a844e081ce11a7b.svg
Arjun Majumdar
user
arjunmajum
64548367ec40bbbd012552d2
Kevin Stone
claimed_verified
2023-08-08T14:20:10.462Z
636d2fb156c0762cfd9b9910
https://cdn-avatars.huggingface.co/v1/production/uploads/636d2fb156c0762cfd9b9910/QgWJ8_52NjeqUexu8vQtM.jpeg
Kevin Lee Stone
user
kevinleestone
64548367ec40bbbd012552d3
Yixin Lin
claimed_verified
2023-05-25T15:21:08.951Z
646f7c22799a974be3191d54
/avatars/51633ce1caf47f543c5d76f53d0aeecf.svg
Yixin Lin
user
1heart
64548367ec40bbbd012552d4
Igor Mordatch
64548367ec40bbbd012552d5
Pieter Abbeel
64548367ec40bbbd012552d6
Aravind Rajeswaran
Masked Trajectory Models for Prediction, Representation, and Control
We introduce Masked Trajectory Models (MTM) as a generic abstraction for
sequential decision making. MTM takes a trajectory, such as a state-action
sequence, and aims to reconstruct the trajectory conditioned on random subsets
of the same trajectory. By training with a highly randomized masking pattern,
MTM learns versatile networks that can take on different roles or capabilities,
by simply choosing appropriate masks at inference time. For example, the same
MTM network can be used as a forward dynamics model, inverse dynamics model, or
even an offline RL agent. Through extensive experiments in several continuous
control tasks, we show that the same MTM network -- i.e. same weights -- can
match or outperform specialized networks trained for the aforementioned
capabilities. Additionally, we find that state representations learned by MTM
can significantly accelerate the learning speed of traditional RL algorithms.
Finally, in offline RL benchmarks, we find that MTM is competitive with
specialized offline RL algorithms, despite MTM being a generic self-supervised
learning method without any explicit RL components. Code is available at
https://github.com/facebookresearch/mtm
Masked Trajectory Models (MTM) are versatile sequential decision-making models that can perform tasks such as forward dynamics modeling, inverse dynamics modeling, and offline RL using generic masks and representations, matching or outperforming specialized methods.
Masked Trajectory Models MTM sequential decision making trajectory state-action sequence forward dynamics model inverse dynamics model offline RL agent state representations traditional RL algorithms