title: Masked Trajectory Models for Prediction, Representation, and Control
summary: We introduce Masked Trajectory Models (MTM) as a generic abstraction for
  sequential decision making. MTM takes a trajectory, such as a state-action
  sequence, and aims to reconstruct the trajectory conditioned on random subsets
  of the same trajectory. By training with a highly randomized masking pattern,
  MTM learns versatile networks that can take on different roles or capabilities,
  by simply choosing appropriate masks at inference time. For example, the same
  MTM network can be used as a forward dynamics model, inverse dynamics model, or
  even an offline RL agent. Through extensive experiments in several continuous
  control tasks, we show that the same MTM network -- i.e. same weights -- can
  match or outperform specialized networks trained for the aforementioned
  capabilities. Additionally, we find that state representations learned by MTM
  can significantly accelerate the learning speed of traditional RL algorithms.
  Finally, in offline RL benchmarks, we find that MTM is competitive with
  specialized offline RL algorithms, despite MTM being a generic self-supervised
  learning method without any explicit RL components. Code is available at
  https://github.com/facebookresearch/mtm
paper_authors._id: 64548367ec40bbbd012552d0
paper_authors.name: Philipp Wu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-26T08:07:36.297Z
paper_authors._id: 62f6adc3d3bdacb7eec0f549
paper_authors.avatarUrl: /avatars/309609c044689b6770f14bcd5b9bbb68.svg
paper_authors.fullname: Philipp
paper_authors.type: user
paper_authors.user: phil80301
paper_authors._id: 64548367ec40bbbd012552d1
paper_authors.name: Arjun Majumdar
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-26T17:14:34.440Z
paper_authors._id: 631bd5f95b628ccf3ced4fc7
paper_authors.avatarUrl: /avatars/5e8c4c73b53754008a844e081ce11a7b.svg
paper_authors.fullname: Arjun Majumdar
paper_authors.type: user
paper_authors.user: arjunmajum
paper_authors._id: 64548367ec40bbbd012552d2
paper_authors.name: Kevin Stone
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-08-08T14:20:10.462Z
paper_authors._id: 636d2fb156c0762cfd9b9910
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/636d2fb156c0762cfd9b9910/QgWJ8_52NjeqUexu8vQtM.jpeg
paper_authors.fullname: Kevin Lee Stone
paper_authors.type: user
paper_authors.user: kevinleestone
paper_authors._id: 64548367ec40bbbd012552d3
paper_authors.name: Yixin Lin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-25T15:21:08.951Z
paper_authors._id: 646f7c22799a974be3191d54
paper_authors.avatarUrl: /avatars/51633ce1caf47f543c5d76f53d0aeecf.svg
paper_authors.fullname: Yixin Lin
paper_authors.type: user
paper_authors.user: 1heart
paper_authors._id: 64548367ec40bbbd012552d4
paper_authors.name: Igor Mordatch
paper_authors._id: 64548367ec40bbbd012552d5
paper_authors.name: Pieter Abbeel
paper_authors._id: 64548367ec40bbbd012552d6
paper_authors.name: Aravind Rajeswaran
paper_title: Masked Trajectory Models for Prediction, Representation, and Control
paper_summary: We introduce Masked Trajectory Models (MTM) as a generic abstraction for
  sequential decision making. MTM takes a trajectory, such as a state-action
  sequence, and aims to reconstruct the trajectory conditioned on random subsets
  of the same trajectory. By training with a highly randomized masking pattern,
  MTM learns versatile networks that can take on different roles or capabilities,
  by simply choosing appropriate masks at inference time. For example, the same
  MTM network can be used as a forward dynamics model, inverse dynamics model, or
  even an offline RL agent. Through extensive experiments in several continuous
  control tasks, we show that the same MTM network -- i.e. same weights -- can
  match or outperform specialized networks trained for the aforementioned
  capabilities. Additionally, we find that state representations learned by MTM
  can significantly accelerate the learning speed of traditional RL algorithms.
  Finally, in offline RL benchmarks, we find that MTM is competitive with
  specialized offline RL algorithms, despite MTM being a generic self-supervised
  learning method without any explicit RL components. Code is available at
  https://github.com/facebookresearch/mtm
paper_ai_summary: Masked Trajectory Models (MTM) are versatile sequential decision-making models that can perform tasks such as forward dynamics modeling, inverse dynamics modeling, and offline RL using generic masks and representations, matching or outperforming specialized methods.
paper_ai_keywords: Masked Trajectory Models MTM sequential decision making trajectory state-action sequence forward dynamics model inverse dynamics model offline RL agent state representations traditional RL algorithms