Elastic Decision Transformer
This paper introduces Elastic Decision Transformer (EDT), a significant
advancement over the existing Decision Transformer (DT) and its variants.
Although DT purports to generate an optimal trajectory, empirical evidence
suggests it struggles with trajectory stitching, a process involving the
generation of an optimal or near-optimal trajectory from the best parts of a
set of sub-optimal trajectories. The proposed EDT differentiates itself by
facilitating trajectory stitching during action inference at test time,
achieved by adjusting the history length maintained in DT. Further, the EDT
optimizes the trajectory by retaining a longer history when the previous
trajectory is optimal and a shorter one when it is sub-optimal, enabling it to
"stitch" with a more optimal trajectory. Extensive experimentation demonstrates
EDT's ability to bridge the performance gap between DT-based and Q
Learning-based approaches. In particular, the EDT outperforms Q Learning-based
methods in a multi-task regime on the D4RL locomotion benchmark and Atari
games. Videos are available at: https://kristery.github.io/edt/
64a650289f3b568c2037fd07
Yueh-Hua Wu
64a650289f3b568c2037fd08
Xiaolong Wang
64a650289f3b568c2037fd09
Masashi Hamaya
Elastic Decision Transformer
This paper introduces Elastic Decision Transformer (EDT), a significant
advancement over the existing Decision Transformer (DT) and its variants.
Although DT purports to generate an optimal trajectory, empirical evidence
suggests it struggles with trajectory stitching, a process involving the
generation of an optimal or near-optimal trajectory from the best parts of a
set of sub-optimal trajectories. The proposed EDT differentiates itself by
facilitating trajectory stitching during action inference at test time,
achieved by adjusting the history length maintained in DT. Further, the EDT
optimizes the trajectory by retaining a longer history when the previous
trajectory is optimal and a shorter one when it is sub-optimal, enabling it to
"stitch" with a more optimal trajectory. Extensive experimentation demonstrates
EDT's ability to bridge the performance gap between DT-based and Q
Learning-based approaches. In particular, the EDT outperforms Q Learning-based
methods in a multi-task regime on the D4RL locomotion benchmark and Atari
games. Videos are available at: https://kristery.github.io/edt/
Elastic Decision Transformer improves upon Decision Transformer by facilitating trajectory stitching through dynamic history adjustment, achieving performance on par with Q Learning in multi-task scenarios.
Elastic Decision Transformer Decision Transformer trajectory stitching history length optimal trajectory sub-optimal trajectory D4RL locomotion benchmark Atari games