GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning
<|reference_start|>GTA: Generative Trajectory Augmentation with Guidance for Offline Reinforcement Learning: Offline Reinforcement Learning (Offline RL) presents challenges of learning effective decision-making policies from static datasets without any online interactions. Data augmentation techniques, such as noise injection and data synthesizing, aim to improve Q-function approximation by smoothing the learned state-action region. However, these methods often fall short of directly improving the quality of offline datasets, leading to suboptimal results. In response, we introduce \textbf{GTA}, Generative Trajectory Augmentation, a novel generative data augmentation approach designed to enrich offline data by augmenting trajectories to be both high-rewarding and dynamically plausible. GTA applies a diffusion model within the data augmentation framework. GTA partially noises original trajectories and then denoises them with classifier-free guidance via conditioning on amplified return value. Our results show that GTA, as a general data augmentation strategy, enhances the performance of widely used offline RL algorithms in both dense and sparse reward settings. Furthermore, we conduct a quality analysis of data augmented by GTA and demonstrate that GTA improves the quality of the data. Our code is available at https://github.com/Jaewoopudding/GTA<|reference_end|>
@article{lee2024gta:,
    title={GTA: Generative Trajectory Augmentation with Guidance for Offline
  Reinforcement Learning},
    author={Jaewoo Lee, Sujin Yun, Taeyoung Yun and Jinkyoo Park},
    journal={arXiv preprint arXiv:2405.16907},
    year={2024},
    archivePrefix={arXiv},
    eprint={2405.16907},
    primaryClass={cs.AI cs.LG}
}