ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control
<|reference_start|>ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control: We consider deep deterministic policy gradient (DDPG) in the context of reinforcement learning with sparse rewards. To enhance exploration, we introduce a search procedure, \emph{${\epsilon}{t}$-greedy}, which generates exploratory options for exploring less-visited states. We prove that search using $\epsilon t$-greedy has polynomial sample complexity under mild MDP assumptions. To more efficiently use the information provided by rewarded transitions, we develop a new dual experience replay buffer framework, \emph{GDRB}, and implement \emph{longest n-step returns}. The resulting algorithm, \emph{ETGL-DDPG}, integrates all three techniques: \bm{$\epsilon t$}-greedy, \textbf{G}DRB, and \textbf{L}ongest $n$-step, into DDPG. We evaluate ETGL-DDPG on standard benchmarks and demonstrate that it outperforms DDPG, as well as other state-of-the-art methods, across all tested sparse-reward continuous environments. Ablation studies further highlight how each strategy individually enhances the performance of DDPG in this setting.<|reference_end|>
@article{futuhi2024etgl-ddpg:,
    title={ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse
  Reward Continuous Control},
    author={Ehsan Futuhi, Shayan Karimi, Chao Gao, Martin M"uller},
    journal={arXiv preprint arXiv:2410.05225},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05225},
    primaryClass={cs.LG cs.RO stat.ML}
}