  Diffusion models have emerged as a prominent class of generative models,
surpassing previous methods regarding sample quality and training stability.
Recent works have shown the advantages of diffusion models in improving
reinforcement learning (RL) solutions, including as trajectory planners,
expressive policy classes, data synthesizers, etc. This survey aims to provide
an overview of the advancements in this emerging field and hopes to inspire new
avenues of research. First, we examine several challenges encountered by
current RL algorithms. Then, we present a taxonomy of existing methods based on
the roles played by diffusion models in RL and explore how the existing
challenges are addressed. We further outline successful applications of
diffusion models in various RL-related tasks while discussing the limitations
of current approaches. Finally, we conclude the survey and offer insights into
future research directions, focusing on enhancing model performance and
applying diffusion models to broader tasks. We are actively maintaining a
GitHub repository for papers and other related resources in applying diffusion
models in RL: https://github.com/apexrl/Diff4RLSurvey

Machine Learning;Artificial Intelligence