Improved off-policy training of diffusion samplers
<|reference_start|>Improved off-policy training of diffusion samplers: We study the problem of training diffusion models to sample from a distribution with a given unnormalized density or energy function. We benchmark several diffusion-structured inference methods, including simulation-based variational approaches and off-policy methods (continuous generative flow networks). Our results shed light on the relative advantages of existing algorithms while bringing into question some claims from past work. We also propose a novel exploration strategy for off-policy methods, based on local search in the target space with the use of a replay buffer, and show that it improves the quality of samples on a variety of target distributions. Our code for the sampling methods and benchmarks studied is made public at https://github.com/GFNOrg/gfn-diffusion as a base for future work on diffusion models for amortized inference.<|reference_end|>
@article{sendera2024improved,
    title={Improved off-policy training of diffusion samplers},
    author={Marcin Sendera, Minsu Kim, Sarthak Mittal, Pablo Lemos, Luca Scimeca,
  Jarrid Rector-Brooks, Alexandre Adam, Yoshua Bengio, Nikolay Malkin},
    journal={arXiv preprint arXiv:2402.05098},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.05098},
    primaryClass={cs.LG stat.ML}
}