FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning
<|reference_start|>FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning: In this work, we investigate how to leverage pre-trained visual-language models (VLM) for online Reinforcement Learning (RL). In particular, we focus on sparse reward tasks with pre-defined textual task descriptions. We first identify the problem of reward misalignment when applying VLM as a reward in RL tasks. To address this issue, we introduce a lightweight fine-tuning method, named Fuzzy VLM reward-aided RL (FuRL), based on reward alignment and relay RL. Specifically, we enhance the performance of SAC/DrQ baseline agents on sparse reward tasks by fine-tuning VLM representations and using relay RL to avoid local minima. Extensive experiments on the Meta-world benchmark tasks demonstrate the efficacy of the proposed method. Code is available at: https://github.com/fuyw/FuRL.<|reference_end|>
@article{fu2024furl:,
    title={FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning},
    author={Yuwei Fu, Haichao Zhang, Di Wu, Wei Xu, Benoit Boulet},
    journal={arXiv preprint arXiv:2406.00645},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.00645},
    primaryClass={cs.LG cs.AI cs.CV}
}