Learning the Target Network in Function Space
<|reference_start|>Learning the Target Network in Function Space: We focus on the task of learning the value function in the reinforcement learning (RL) setting. This task is often solved by updating a pair of online and target networks while ensuring that the parameters of these two networks are equivalent. We propose Lookahead-Replicate (LR), a new value-function approximation algorithm that is agnostic to this parameter-space equivalence. Instead, the LR algorithm is designed to maintain an equivalence between the two networks in the function space. This value-based equivalence is obtained by employing a new target-network update. We show that LR leads to a convergent behavior in learning the value function. We also present empirical results demonstrating that LR-based target-network updates significantly improve deep RL on the Atari benchmark.<|reference_end|>
@article{asadi2024learning,
    title={Learning the Target Network in Function Space},
    author={Kavosh Asadi, Yao Liu, Shoham Sabach, Ming Yin, Rasool Fakoor},
    journal={arXiv preprint arXiv:2406.01838},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.01838},
    primaryClass={cs.LG cs.AI}
}