Highway Value Iteration Networks
<|reference_start|>Highway Value Iteration Networks: Value iteration networks (VINs) enable end-to-end learning for planning tasks by employing a differentiable "planning module" that approximates the value iteration algorithm. However, long-term planning remains a challenge because training very deep VINs is difficult. To address this problem, we embed highway value iteration -- a recent algorithm designed to facilitate long-term credit assignment -- into the structure of VINs. This improvement augments the "planning module" of the VIN with three additional components: 1) an "aggregate gate," which constructs skip connections to improve information flow across many layers; 2) an "exploration module," crafted to increase the diversity of information and gradient flow in spatial dimensions; 3) a "filter gate" designed to ensure safe exploration. The resulting novel highway VIN can be trained effectively with hundreds of layers using standard backpropagation. In long-term planning tasks requiring hundreds of planning steps, deep highway VINs outperform both traditional VINs and several advanced, very deep NNs.<|reference_end|>
@article{wang2024highway,
    title={Highway Value Iteration Networks},
    author={Yuhui Wang, Weida Li, Francesco Faccio, Qingyuan Wu, J"urgen
  Schmidhuber},
    journal={arXiv preprint arXiv:2406.03485},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.03485},
    primaryClass={cs.LG cs.AI}
}