PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators
<|reference_start|>PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful Navigators: We present PoliFormer (Policy Transformer), an RGB-only indoor navigation agent trained end-to-end with reinforcement learning at scale that generalizes to the real-world without adaptation despite being trained purely in simulation. PoliFormer uses a foundational vision transformer encoder with a causal transformer decoder enabling long-term memory and reasoning. It is trained for hundreds of millions of interactions across diverse environments, leveraging parallelized, multi-machine rollouts for efficient training with high throughput. PoliFormer is a masterful navigator, producing state-of-the-art results across two distinct embodiments, the LoCoBot and Stretch RE-1 robots, and four navigation benchmarks. It breaks through the plateaus of previous work, achieving an unprecedented 85.5% success rate in object goal navigation on the CHORES-S benchmark, a 28.5% absolute improvement. PoliFormer can also be trivially extended to a variety of downstream applications such as object tracking, multi-object navigation, and open-vocabulary navigation with no finetuning.<|reference_end|>
@article{zeng2024poliformer:,
    title={PoliFormer: Scaling On-Policy RL with Transformers Results in Masterful
  Navigators},
    author={Kuo-Hao Zeng, Zichen Zhang, Kiana Ehsani, Rose Hendrix, Jordi
  Salvador, Alvaro Herrasti, Ross Girshick, Aniruddha Kembhavi, Luca Weihs},
    journal={arXiv preprint arXiv:2406.20083},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.20083},
    primaryClass={cs.RO cs.CV}
}