Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning
<|reference_start|>Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning: Recently, various pre-training methods have been introduced in vision-based Reinforcement Learning (RL). However, their generalization ability remains unclear due to evaluations being limited to in-distribution environments and non-unified experimental setups. To address this, we introduce the Atari Pre-training Benchmark (Atari-PB), which pre-trains a ResNet-50 model on 10 million transitions from 50 Atari games and evaluates it across diverse environment distributions. Our experiments show that pre-training objectives focused on learning task-agnostic features (e.g., identifying objects and understanding temporal dynamics) enhance generalization across different environments. In contrast, objectives focused on learning task-specific knowledge (e.g., identifying agents and fitting reward functions) improve performance in environments similar to the pre-training dataset but not in varied ones. We publicize our codes, datasets, and model checkpoints at https://github.com/dojeon-ai/Atari-PB.<|reference_end|>
@article{kim2024investigating,
    title={Investigating Pre-Training Objectives for Generalization in Vision-Based
  Reinforcement Learning},
    author={Donghu Kim, Hojoon Lee, Kyungmin Lee, Dongyoon Hwang, Jaegul Choo},
    journal={arXiv preprint arXiv:2406.06037},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.06037},
    primaryClass={cs.LG cs.AI cs.CV}
}