Answers.Body: <p>Residual Network are usually deeper and hence take more time to train. EfficientNet are trying to tackle this. However, the latest advice show that the architecture tend to play a crucial role in the performance of an RL algorithm, which might motivate you to do this.</p>
  <p>There is recent work on Neural Architecture Search applied to RL tasks (cf <a href="https://arxiv.org/pdf/2106.02229.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2106.02229.pdf</a>)</p>
  <p>However, I can advise you to look at Rational Activation Functions (cf <a href="https://arxiv.org/pdf/2102.09407.pdf" rel="nofollow noreferrer">https://arxiv.org/pdf/2102.09407.pdf</a>) employed in RL which provide a huge boost to the agents, while adding negligible parameters and negligible extra training time.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-01T16:03:22.320Z
Answers.LastActivityDate: 2021-12-01T16:03:22.320Z
Answers.UserDisplayName: Quentin Delfosse
Body: <p>I'd like to use a residual network to improve learning in image-based reinforcement learning, specifically on Atari Games.</p>
  <p>My main question is divided into 3 parts.</p>
  <ol>
  <li><p>Would it be wise to integrate a generic ResNet with a DQN variant?</p>
  </li>
  <li><p>I believe ResNets take a long to train, and therefore would it be realistic to train on an Atari simulator? What would the downsides be?</p>
  </li>
  <li><p>Are there any fast ResNets that can be used for such purposes? Perhaps a fast ResNet that is specifically designed for online settings?</p>
  </li>
  </ol>
  
Tags: <reinforcement-learning><dqn><algorithm-request><residual-networks><atari-games>
Title: How can I use a ResNet as a function approximator for pixel based reinforcement learning?
UserDisplayName: desert_ranger
ThreadText: # How can I use a ResNet as a function approximator for pixel based reinforcement learning?
  
  (Asked by: username_0 on 2021-10-25)
  
  I'd like to use a residual network to improve learning in image-based reinforcement learning, specifically on Atari Games.
  
  My main question is divided into 3 parts.
  
  1. Would it be wise to integrate a generic ResNet with a DQN variant?
  2. I believe ResNets take a long to train, and therefore would it be realistic to train on an Atari simulator? What would the downsides be?
  3. Are there any fast ResNets that can be used for such purposes? Perhaps a fast ResNet that is specifically designed for online settings?
  
  **Tags:** reinforcement-learning, dqn, algorithm-request, residual-networks, atari-games
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-12-01. Score: 2 | Accepted answer
  
  Residual Network are usually deeper and hence take more time to train. EfficientNet are trying to tackle this. However, the latest advice show that the architecture tend to play a crucial role in the performance of an RL algorithm, which might motivate you to do this.
  
  There is recent work on Neural Architecture Search applied to RL tasks (cf <https://arxiv.org/pdf/2106.02229.pdf>)
  
  However, I can advise you to look at Rational Activation Functions (cf <https://arxiv.org/pdf/2102.09407.pdf>) employed in RL which provide a huge boost to the agents, while adding negligible parameters and negligible extra training time.
  
  
  ---
  
  