Answers.Body: <p>Actually, I just started inspecting the entries further down in the leaderboard list, and there are in fact more modest architectures, e.g. <a href="https://gist.github.com/heerad/1983d50c6657a55298b67e69a2ceeb44#file-ddpg-pendulum-v0-py" rel="nofollow noreferrer">this one</a>, which uses 3 hidden layers with 8 units each.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-20T05:20:32.963Z
Answers.LastActivityDate: 2020-07-20T21:16:14.750Z
Answers.LastEditDate: 2020-07-20T21:16:14.750Z
Answers.UserDisplayName: Kris
Body: <p>I'm looking at some baseline implementations of RL agents on the <a href="https://gym.openai.com/envs/Pendulum-v0/" rel="nofollow noreferrer">Pendulum</a> environment. My guess was to use a relatively small neural net (~100 parameters).</p>
  <p>I'm comparing my solution with some baselines, e.g. the top entry on the <a href="https://github.com/openai/gym/wiki/Leaderboard#pendulum-v0" rel="nofollow noreferrer">Pendulum leaderboard</a>. The models for these solutions are typically <strong>huge</strong>, i.e. ~120k parameters. What's more, they use very large replay buffers as well, like ~1M transitions. Such model sizes seem warranted for Atari-like environments, but for something as small as the Pendulum, this seems like complete overkill to me.</p>
  <p>Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?</p>
  
Tags: <reinforcement-learning><architecture><environment><ddpg><weights>
Title: Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?
UserDisplayName: Kris
ThreadText: # Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?
  
  (Asked by: username_0 on 2020-07-20)
  
  I'm looking at some baseline implementations of RL agents on the [Pendulum](https://gym.openai.com/envs/Pendulum-v0/) environment. My guess was to use a relatively small neural net (~100 parameters).
  
  I'm comparing my solution with some baselines, e.g. the top entry on the [Pendulum leaderboard](https://github.com/openai/gym/wiki/Leaderboard#pendulum-v0). The models for these solutions are typically **huge**, i.e. ~120k parameters. What's more, they use very large replay buffers as well, like ~1M transitions. Such model sizes seem warranted for Atari-like environments, but for something as small as the Pendulum, this seems like complete overkill to me.
  
  Are there examples of agents that use a more modest number of parameters on Pendulum (or similar environments)?
  
  **Tags:** reinforcement-learning, architecture, environment, ddpg, weights
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_0 on 2020-07-20. Score: 1
  
  Actually, I just started inspecting the entries further down in the leaderboard list, and there are in fact more modest architectures, e.g. [this one](https://gist.github.com/heerad/1983d50c6657a55298b67e69a2ceeb44#file-ddpg-pendulum-v0-py), which uses 3 hidden layers with 8 units each.
  
  
  ---
  
  