Body: <p>When I read <a href="https://arxiv.org/abs/1801.01290" rel="nofollow noreferrer">SAC paper</a>, they use state value function <span class="math-container">$V_\psi(s_t)$</span> and as do <a href="https://github.com/haarnoja/sac" rel="nofollow noreferrer">their implementation</a>. But in other SAC implementations like <a href="https://github.com/DLR-RM/stable-baselines3" rel="nofollow noreferrer">stable-baseline3</a>, <a href="https://github.com/pranz24/pytorch-soft-actor-critic" rel="nofollow noreferrer">pytorch-soft-actor-critic</a>, it seems that they don't use seperate state value function. So, I wondered they are improved version than the original paper?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-07T11:22:22.613Z
Comments.Text: For people that aren't so familiar with the details of a "stable value function", can you briefly describe what it is?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-01-13T12:12:08.470Z
Comments.Text: Most implementations follow [this paper](https://arxiv.org/abs/1812.05905) instead. They concluded that the state-value function is not necessary, making the implementation faster (as we have one network less to train)
Comments.UserDisplayName: matorbi
Tags: <reinforcement-learning><deep-learning><deep-rl><soft-actor-critic>
Title: Why don't SAC implementations use state value function?
UserDisplayName: DevSlem
ThreadText: # Why don't SAC implementations use state value function?
  
  (Asked by: username_0 on 2022-12-31)
  
  When I read [SAC paper](https://arxiv.org/abs/1801.01290), they use state value function $V\_\psi(s\_t)$ and as do [their implementation](https://github.com/haarnoja/sac). But in other SAC implementations like [stable-baseline3](https://github.com/DLR-RM/stable-baselines3), [pytorch-soft-actor-critic](https://github.com/pranz24/pytorch-soft-actor-critic), it seems that they don't use seperate state value function. So, I wondered they are improved version than the original paper?
  
  **Tags:** reinforcement-learning, deep-learning, deep-rl, soft-actor-critic
  
  > **Comments:**
  > **username_1**: For people that aren't so familiar with the details of a "stable value function", can you briefly describe what it is?  
  > **username_2**: Most implementations follow [this paper](https://arxiv.org/abs/1812.05905) instead. They concluded that the state-value function is not necessary, making the implementation faster (as we have one network less to train)  
  
  ---
  
  ## 0 answers
  
  