<p>I am using stable-baseline3 implementation of the Soft-Actor-Critic (SAC) algorithm. The plotted training curves look promising. However, I am not fully sure how to interpret the actor and critic losses. The entropy coefficient <span class="math-container">$\alpha$</span> is automatically learned during training. As the entropy decreases, the critic loss and actor loss decrease as well.</p>
<ul>
<li>How does the entropy coefficient affect the losses?</li>
<li>Can this be interpreted as the estimations becoming more accurate as the focus is shifted from exploration to exploitation?</li>
<li>How can negative actor losses be interpreted, what do actor losses tell in general?</li>
</ul>
<p><a href="https://i.sstatic.net/0Rt2h.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/0Rt2h.png" alt="enter image description here" /></a></p>
<p>Thanks a lot in advance</p>

CC BY-SA 4.0
2021-12-06T16:29:28.327Z
I am getting negative actor loss in SAC implementation from stable-baseline3. Anything on this?
devil in the detail
CC BY-SA 4.0
2022-06-06T15:12:38.870Z
@devilinthedetail that's expected and good, if expected return is positive :)
kmf
<deep-rl><actor-critic-methods><loss><soft-actor-critic><learning-curve>
How to interpret the training loss curves in Soft-Actor-Critic (SAC)?
Manuel
# How to interpret the training loss curves in Soft-Actor-Critic (SAC)?

(Asked by: username_0 on 2021-07-01)

I am using stable-baseline3 implementation of the Soft-Actor-Critic (SAC) algorithm. The plotted training curves look promising. However, I am not fully sure how to interpret the actor and critic losses. The entropy coefficient $\alpha$ is automatically learned during training. As the entropy decreases, the critic loss and actor loss decrease as well.

* How does the entropy coefficient affect the losses?
* Can this be interpreted as the estimations becoming more accurate as the focus is shifted from exploration to exploitation?
* How can negative actor losses be interpreted, what do actor losses tell in general?

[![enter image description here](https://i.sstatic.net/0Rt2h.png)](https://i.sstatic.net/0Rt2h.png)

Thanks a lot in advance

**Tags:** deep-rl, actor-critic-methods, loss, soft-actor-critic, learning-curve

> **Comments:**
> **username_1**: I am getting negative actor loss in SAC implementation from stable-baseline3. Anything on this?  
> **username_2**: @devilinthedetail that's expected and good, if expected return is positive :)  

---

## 0 answers

