<p>I would say it is the nature of data. Generally speaking, you are trying to predict a random sequence, especially if you use the history data as an input and try to get the future value as an output.</p>

CC BY-SA 4.0
2020-08-04T12:05:45.507Z
Yes , I am trying another approach now by extracting more data from the temporal series ( historical prices ) by augmenting the data itself, it shows better results but still unstable.
Zahra
CC BY-SA 4.0
2020-06-16T08:57:18.200Z
2020-06-16T08:57:18.200Z
oleg.mosalov
<p>I'm trying to implement a soft actor-critic algorithm for financial data (stock prices), but I have trouble with losses: no matter what combination of hyper-parameters I enter, they are not converging, and basically it caused bad reward return as well. It sounds like the agent is not learning at all.</p>
<p>I already tried to tune some hyperparameters (learning rate for each network + number of hidden layers), but I always get similar results.
The two plots below represent the losses of my policy and one of the value functions during the last episode of training.</p>
<p><a href="https://i.sstatic.net/r1xdn.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/r1xdn.png" alt="enter image description here" /></a></p>
<p><a href="https://i.sstatic.net/3LtkW.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/3LtkW.png" alt="enter image description here" /></a></p>
<p>My question is, <strong>would it be related to the data itself (nature of data) or is it something related to the logic of the code?</strong></p>

<reinforcement-learning><tensorflow><actor-critic-methods><hyperparameter-optimization><soft-actor-critic>
Why is my Soft Actor-Critic's policy and value function losses not converging?
Zahra
# Why is my Soft Actor-Critic's policy and value function losses not converging?

(Asked by: username_0 on 2020-06-14)

I'm trying to implement a soft actor-critic algorithm for financial data (stock prices), but I have trouble with losses: no matter what combination of hyper-parameters I enter, they are not converging, and basically it caused bad reward return as well. It sounds like the agent is not learning at all.

I already tried to tune some hyperparameters (learning rate for each network + number of hidden layers), but I always get similar results.
The two plots below represent the losses of my policy and one of the value functions during the last episode of training.

[![enter image description here](https://i.sstatic.net/r1xdn.png)](https://i.sstatic.net/r1xdn.png)

[![enter image description here](https://i.sstatic.net/3LtkW.png)](https://i.sstatic.net/3LtkW.png)

My question is, **would it be related to the data itself (nature of data) or is it something related to the logic of the code?**

**Tags:** reinforcement-learning, tensorflow, actor-critic-methods, hyperparameter-optimization, soft-actor-critic


---

## 1 answers

### Answer by username_1 on 2020-06-16. Score: 0 | Accepted answer

I would say it is the nature of data. Generally speaking, you are trying to predict a random sequence, especially if you use the history data as an input and try to get the future value as an output.

> **Comments:**
> **username_0**: Yes , I am trying another approach now by extracting more data from the temporal series ( historical prices ) by augmenting the data itself, it shows better results but still unstable.  

---

