<p>As I understand from literature, normally, the last activation in an actor (policy) network in TD3 and SAC algorithms is a Tanh function, which is scaled by a certain limit.</p>
<p>My action vector is perfectly described as a vector, where all values are between 0 and 1, and which should sum up to 1. This is perfect for a Softmax function. But these values are not probabilities of discrete actions. Each value in action vector should be a percentage from the whole portfolio to be invested in a certain stock.</p>
<p>But I cannot figure out, if it would be mathematically fine to use Softmax as an activation layer in TD3 or SAC?</p>

CC BY-SA 4.0
2021-08-09T23:12:44.053Z
You should check out discrete SAC. Original SAC was designed for continuous action spaces (the code was, at least — I believe theoretically the paper works with both but was motivated by continuous action spaces) but I believe they developed a discrete version. I don’t have the reference to hand but a Google search should turn up results.
David
CC BY-SA 4.0
2021-08-10T07:53:22.007Z
@DavidIreland, my action space is not discrete actually. I did not imply probabilities by these values. Each value in action vector is actually a percentage from the whole portfolio to be invested in a certain stock.
Bi0max
CC BY-SA 4.0
2021-08-10T12:31:25.143Z
Ah, okay, I apologise. In which case then you can just use SAC with a soft max activation. I believe tanh is usually used because they have symmetric intervals for the action space, it is not something it relies on theoretically.
David
CC BY-SA 4.0
2021-08-11T13:08:35.863Z
@DavidIreland, thank you for the answer. I also assumed so, but I was not sure whether it's mathematically fine or not.
Bi0max
CC BY-SA 4.0
2021-08-11T16:15:29.893Z
Yes, it should be fine :)
David
<reinforcement-learning><deep-rl><activation-functions><soft-actor-critic><td3>
Is it possible to use Softmax as an activation function for actor (policy) network in TD3 or SAC Reinforcement learning algorithms?
Bi0max
# Is it possible to use Softmax as an activation function for actor (policy) network in TD3 or SAC Reinforcement learning algorithms?

(Asked by: username_0 on 2021-08-09)

As I understand from literature, normally, the last activation in an actor (policy) network in TD3 and SAC algorithms is a Tanh function, which is scaled by a certain limit.

My action vector is perfectly described as a vector, where all values are between 0 and 1, and which should sum up to 1. This is perfect for a Softmax function. But these values are not probabilities of discrete actions. Each value in action vector should be a percentage from the whole portfolio to be invested in a certain stock.

But I cannot figure out, if it would be mathematically fine to use Softmax as an activation layer in TD3 or SAC?

**Tags:** reinforcement-learning, deep-rl, activation-functions, soft-actor-critic, td3

> **Comments:**
> **username_1**: You should check out discrete SAC. Original SAC was designed for continuous action spaces (the code was, at least — I believe theoretically the paper works with both but was motivated by continuous action spaces) but I believe they developed a discrete version. I don’t have the reference to hand but a Google search should turn up results.  
> **username_0**: @username_1Ireland, my action space is not discrete actually. I did not imply probabilities by these values. Each value in action vector is actually a percentage from the whole portfolio to be invested in a certain stock.  
> **username_1**: Ah, okay, I apologise. In which case then you can just use SAC with a soft max activation. I believe tanh is usually used because they have symmetric intervals for the action space, it is not something it relies on theoretically.  
> **username_0**: @username_1Ireland, thank you for the answer. I also assumed so, but I was not sure whether it's mathematically fine or not.  
> **username_1**: Yes, it should be fine :)  

---

## 0 answers

