<p>I'm working on a project where a non differentiable loss is there. I'm thinking about how should I deal with them.</p>
<p>My model is a very big lstm model (about 1M parameter), and after 500 steps (not sure if it's enough) of <a href="https://en.m.wikipedia.org/wiki/Differential_evolution" rel="nofollow noreferrer">differential evolution</a>, the loss didn't decrease at all.</p>
<p>I'm thinking that actor critic can see as follows:
The policy network updates by value function, which is calculated from a reward estimator (Critic).</p>
<p>I'm wondering if I can take the thought of &quot;estimator&quot;, and use it on loss estimation, then I can replace the non differentiable loss as that estimator, then also train the estimator each step. Maybe someone has already done that, if so, I want to know the name of the method.</p>

CC BY-SA 4.0
2024-01-24T09:54:27.810Z
what's your loss? what differential evolution method did you use? CE/CMA-ES?
Alberto
CC BY-SA 4.0
2024-01-24T10:36:53.033Z
The part make the loss function not differentiable is "argmax" and "split the list by a special element". And I think DE is an algorithm just like cma-es.
TWTom
CC BY-SA 4.0
2024-01-24T12:27:59.140Z
i mean you can definitely try reinforce, not quite sure how to help without a formal definition of the loss
Alberto
<objective-functions><optimization><actor-critic-methods>
Non differentiable loss function train with actor critic style
TWTom
# Non differentiable loss function train with actor critic style

(Asked by: username_0 on 2024-01-24)

I'm working on a project where a non differentiable loss is there. I'm thinking about how should I deal with them.

My model is a very big lstm model (about 1M parameter), and after 500 steps (not sure if it's enough) of [differential evolution](https://en.m.wikipedia.org/wiki/Differential_evolution), the loss didn't decrease at all.

I'm thinking that actor critic can see as follows:
The policy network updates by value function, which is calculated from a reward estimator (Critic).

I'm wondering if I can take the thought of "estimator", and use it on loss estimation, then I can replace the non differentiable loss as that estimator, then also train the estimator each step. Maybe someone has already done that, if so, I want to know the name of the method.

**Tags:** objective-functions, optimization, actor-critic-methods

> **Comments:**
> **username_1**: what's your loss? what differential evolution method did you use? CE/CMA-ES?  
> **username_0**: The part make the loss function not differentiable is "argmax" and "split the list by a special element". And I think DE is an algorithm just like cma-es.  
> **username_1**: i mean you can definitely try reinforce, not quite sure how to help without a formal definition of the loss  

---

## 0 answers

