Hi Gdx!



 gdx:

How to design a loss function to make sure that y1 - y2 > 0?


First a comment:
Because neural networks are trained by “backpropagating” a
differentiable loss function using gradient-descent methods, it
is much more natural to use a differentiable penalty to penalize
violations of your desired constraint, rather than impose a hard
constraint.  Note that such an approach does permit your constraint
to be violated,  but if your penalty is large enough, your constraint
won’t be violated by very much.
Along these lines, consider using:
penalty_loss = torch.exp (alpha * (y2 - y1))

(This is a straightforward choice for the penalty function but many
other choices would also make sense.)
The greater you make alpha the more you will penalize values
for which y1 - y2 < 0 (and the less the network will care about
the value of y1 - y2, as long as it is greater than zero).
I might imagine starting training with alpha set to a moderate value,
and then increasing it as training progresses.
Best.
K. Frank