Hi wgu!



 wgu93:

my loss function is to penalize the violation of monotonicity of two series of data.


If I understand your use case correctly, when you have
x[i + 1] > x[i], you would also like to have y[i + 1] > y[i],
and you would like to impose a penalty if y[i + 1] < y[i].  (And,
analogously, switching < for >.)
But you don’t care how much this joint monotonicity is violated.  If
y[i + 1] is smaller than y[i] by just a little bit, you want to penalize
it by essentially the same amount as when y[i + 1] is smaller by a
lot.
Also, I assume that you want the gradient of your “non-monotonicity
loss” with respect to the values of x and y.  For example, perhaps
x and y are generated by a neural network, and you want to train
that network to only generate such data series that are monotone
with respect to one another.  Thus, you would want to backpropagate
the gradient of your loss through x and y to the network parameters
that generated them.
You don’t need (nor I think want) to use sorting for this.
torch.tanh() is a differentiable (“soft”) approximation to the
sign() function.  So you could use (in vector form):
loss = (1.0 - (torch.tanh (alpha * (x[1:] - x[:-1])) * torch.tanh (alpha * (y[1:] - y[:1])))).mean()

where alpha can be used to sharpen (or smooth out) the abruptness
with which tanh() approximates sign().
Best.
K. Frank