I see. Thank you
Okay, I will see if I can create differentiable variation of the loss function I am looking for.
Let me rephrase your passage about softmax, just to confirm that I understood correctly.
Softmax is essentially continuous version of argmax as it also takes in the same input but returns very similar outputs (1 if the greatest else 0).
And while argmax is not differentiable softmax is, so it can nicely replaces argmax while enabling backprop through the framework.
I think this is pretty brilliant and I feel like I kinda know what I need to do. Thanks