Thank you for the tips. Plotting the values certainly provides some insight. Enabling custom losses already allowed my model to reach ~20% accuracy, that’s over ~6% of what vanilla losses offered.
Cheers!