Hi @Hdk,
There could be a lot of things at play here depending on your custom_loss. I would typically try the following things,

If possible, try to make the custom loss’s lower bound as zero.
If the above is not possible, do Total_loss = (10/2.4)*cross_entropy_loss + custom_loss