duddal:

all the other losses except txty_loss are 0, but txty_loss ranges between 10 to 17. I think the model gives bad predictions because it is dominated by txty_loss. Could this be right?


Wouldn’t this mean that the model training is respecting the other losses primarily (as they are reduced to 0), while txty_loss might increase?



 duddal:

If so, how do I add weights to these losses, as in: Assigning coefficients to each loss but keeping the sum of those coefficients 1 ?


Your approach sounds valid and you could assign coefficients to each loss term (and normalize them so that their sum equals 1).
E.g.:
total_loss = 0.2 * cls_loss + 0.2 * txty_loss + 0.2 * twth_loss + 0.2 * iou_loss + 0.2 * dep_loss 

(of course you might want to change the values)