I’m not sure, how you are calculating the loss_mask, but do you expect it to be negative, as it’s a bit unusual? If not, I guess this might cause your model to diverge and create invalid outputs in the end.