It seems to be viable, but if you turn off the first two losses and only train with the custom_loss, I am not sure if there is intra-influence between the first two loss and the last one. You could try it and adopt a suitable way.