I am trying to design an ensemble loss function for an autoencoder to minimize the loss of the target class as well as maximize the loss of the non-target class.
My current loss function works fine with just minimizing loss target_loss but starts giving nan values when I add the non_target_loss term to the overall loss.
target_loss = MSE(input, output) + sparsity_term
non_target_loss = MSE(input, output_hat) + sparsity_term_hat

# I subtract because the second term has to be maximized.
loss = target_loss - (Coeff)*non_target_loss.

I am unsure about my design, any suggestions for a better design?