I had posted something similar before 3 but couldn’t get around the problem.
I want to how to properly scale the final loss when it’s composed of two or several other individual loss terms as is common in many use-cases like Image Generation. For example, for image generation, a typical loss function is the following:
total_loss = (alpha * perceptual_loss) + (beta * l1_loss)

Note that the non-reduced versions of perceptual_loss and l1_loss do have the same shapes. So, I guess it becomes necessary to reduce them before summing them up. The official guide 2 does provide some guidance but I could not find something concrete around this particular use-case.
Any pointers regarding this would be very helpful.