In a multioutput deep learning model, is there a benefit to normalizing the output dimensions if they are of different magnitudes? I am building a multi output deep learning model where the output consists of five dimensions (the specific architecture is a modification of YOLO). These have different magnitudes (ranges: [0, 1.2], [0, 20], [0, 1.2], [0, 1.5, 0, 2000])as shown in . I was wondering if scaling the output dimensions, for example by using max-min or standard scaling, such that each predicted dimension is roughly in [0, 1] improves training (as opposed to having different scales of magnitudes in each output dimension)? Currently, the largest dimension of the (un-scaled) output monopolizes the loss value but I'm not sure if that hampers the training in some way.
Details
Input consists of a sequence of real numbers, each number is in [0, 1]  example input: [0.15, 0.8, 0.67, 0.43]
Fully convolution network with ReLU activations and BatchNorm
Output layer is linear
Loss function is MSE for all outputs.

In theory, no. In practice, probably. The loss function you are minimizing is basically
$$ \min \ \frac{1}{N}\sum_{i=1}^{N}\sum_{k=1}^{K} (y_{ik} - f_k(x_i))^2 $$
where $x$ is features, $y$ is $K$-dimensional output.
Scaling the $y$'s makes no difference at all in terms of the global minima. However in practice and as you've noticed, the differing scales means that in the gradient steps the larger outcomes contribute "more" (gradients are larger) to the loss function. Thus a tiny improvement in the last output may be the same as a huge improvement in the first output where the scale is much smaller.
Whether this actually makes a difference largely depends on the dataset and your optimizer (for example if you are decaying the learning rate it might not matter much). Since neural networks generally converge to local minima and there's no downside to it, I would normalize the outputs for more stable training.
