Yes, exactly.
If the statistics differ a lot for both feature sets, the parameters might not be able to get useful information from both inputs.
It’s similar to passing a normalized image after training on unnormalized ones (in [0, 255]).
The normalized image might just “look” like a black image to the model.