That’s because all usual weight initializers are trying to preserve mean & variance, but it is only possible if input moments are known - so they assume normalized inputs (0,1). With input mean != 0, depending on layer and weight shape, layer output’s mean can be pushed far away from zero, variance may increase, therefore you can get a bunch of well-known problems (saturation, flat loss region, etc)
If you mean that final output normalization helped you, this again means that before normalization network outputs were in a bad region (i.e. mean 100)