Especially if your model is deep, i.e. it contains a lot of layers, the initialization is an important step to let the model train. It was one of the key ingredients to be able to train deep neural networks at all, so I’m not surprised if a lot if initialization methods fail.
If your loss is increasing, your model might be diverging. Sometimes this is due to the missed step of zeroing out the gradients, but I assume you haven’t changed the training loop.