When adding batch norm layer do I need to added to all layers in DNN? While developing deepfm model network I want to add batch norm layer because model seems to suffer from vanishing gradient. There are embedding layers,  2 layers a in deep model part and one dense that connects fm model and deep model. If I want to add batch norm which layer do I added on? What do I have to look at to judge certain layers needs batch norm?


What do I have to look at to judge certain layers needs batch norm?

If the layer inputs suffer from small/large values that jeopardize the training, then batch normalization will be useful there. For this, you need to diagnose your model by plotting the histograms of each layer's input to see where it gets out of control.
