BatchNorm helps in the training of most Deep Networks, that said if your batch size is small (less than 8), it is known training with  BatchNorm yields sub optimal results, and using other normalization tricks like GroupNorm is helpful.
In ResNets BatchNorm is placed usually before a skip connection or just before the activation layer (ReLU).
Also, if you are using a shallow network (less than 6 layers), you can train your model without BatchNorm, and might experience marginally slow convergence, but forward pass for networks without BatchNorm is certainly less time consuming, and hence it might be a better trade off to train your model without BatchNorm, but experiments are certainly needed to evaluate this trade off, as things often depend on your dataset and other problem specific parameters.