This is a snippet from my network code:
conv_block += [nn.Conv2d(in_dim, out_dim, kernel_size=3, stride=1, padding=1), nn.BatchNorm2d(out_dim), nn.ELU()]
with BatchNorm2d. The network gave good results for first few training epochs but after that it get unstable.
During debugging, I removed BatchNorm2d from the network to analysis the effect:
conv_block += [nn.Conv2d(in_dim, out_dim, kernel_size=3, stride=1, padding=1), nn.ELU()]
but result were very bad and was not even comparable with the version with BN.
Why is it so?