Answering my own question: apparently there was a change in how the weights (gamma) are initialized from BatchNorm to BatchNorm2d, going from random uniform to a vector of ones. Unfortunately, I don’t think this was documented anywhere and, for reasons I don’t really understand, had a huge impact on convergence for the model I was training. The good news is that I can initialize those weights myself and recover the previous behavior.
Anyway, thanks for your help!