May not be a bug. I think batchnorm generally (conceptually) doesn’t play nicely with Dropout. Maybe do an ablation study

no batchnorm, no dropout
just batchnorm, no dropout
just dropout, no batchnorm
both batchnorm and dropout

and see if you get further insights whether there’s a bug or not. If you have a bug, I suspect 3) will be much worse than 1) etc.