Thanks for the timely reply! It’s very helpful. My reason for blaming BN layers is that I observe unreasonably large output (>1000) after BN layer with model.eval() is on. Also, models without BN layers does not suffer the discrepancy between Pytorch versions. I will try to hack the BN implementation and see if it resolves the problem.