That’s interesting. I’ve run 167 epochs of the 1 GPU test on PyTorch 0.2, but the error is still at 0.896. (Do you mean something different in terms of final accuracy?) I did confirm that it converges on 0.1.12. So it sounds like the behavior change dat some point between 0.1 and 0.2.