Thank you for the example. One other difference is that you’ve used MNIST, whilst in my code I was training on the FashionMNIST dataset, presumably that’s why your validation accuracy in the end reaches ~98%, while in my Keras network it was only ~88% (+10% boost in your case, w00t!).
I have to spend a bit more time playing with your code, as not all Pytorch ecosystem is known to me, for instance I just now discovered torchvision.datasets.FashionMNIST or torch.utils.data.DataLoader 
Thanks