I stumbled upon a rather weird issue while debugging a larger code in PyTorch. A simple instance to reproduce the issue is here: https://colab.research.google.com/drive/11kBaxMOxN9i0X1vtaX47Yz7rkkHz7Kpu 2
As you can see, the variable x gets updated in every epoch (see that their norms are different), however, the norm of their difference (rel_diff) turns out to be zero. Any help is deeply appreciated.