Its a toy example to compare an implementation.
Everything is float 64 is in mine but I was curious if pytorch is doing things in float32.
Or perhaps it calls everything underneath in C and then maps it back to python. That communication layer between C and python might do some sort of compression. This is for the sake of increasing speeding of course
So in actual training performance is not affected (or maybe float64 is overkill)
The matrix below is the stored matrix in pytorch through the copy command. So after the copy it makes this minor modification.