Solved by Abhilash_Srivastava in post #5 
              
              
                The implementation you posted above is different from the implementation in the Pytorch tutorial. I think your understanding is correct in both cases.