Solved by ptrblck in post #13 
              
              
                Your initialization might be “unlucky”, as I’ve got some valid gradients in a couple of iterations (I also got a full zero gradient output): 
tensor([[ 0.0000,  0.0000,  0.0000],
        [ 6.8986, 10.3479, 10.3479],
        [ 0.0000,  0.0000,  0.0000],
        [13.0105, 19.5157, 19.5157],
        [ …