Solved by dpernes in post #2 
              
              
                Nvm, this does the job: 
x = torch.tensor([[1.], [1.]], requires_grad=True)  # shape (2, 1)
y = x**2  # shape (2, 1)
y_x = grad(y, x, grad_outputs=torch.ones_like(y), create_graph=True)