For what concerns the loss value I believe the printed value (0.5) is correct.
In fact if you calculate the loss (output[i]-target[i])^2 element by element you obtain a tensor with values 1, 1, 0, 0 (read row-wise).
So when you sum up you obtain 2 and you have to divide by the total number of elements (4) because you choose implicitly reduction='mean' when you create the MSELoss class. So in the end you obtain 0.5.
I do not understand very well the meaning of the last line of your code:

grad = loss.grad_fn(torch.tensor(1).float())