You can simply avoid such error by only calling backward() if your output requires_grad. That will have the behavior you expect.:
loss =  torch.sum(x * x)
if x.requires_grad:
   x.backward()

In  general, we choose to raise an error in this case because in most user cases, users expect backward to actually happen when they call.backward(). So having a Tensor that doesn’t require gradients is a bug in the user code in most cases.