Try this instead:
a = torch.tensor(0.5)
a.requires_grad = True
lr = 1e-3
for i in range(20):
    yy = torch.cos(a*x)
    loss = torch.mean((yy-y)**2)
    loss.backward()
    with torch.no_grad():
        a -= lr * a.grad
        a.grad.zero_()

The following also works (but may be less efficient?) :
    with torch.no_grad():
        a = a - lr * a.grad
        a.requires_grad = True

The problem you are running into is that when you do a = a - a.grad you are getting a new a which does not have requires_grad set. So either you need to do in-place operations on a (preserving the flag) or manually re-enable it. Also, the gradient descent step has to be done inside a no_grad() context.