To be clear:
the following code
with torch.no_grad():
        w1 -= learning_rate * w1.grad
        w2 -= learning_rate * w2.grad

        # Manually zero the gradients after updating weights
        w1.grad.zero_()
        w2.grad.zero_()

does run properly with
with torch.no_grad():

it is just the piece got error without the with condition.
w1 -= learning_rate * w1.grad
w2 -= learning_rate * w2.grad

it is just bit confusing when I am trying the autograd features but with this condition of require_grad=False maner.
Thanks Tom