Hi,
What happens is that if you set requires_grad=False, the gradient will not be computed and thus param.grad=None.
As you can see for example in the SGD optimizer here 182 the check is done every time.