Hi,
Setting requires grad to False means that no gradients will be computed for this Tensor. And so your_tensor.grad will contain it’s previous value (None or a Tensor) and will never be updated.
The thing is that when you have momentum (or l2 normalization), even having a gradient of 0 will make your weights change. So if you don’t want to optimize these weights, you want to exclude them from the optimizer to be sure (note that some optimizer will completely ignore tensors that have a grad field set to None and what you proposed will work. But this is not True for all optimizers and thus not safe).