Solved by ayalaa2 in post #4 
              
              
                Assuming that you set all the requires_grad flags before training at all, there is very little difference in using filter. 
In the optimizer code, it iterates over each parameter and if the gradient is None, it continues. If requires_grad=False, the parameter’s gradient will never be updated thus it…