If the optimizer is smart enough so that it looks at the requires_grad, why do people pass lambda functions into the optimizer, why not just pass in model.parameters()?
As for my training time, because I was setting all the BatchNorm to be unfrozen, and those are scattered literally all throughout the model, backdrop still had to be done, even though it wasn’t updating the parameters, since everything with backpropogation is chained/dependent.  I tried to just freeze the classifier and saw a difference in speed, but since the actual updates to the weights is super quick, its the actual calculations that take all the time, that is why I wasn’t seeing a speed increase.
So I guess I am still unclear though.  Is there any difference or benefit to passing in model.parameters() vs passing in filter(lambda p: p.requires_grad, model.parameters()) when you have parts of your model frozen?