Solved by ptrblck in post #4 
              
              
                They will yield the equivalent results, yes. 
Filtering out the parameters is explicit and could thus increase the code readability and will also avoid iterating over parameters without a grad attribute in the step method. 
Approach (1) would allow you to unfreeze the parameters later in training di…