Solved by ptrblck in post #2 
              
              
                All parameters without a .grad attribute will be skipped as shown here. You could avoid this step by filtering out all parameters which don’t require gradients.