Solved by ptrblck in post #2 
              
              
                Your approach should be valid code-wise. 
The optimizers would only apply their step() method to all parameters, which have a valid gradient, so there shouldn’t be a problem.