Solved by ptrblck in post #2 
              
              
                Usually you don’t need to set this attribute manually. 
The parameters inside the layers will be created as nn.Parameter, which will automatically have requires_grad=True. 
A use case for setting this attribute manually, would be e.g. if you use some extra parameters in your model (for your fancy op…