Solved by ResidentMario in post #2 
              
              
                The PyTorch autograd docs state: 

If there’s a single input to an operation that requires gradient, its output will also require gradient. Conversely, only if all inputs don’t require gradient, the output also won’t require it. 

The input to a model like this one will be a vector with no_grad=Fals…