Solved by Oli in post #2 
              
              
                I’m sorry Chloe, but there are a magnitude of weird things here. Your loss function should be separate from your model. It most often only consist of a function (not using nn.Parameter - thats the model) and returns a scalar. 
Dont do zero_grad before your backward and step. Do the zero grad before …