Thanks for info, but I have gone through the tutorial at a high level. It is exactly how the optimizer knows that I am asking about. So, we have a loss function with gradients on the variables that decide (along with step size/learning rate) what the next values of variables should be. It seems odd that the construct - torch.optim instance - that takes the step knows only about the parameters ( it looks that way from the statement “optimizer = …” ) and not the loss function. How does it pick up gradient values correctly? In other words, what if I had another loss function on the same set of parameters (for e.g., loss_2 = nn.NLLLoss(outputs, labels) where outputs and labels are as above)? How would it know which gradient to use to take the next “step”?  One pictures a setup where the forward-backward construct (net), the loss function and the optimizer have to work in tandem and not seeing it.