Solved by ptrblck in post #2 
              
              
                a) Not quite. You are creating a single optimizer and are overwriting the object, so only the last model would be updated. Besides that you are currently storing the loss tensor in the losses list which will not only store the actual loss value but the entire computation graph and would thus increas…