Solved by learner47 in post #2 
              
              
                data.to(device) moves the data to cpu or GPU based on what device is. This is required for faster computations. 
In PyTorch, the gradients are accumulated using loss.backward() and then the gradients are applied using optimizer.step(). The stale gradients from the previous back propagation need to b…