@ptrblck, Thanks. If I understand it correctly, calling .backward() with in the loop and step() outside of the loop will make the gradients to be computed at ever sampling time, and the trainable variables to be updated in the end of the sampling process. And this will have exactly the same effects (in terms of learning) to the network, but more memory efficient. Am I right?