Brando_Miranda:

batch_xs, batch_ys = get_batch2(data.X_train,data.Y_train,M,dtype) # [M, D], [M, 1]


Am I right to assume you are getting your data wrapped in Variables when you call get_batch2?
As far as I understand, a new computational graph is constructed when you instantiate new Variables (leaf variables) and do operations on them. Everytime you reuse the variable names, you get new objects and, hence, a new computational graph will be constructed to record the operations you are executing.
Take a look at the gif about autograd in this page, I think it’s helpful: http://pytorch.org/about/ 18