As far as I understand, what Simone said is correct: the computational graph only considers operations between Variables and you are creating a new graph every iteration, since when you instantiate new Variables at the beginning of the training iteration, so it should be fine. Also, if you operate directly on the weight’s tensors (weight.data), it should not be included in the graph.
Taking your last example, I think you should change it to use W.data property if you want to be sure. Also, I don’t think mdl.zero_grads() is doing anything where it is, because you haven’t called backward() yet to calculate the gradients anywhere. However, you should make sure train_for_T_iterations function does call zero_grads() before calling backward(), but I believe you are already doing it, otherwise you would be always considering the past iterations’ gradients.