Solved by tom in post #2 
              
              
                I think you are running in the case described in the PyTorch 0.4 migration guide. 
.detach() is safer in that it catches inplace modifications that will cause autograd to give wrong results when backpropagating through the graph that you have “detached from”. 
Consider the following more minimal exa…