Solved by yvanscher in post #4 
              
              
                Yea my bad. So it looks like mul_ and add_ are ‘inplace’ operations which means that the underlying data will get modified. That’s probably why this is happening. Use: 
http://pytorch.org/docs/master/torch.html#torch.add 
http://pytorch.org/docs/master/torch.html#torch.mul