Yea my bad. So it looks like mul_ and add_ are ‘inplace’ operations which means that the underlying data will get modified. That’s probably why this is happening. Use:
http://pytorch.org/docs/master/torch.html#torch.add
http://pytorch.org/docs/master/torch.html#torch.mul