I had some similar issue during the backward() call. I replaced the squeeze() and unsqueeze() with view() calls. It seems that the usage of squeeze() causes this inplace replacement behavior but I can’t actually capture the dynamics of it.