Hello,
Was anyone able to figure out an answer to this issue? I’m experiencing the same problem (for a very unrelated reason). I am trying to save a variable during the forward pass of the network and access it later on. This works fine on 1 GPU, but causes issues on multi-gpu.