Right, that is my understanding. But I don’t know what to do to ensure something is accessible in both clones.  I am getting the error in a member function
    def memberFunction(self):
            self.otherFunction(self.currentGrad)

From Multi GPU backwards hook on wrong device 2 the register_buffer fixed this error for all the places where I do know the tensor size ahead of time.  But the question is essentially, how do i register_buffer so it works on multiple GPUs if the size is dependent on the input.