I’m trying to save a value during a backwards hook function with self.X=value1.  This works perfectly with 1 GPU but on multiple GPU’s is does not.  I can even make it a constant with self.testval = 5.  With CUDA_VISIBLE_DEVICES=0 set before the call to backward() it says it has no attribute named testval, but right after it shows it is 5.  with CUDA_VISIBLE_DEVICES=0,1 it says no attribute both times.  I am completely stuck here.