@ngimel : Thanks, I get it now.
Is there any way around this situation? I am training a model on multiple GPUs and need to access some model attributes assigned during forward pass for debugging purposes.