I am fully aware of the ability of accessing my network through nn.DataParallel.module. However, I need to make sure that my code is executable (no AttributeError when I am trying to access my defined-variable in my network) whether the user is using single GPU or multiple GPUs. Hence, the DataPrallelWrapper.
The only problem is I do not know whether this is a good approach or not (will my approach invite any data race or async problem?).