I have a small question regarding DataParallel model’s transferring of CPU tensors to the appropriate GPUs.
While manually copying tensors from CPU to GPU, if the CPU tensors are memory pinned (by calling pin_memory()), one can pass async=True in the .cuda() call to enable faster asynchronous memory copy as explained in the docs 117. But I am wondering how I can enforce DataParallel model to use asynchronous memory copy? Does DataParallel model know anything about the pinned memory nature of the input tensors?