Hi,
Are you using a dataloader with multiple workers? If so, you might want to have your dataloader work with cpu tensors (and pinned memory) and send the tensors to the gpu in your training loop. Or not use multiple workers to avoid any issue with multiprocessing,