I thought that too but I’ve noticed that when I pin it, some stuff get loaded on the GPU according to nvidia-smi.
I’ve tried changing the device before pinning it as in:
import torch
x_tensor = torch.FloatTensor(1)
with torch.cuda.device(2):
x_tensor.pin_memory()
and I got this:

image.png1266×818 73.8 KB

I don’t know the inner working of pinning memory but this led me to believe some preparations take place on the GPU as well (setting the device before starting a DataLoader doesn’t seem to affect it).
If I set the .cuda(device) on model/inputs they get sent to the correct GPU but that memory usage from pinning persists until I finish so I decided to ask on here for some opinions