If this is not inside the forward method, nn.DataParallel won’t be able to replicate it on each device.
Since you are running it outside, you could try to not use the default device, as it’s memory usage will usually be bigger than from the other devices.