Could you add some fake annotations fir dataloader2 just to potentially exclude the lack of target annotations as a root cause?
If you are passing a single GPU id (or only one GPU is found), nn.DataParallel will just use the single GPU as seen in these lines of code 3.