at the end of you error message, I guess it’s the device problem.
you set device to a single gpu (device = ‘cuda:0’), how to  parallel?
asume you have server gpus, let’s say 8. you wanna use 4 of them.
os.environ['CUDA_VISIBLE_DEVICES'] = '0,1,2,3'

this will using first 4 gpus. Then
device = torch.device('cuda') 

set the decvice to the 4 gpus. After that, just parallel:
if torch.cuda.device_count() > 1:
    print("Let's use", torch.cuda.device_count(), "GPUs!")
    model = torch.nn.DataParallel(model)
model.to(device)

then training:
for step, (img, target) in enumerate(train_loader):
    img = img.to(device)
    loss = model(img)
    ...