My code works fine when using just 1 GPU using torch.cuda.set_device(0) but it takes a lot of time to train in single GPU.
I tried various ways to Parallelize it, but nothing seems to work.
Currently Iam trying :
gpu_ids = [0,1,2,3]
model = torch.nn.DataParallel(model,device_ids=gpu_ids)
model.cuda()

But when i try to access some model methods, like model.encoder.eval() it throws error saying 'DataParallel' object has no attribute 'encoder'
What am I missing ?