I was using 1 GPU and batch size was 64 and I got cuda out of memory. So I reduced the batch size to 16 to solve it. But when I am using 4 GPUs and batch size 64 with DataParallel then also I am getting the same error:
my code:
device = torch.device(‘cuda’ if torch.cuda.is_available() else ‘cpu’)
device_ids = os.environ[“CUDA_VISIBLE_DEVICES”].split(",")
device_ids = [int(i) for i in device_ids]
encoder = EncoderRNN(input_size, hidden_size, SRC).to(device)
decoder = DecoderRNN(hidden_size, output_size, TRG).to(device)
model = nn.DataParallel(Model(encoder, decoder), device_ids = device_ids).to(device)
With DataParallel we can use multiple GPU and hence increase batch size.