Hello,
I am training a 3D CNN and I set batch_size = 1 in the dataloader object since a batch_size =2 gives cuda out of memory. So I thought about another way to increase the batch_size to 2. The crux of the code looks something like (in the trainer loop of the program) :
for batch_idx, (subject) in enumerate(train_loader):
        # Load the subject and its ground truth
        itr+=1
        image = subject['image']
        mask = subject['gt']
        # Loading images into the GPU and ignoring the affine
        image, mask = image.float().cuda(), mask.float().cuda()
        #Variable class is deprecated - parameteters to be given are the tensor, whether it requires grad and the function that created it
        optimizer.zero_grad()
        image, mask = Variable(image, requires_grad = True), Variable(mask, requires_grad = True)
        # Forward Propagation to get the output from the models
        output = model(image.float())
        # Computing the loss
        if itr%2==1 :
            loss_odd = loss_fn(output.double(), mask.double(), n_classes)
        if itr%2==0 :
            loss_even = loss_fn(output.double(), mask.double(), n_classes)
            loss = loss_odd + loss_even
            loss.backward()
            optimizer.step()

In the above dataloader I have set the batch_size =1 and I am trying to manually make the network “feel” as if the batch_size = 2 by accumulating the loss function and doing back prop after every 2 examples…
Am I doing this right?
Or am I missing something?
Thanks a lot in advance