I’m a beginner with pytorch.
I’m using pytorch to build a CNN for object detection.
I encounter something really weird when using DataLoader to feed data.
The size of my input images are [3, 640, 640].
Normally, with 2 GPUs of 12 GB, I can only feed about 8 images once a time. However, when I used DataLoader with batchsize = 8, I marked a very low usage of GPU memory.(About 20%) Then I increased batchsize = 40, I started to see a 90% usage.
To figure out what’s wrong. I run a sample example with resnet50 backbones.
import torch.utils.data
import torchvision


net = torchvision.models.resnet50(pretrained=True)
#net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))
net.cuda()


class RandomDataset(torch.utils.data.Dataset):
    def __init__(self):
        super(RandomDataset, self).__init__()
    
    def __getitem__(self, image_id):
        return torch.rand([3,640,640])
    
    def __len__(self):
        return 1000

ds = RandomDataset()

dl = torch.utils.data.DataLoader(ds, batch_size=15, shuffle=False, num_workers=8)

with torch.no_grad():
    for batch_idx, inputs in enumerate(dl):
        net.eval()
        inputs = inputs.cuda()
        out = net(inputs)
        
        
print((torch.cuda.max_memory_cached(0)+torch.cuda.max_memory_allocated(0))/1024/1024/1024)

So, 1 GPU can only feed 15 images normally. (If I don’t use dataloader, GPU reaches 100% usage) While, in DataLoader, it costs only 2.5Gb. It reaches 100% when I set batch_size = 100 which is totally not reasonable.
Can anyone help me with this? Do I make some mistakes somewhere? Or I misunderstand the meaning of batch_size?