This still doesn’t work. Even if I don’t pass a dictionnary of tensors, torch tries to use the default collate function (so torch.stack) and fails. I don’t understand because my code is basically exactly the RCNN example, and this example doesn’t implement a custom collate function.
EDIT: the PyTorch example actually does implement a custom collate function that look something like this:
def collate_fn(batch):
    return tuple(zip(*batch))  

doing dataloader = DataLoader(dataset, batch_size=bsize, shuffle=True, drop_last=True, collate_fn=collate_fn) then seems to solve my problem.