Update:
Don’t know why but I kind of solve the problem…?
Instead of sending two lists into the dataset, I send two numpy arrays and alter the dataset class as the following:
class MyDataset(Dataset):
    def __init__(self, X, y=None):
        self.data = torch.from_numpy(X).float()
        if y is not None:
            y = y.astype(np.int)
            self.label = torch.LongTensor(y)
        else:
            self.label = None
    def __getitem__(self, idx):
        if self.label is not None:
            return self.data[idx], self.label[idx] # -> a batch 
        else:
            return self.data[idx]

    def __len__(self):
        return len(self.data)

And then I define my own simple collate function that works inside the dataloader (its work is outputting batches) like this:
def my_collate(batch):
    """Define collate_fn myself because the default_collate_fn throws errors like crazy"""
    # item: a tuple of (img, label)
    data = [item[0] for item in batch]
    target = [item[1] for item in batch]
    data = torch.stack(data)
    target = torch.LongTensor(target)
    return [data, target]

then use mine instead of the default_collate_fn:
train_loader = DataLoader(concat_dataset, batch_size=batch_size, collate_fn = my_collate, shuffle= True, num_workers=2, pin_memory=True)

Then it works. At least for the following training I don’t get errors anymore. Still dunno what causes that original error but hope ppl with the same problem find this useful.