Hi,
I’m trying to concatenate more than 1 dataset and after the concatenation, looks like the index of the dataset would be not in order.
I’m having 6 images and labels related datasets, and want to pass them in a similar sequence.
This is the length of each dataset:
666
80
59
39
22
72
and this is how i concatenate them:
train_all = torch.utils.data.ConcatDataset({train_good,train_blow_hole,train_break,train_crack,train_fray,train_uneven})
train_all_label = torch.utils.data.ConcatDataset({train_label_good,train_label_blow_hole,train_label_break,train_label_crack,train_label_fray,train_label_uneven})
After the concatenation:
print(train_all.cumulative_sizes)
[72, 738, 797, 877, 899, 938]
print(train_all_label.cumulative_sizes)
[72, 111, 777, 836, 858, 938]
The index is not the same anymore and this would causing issue in the training stage later. Is there an efficient way to concatenate the dataset while preserving their index in the concatenated dataset?
Thanks