I am getting to into the habit of creating custom torch.utils.data.Dataset classes to load and prepare my data – its text data so I am tokenizing it “on the fly”.
The problem I have, is that my data is imbalanced and I want to use WeightedRandomSampler which requires that I know the distribution of my target class. This is unknown, owing to the fact that I am using the random_split function to split my data:
train_size = int(0.8 * len(csv_dataset))
valid_size = int(0.1 * len(csv_dataset))
train_ds, valid_ds, test_ds = torch.utils.data.random_split(csv_dataset, [train_size, valid_size, valid_size+1])

A single loop through the data set takes a long time because it contains quite a bit of information (owing to the transformations done). Is there another way around this?