I hava stored my input images in hdf5 files, each training, evaluation and testing in a separate group. Each group contains datasets ‘inputs’ and ‘labels’. I do this, because I cannot load my whole dataset in the memory. The dataset class looks the following:
class dataset_h5(torch.utils.data.Dataset):
    """
    Reads in a dataset
    """
    def __init__(self, in_file, mode = 'training'):
        super(dataset_h5, self).__init__()
        self.cuda = torch.device('cuda:0')

        self.file = h5py.File(in_file, 'r')[mode]
        self.n_images, self.channels,self.nx, self.ny = self.file['inputs'].shape
        self.n_images_check, self.nfeatures = self.file['labels'].shape

        if self.n_images != self.n_images_check:
            print('Number of input samples does not match number of label samples!')

        norm_set = h5py.File(in_file, 'r')['normalization']
        self.data_mean = norm_set['mean'][0]
        self.data_std = norm_set['std'][0]
        self.transform =transforms.Normalize((self.data_mean,), (self.data_std,))


    def __getitem__(self, index):
        input = self.transform(torch.tensor(self.file['inputs'][index,:,:].astype('float32')))
        labels = torch.tensor(self.file['labels'][index,:].astype('float32'))
        # Transform to tensor to move to GPU
        return input.to(self.cuda), labels.to(self.cuda)


    def __len__(self):
        return self.n_images

Then I load the data and create a dataloader, like this:
train = util.dataset_h5(datafile_name, mode = 'training')
train_loader = torch.utils.data.DataLoader(train, batch_size=batch_size, shuffle = False, num_workers = 0)

But the process is insanely slow. I don’t know why, but loading a batch from the dataset takes the same time as training the network. Does anybody know how I could improve my dataset function?
Thanks!