Well, to avoid unnecessary pain, you would split the data so that each sample tensor will have its own .pt file. Then, in the __getitem__, you call the load, like this
class MyDataset(torch.util.data.Dataset):
   def __init__(self, root):
        self.root = root
        self.files = os.listdir(root) # take all files in the root directory
   def __len__(self):
        return len(self.files)
   def __getitem__(self, idx):
        sample, label = torch.load(os.path.join(self.root, self.files[idx])) # load the features of this sample
        return sample, label

And then use this dataset in the same way. Keeping the data in several chunks rather than individual samples would be … problematic.