If you want to train with CrossEntropyLoss, you can define a simple dataset like
class MyDataset(Dataset):
    def __init__(self, mapping):
        self.mapping = {filename : mapping[filename].index(1) for filename in mapping}
        self.filenames = self.mapping.keys()
        self.len = len(mapping)

    def __len__(self):
        return self.len

    def __getitem__(self, idx):
        return Image.open(self.filenames[idx]), self.mapping[self.filenames[idx]]

The ImageNet example: examples/main.py at master · pytorch/examples · GitHub 3 is a good reference for how to train a classification model.