I’m not sure, if you are using MNIST as in the original question, but this code takes ~1 second on my laptop:
dataset = datasets.MNIST(root=PATH)

labels = []
for _, label in dataset:
    labels.append(label)