My guess is for simplicity; as you noticed, the output of the network itself is one-hot encoded, so it can simplify things if this is the format that is used for the labels as well. In practice, storing labels this way on disk also wouldn’t really matter as compression will handle the long strings of zeros in one-hot encoded labels efficiently.