This shouldn’t hurt the performance of the network, and in reality a one-hot formatting vs. a label wouldn’t be related to things like ordinality unless the loss function itself explicitly included ordinality in the loss computation (the typical cross-entropy loss doesn’t do this).
If you want to convert one-hot to class labels, a simple way to do this is using the max function.
>>> import torch
>>> onehot_labels = torch.zeros(64, 26, dtype=torch.long)
>>> for i in range(64):
...     onehot_labels[i,i%26] = 1
...
>>> _, class_labels = torch.max(onehot_labels, axis=1)
>>> print(class_labels)
tensor([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10, 11, 12, 13, 14, 15, 16, 17,
       18, 19, 20, 21, 22, 23, 24, 25,  0,  1,  2,  3,  4,  5,  6,  7,  8,  9,
       10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25,  0,  1,
        2,  3,  4,  5,  6,  7,  8,  9, 10, 11])

Note that having the architecture of the network producing a one-hot output while the loss function itself is taking a class label is not a problem, but rather an implementation detail of how the labels are being stored and passed to the loss function.