You might want to check if there is a data loading issue or some other problem in the training pipeline. A properly initialized model with 79,077 classes should have an initial cross entropy loss of -ln(1/79077) or about 11.3. You can try passing in random data into your model to see if this is the loss you get. If you do see the correct loss, then it likely means the problem is with data loading somewhere.