Depending on the use case, i.e. which loss function you are using, number of classes etc., you could calculate the “expected” loss for random outputs.
If the initially high loss value does not match the use case, e.g. your initialization might be bad for the use case and thus your model would perform worse than predicting random outputs (e.g. it could be very confident in wrong labels).