nschleicher:

which is weird as the loss should be starting at 3.19.


I guess you’ve calculated this expected initial loss by assuming random predictions?
If so, I don’t think you can assume that a pretrained model will output random classes (a randomly initialized one should).