I was training a very simple model 3 convs 1 fully connected, to many epochs. If I do that using the pytorch cross entropy the accuracy gets to around 1 and then suddenly jumps down to 0 (not chance, 0). If I train the same model in pytorch but with the multiclass hinge loss I don’t get that issue (I’ve done this experiment 4 times on both losses).
Is this hinting there my be a bug in the CrossEntropy module in pytorch?
Note the only lines I changed was:
#criterion = torch.nn.CrossEntropyLoss()
criterion = torch.nn.MultiMarginLoss()