Hi every one, recently I encountered a strange issue and i don’t really understand it.
I have a CNN model with Dropout layers (with p=0.5). I intented to train and evaluate model at every epochs.
Case 1: I didn’t set model.train() for train phase, set model.eval() for evaluation phase -> model is in train mode for only the first epoch, and in eval mode for the rest (it’s just a mistake)
Case 2: set model.train() for train phase, model.eval() for test phase (this is what we tend to do)
But when I check, in case 1, the loss on train set reduces more quickly and far smaller than in case 2 (0.006 vs 0.028). Case 1 also gives better macro-F1 score (about 17%). So can I say “training model in eval mode gives better result” ?!
Can someone explain it?