Hello @ptrblck thank you for your response! To answer your question: yes, the model is set to eval() in the beginning of training. See here for example: Mean-Shifted-Anomaly-Detection/main.py at a02ac3001f96fcfa2f9b6a5fcb58cb3e9ba7f5b0 · talreiss/Mean-Shifted-Anomaly-Detection · GitHub 1
What is perplexing is that when I change the eval just for the training (Mean-Shifted-Anomaly-Detection/main.py at a02ac3001f96fcfa2f9b6a5fcb58cb3e9ba7f5b0 · talreiss/Mean-Shifted-Anomaly-Detection · GitHub 1) I get worser results. They show that using their approach they get SOTA results on CIFAR-10, so I am trying to understand how this eval() thing can be applied to other architectures.
I do not think I understand your comment If that’s the case and if only dropout and batchnorm layers are used.. did you mean ..NOT used? in eval() mode my understanding was that batchnorm uses stats (m/v) which is stored and the dropout is disabled, meaning they are NOT used. is my understanding correct?
Thank you again.