You are welcome!
I do not know definition of your model, but I get you are using BatchNorm and maybe Dropout. This layers act differently during eval and train mode. For instance, if model.train is used, then mean/variance in BatchNorm will be updated even on validation set, for every batch, you will have different parameters for all BatchNorm which is not desirable as we looking to update mean/variance in BatchNorm only in training step not testing or validation. That is, after training, in validaiton step, each time a different BatchNorm is being used which has been updated based on previous batch in validation step. So, overall loss will not be same.

  
    
    
    What does model.eval() do for batchnorm layer? 
  
  
    Hi Everyone, 
When doing predictions using a model trained with batchnorm, we should set the model to evaluation model. I have a question that how does the evaluation model affect barchnorm operation? What does evaluation model really do for batchnorm operations? Does the model ignore batchnorm?
  



  
    
    
    'model.eval()' vs 'with torch.no_grad()' 
  
  
    When i test my model, do I have to use model.eval() even though I am using 'with torch.no_grad() ?
  


bests