thanks for your reply.
@1chimaruGin
So my network is almost like Unet.
and let’s say I train/evaluate my model in the following ways:



model.eval()
evaluate the model on test set
model.train()
train the model on train set



model.train()
evaluate model on test set
model.train()
train the model on train set
my batch size is 5.
So, the only difference is that in the second case, before evaluating the model on test set I don’t call model.eval().
In the second case I get a better performance. I have both dropout and batch norm in my network and I know that they behave differently if calling model.eval(). I think the problem is because of batchnorm and maybe running stats but I don’t know how to fix it.