Thank you so much for the reply. It is good to know that model.eval() does not change BN statistics.
As for my problem, I think I had a fundamental issue in my training. Now I am not facing the previous issue.