Thanks for the reply. Yes, train_epoch() is training the model but since I set lr=0,  I thought that the validation results will be the same.  But, it seems like the batch norm is changing the parameters even though the learning rate is 0. Is there any way to prevent it?