Thank you for your response.
How if the following two settings? are they equivalent?
Setting 1 :

bn_layer.eval()
bn_layer.weight.requires_grad = False
bn_layer.biast.requires_grad = False

Setting 2 :
bn_layer.train()
bn_layer.momentum = 0
bn_layer.weight.requires_grad = False
bn_layer.biast.requires_grad = False

My goal is to explore the momentum such that the bn_layer only peforms very minimal learning.
However, following Setting 2, the performance drops drastiscally even setting momentum = 0.00001.
Thank you.