jitesh:

Will setting model.train() or model.eval() (in terms of Batchnorm and Dropout Layers) make a difference for those layers whose parameters have requires_grad set to False ?


Yes, eval()/train() do not change anything about the gradients and if parameters require gradients, but change the behavior (code paths) of some layers.



 jitesh:

And could you also mention if there are any other layers apart from BatchNorm and Dropout whose behaviour changes upon using .train() and .eval() ?


If I’m not mistaken, batchnorm and dropout layers are the only layers at the moment in the PyTorch core, which change the behavior.
However, custom layers (e.g. if you are using a specific repository from a 3rd party) can easily use the self.training flag to change the layer.