I was having a lot of confusion on this as well. Although the nn.modules layers give us the .train() and .eval() flexibilities, but if we initialize an activation or pooling operation as layer having its weight, then I would like to know that how giving parameters to these operations help in achieving the goal for which we have the model for?
This question is bumping in my mind a lot and I would like to end it once and for all so please help.
Thanks,