Hi @albanD,
Thanks a lot for your quick response.
So just to be clear, if any one of the following:

The input to the model
All model parameters

Has requires_grad=True, then gradients will be tracked, correct? This is because all operations involve one of these two components right?
Also, does model.train() set requires_grad=True for all the parameters? If not, what model.train() actually do internally? Similarly, does model.eval() set requires_grad=False for all parameters? If not, what does it do internally?