I’m unsure which “effect of activating/deactivating the gradient calculation” you are referring to, so could you explain this effect a bit more? You don’t have to use the output of the last layer of a model to compute the gradients, since e.g. Inception models use an additional auxiliary loss.


For the same input, the output of the model after calling model.eval() is expected to be the same (up to numerical precision) as long as no parameter updates were performed. Is this what you are seeing? If not, could you also explain the issue in more detail?