Consider a model like the one in https://github.com/pytorch/examples/blob/master/mnist/main.py 75 (I take this example just to help to formulate the problem).
The model have a dropout so If I calculate the loss at the training it will not reflect the prediction model loss status for training since that will be without dropout.
From the other side calculating the more faithfull training loss will be computationally very expensive.
I wonder how people deal with that.