there is another subtle point, even if SGD wasn’t changing the model which screws up this estimate as being exactly the train set, if the batch sizes are not all the same size then its not the right estimate because 1/n assumes all the batches are of the same size.