Well, a common (erroneous) approach to compute the loss of the dataset is summing the losses for all minibatches and dividing this sum by the total number of minibatches. Unless the size of the dataset is a multiple the batch size, this will not be the same as summing the loss for each example in the dataset and dividing this sum by the total number of examples. Moreover, if minibatches are sampled randomly, computing the loss using the former approach may produce different results in two consecutive calls.
Toy example:
Consider a dataset with N = 3 examples A, B, C with losses LA = 0.5, LB = 0.1, LC = 1.1 and a batch size of M=2. Note that N is not a multiple of M.
Scenario 1: The first minibatch is (A, B) and the second is C. Thus, the loss of the first minibatch is (0.5+0.1)/2 = 0.3 and the loss of the second is 1.1. Therefore, the average loss per minibatch is (0.3+1.1)/2=0.7
Scenario 2: The first mini batch is (B, C) and the second is A. Using analogous computations, you get an average loss per minibatch of 0.55.
Regarding your problem, make sure you call model.eval() before computing the loss if you are using any Dropout or BatchNorm layers, like @ptrblck said.