Yes, both of your suggestions might be valid in case you are continuing the training.
Assuming you are checking the training performance of the model, the order of the samples would matter, since each update depends on them.
Also, if you are not restoring the optimizer, the first update step might increase the loss, especially if internal states were used in the optimizer.
However, the validation set should yield the same result up the the limited floating point precision.
Make sure you are calling model.eval() before calculating the validation performance.