Could you explain the use case you are working on and when exactly the accuracy and loss change?
Are you seeing different losses for the same input batch while the model is in evaluation mode via model.eval()?