What do you mean by “but seems like it retrain the model and then do the prediction”?
After loading your model, you usually set it to evaluation mode using model.eval(), since some layers behave differently.
BatchNorm and Dropout are examples of the different behavior between training and evaluation.
Could you explain a bit more?