Thank you so much for your help. I solved the problem. The problem was in saving the model. My model works fine now with using
torch.save(model.state_dict(), PATH)

but when I use a general checkpoint for inference using
torch.save({
            'epoch': epoch,
            'model_state_dict': model.state_dict(),
            'optimizer_state_dict': optimizer.state_dict(),
            'loss': loss,
            ...
            }, PATH)

the error of size mismatching and missing keys occur.