Thanks for your reply.

As transformerdecoder layer has dropout within it I trained my model with model.train(). But result was same.
Your suspicion is correct. It is giving different results on the same training data. To test that only I am running inference with train_dataloader.
Don’t understand why model loading is failing. I am calling the training and inference code in the same jupyter notebook. I also tried (after your comment) saving the model after training (in the same jupyter cell in which training is happening) and then loading it later for inference. But even that isn’t working.

torch.save(model.state_dict(), './dl_class.pth')
model = ratingModel(nhead=8, dim_model=384, dim_ff= 2048)
model.load_state_dict(torch.load('./dl_class.pth'))