Okay I think I found it.
I had to set
num_beams=1, do_sample=False
manually inside model.generate() to get the correct logit values. These values are set by default according to huggingface documentation, but still, I had to manually set them.
Please correct me if I am wrong on this.
Thanks.