Same loss values but different eval result
I am experimenting with low-precision on the pre-trained BERT for SQuAD scenario. 
I am seeing a strange issue: the loss value when fine-tuning the model with FP16 is very similar to the loss value when fine-tuning the model at Int8. However, the eval results are are quite different -- with Int8, the results are quite bad (f1 = 3) compared to f1=88 with FP16.

Any idea what is going on and suggestions for debugging?
I have never tried Int8 in PyTorch.
Can you share some code so we can have a look?<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
