[Fix #3963] GPT2 FP16
fix #3963 GPT2 failing (through run_language_modeling.py) in fp16 mode.
Gunna merge this at 7pm EST barring objections @thomwolf @julien-c <|||||>Pinging @thomwolf especially as he'd want to review this imo