Not able to use FP16 in pytorch-pretrained-BERT. Getting error **Runtime error: Expected scalar type object Half but got scalar type Float for argument #2 target**
I'm not able to work with FP16 for pytorch BERT code. Particularly for BertForSequenceClassification, which I tried and got the issue
**Runtime error: Expected scalar type object Half but got scalar type Float for argument #2 target**
when I enabled fp16.
Also when using 
`logits = logits.half()
labels = labels.half()`
then the epoch time also increased.

The training time without fp16 was 2.5 hrs per epoch after doing logits.half() and labels.half() the runtime per epoch shot up to 8hrs.
Which kind of GPU are you using? `fp16` only works on recent GPU (better with Tesla and Volta series).<|||||>I experienced a similar issue with CUDA 9.1. Using 9.2 solved this for me. <|||||>Yes, CUDA 10 is recommended for using fp16 with good performances.