❓ Why the criterion of XLNet LMHeadModel use ignore_index = -1 ?
In the XLNetLMHeadModel, the criterion used to compute the loss uses `ignore_index=-1` :

https://github.com/huggingface/pytorch-transformers/blob/9f995b99d4c4067662c3bd4f1274315c0839deeb/pytorch_transformers/modeling_xlnet.py#L927-L931

**Why ?**
Isn't it supposed to ignore the padding index ID, i.e. 5 ?
you should set padding and all the labels to be ignored to -1. In Bert/XLNet training, we usually only use 15% of the tokens as labels.