CUDA error: device-side assert triggered(pretrained_model.cuda())
## 🐛 Bug

<!-- Important information -->

Model I am using (XLNet....):

Language I am using the model on (English):

The problem arise when using:
 config = XLNetConfig.from_json_file('/data2/liping/xlnet/xlnet_cased_L-12_H-768_A-12/xlnet_config.json')
   encoder_model = XLNetModel.from_pretrained("/data2/liping/xlnet/xlnet_cased_L-12_H-768_A-12/xlnet_model.ckpt.index", config=config, from_tf=True)
   encoder_model.cuda("cuda:0")

The problem:
~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in _apply(self, fn)
    228                 # `with torch.no_grad():`
    229                 with torch.no_grad():
--> 230                     param_applied = fn(param)
    231                 should_use_set_data = compute_should_use_set_data(param, param_applied)
    232                 if should_use_set_data:

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in <lambda>(t)
    309             Module: self
    310         """
--> 311         return self._apply(lambda t: t.cuda(device))
    312 
    313     def cpu(self):

RuntimeError: CUDA error: device-side assert triggered
Hello! Is your checkpoint the original one from the XLNet repository or one of our TensorFlow checkpoints hosted on S3?<|||||>> Hello! Is your checkpoint the original one from the XLNet repository or one of our TensorFlow checkpoints hosted on S3?

The checkpoint is from the XLNet repository.<|||||>Could you then convert it to one a checkpoint readable by our models by using the script [convert_xlnet_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_xlnet_original_tf_checkpoint_to_pytorch.py)?<|||||>> Could you then convert it to one a checkpoint readable by our models by using the script [convert_xlnet_original_tf_checkpoint_to_pytorch.py](https://github.com/huggingface/transformers/blob/master/transformers/convert_xlnet_original_tf_checkpoint_to_pytorch.py)?

I have tried with the script, but the problem is still exiting.
encoder_model = XLNetModel.from_pretrained("/data2/liping/xlnet/produce/")
    encoder_model.cuda()

-> 230                     param_applied = fn(param)
    231                 should_use_set_data = compute_should_use_set_data(param, param_applied)
    232                 if should_use_set_data:

~/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py in <lambda>(t)
    309             Module: self
    310         """
--> 311         return self._apply(lambda t: t.cuda(device))
    312 
    313     def cpu(self):

RuntimeError: CUDA error: device-side assert triggered<|||||>What happens once you have converted the original checkpoint to PyTorch? What is inside the folder "/data2/liping/xlnet/produce/" ?<|||||>> What happens once you have converted the original checkpoint to PyTorch? What is inside the folder "/data2/liping/xlnet/produce/" ?

Thank you for your help and i have convert the original checkpoint to PyTorch and load the xlnet pre-training model successful.<|||||>@lipingbj  Good to hear that you've fixed the problem. i just met the same problem when i use run_lm_finetuning.py and when i try convert with convert_bertabs_original_pytorch_checkpoint.py it just returned no module named'model_bertabs'. By the way i put the pytorch_model.bin that i have trained before in convert file ,is that right and how you fix the problem.Will be really appreciate for  your reply!