[RuntimeError: sizes must be non-negative]  : XLnet, Large and Base
## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): XLNet

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [ ] the official example scripts: (give details)
* [x] my own modified scripts: (give details)




The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details)

## To Reproduce

Steps to reproduce the behavior:

1.
2.
3.

<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->

## Expected behavior

<!-- A clear and concise description of what you expected to happen. -->

## Environment

* OS: Ubuntu
* Python version: 3.5.2
* PyTorch version: 1.20, torch 0.4.1
* PyTorch Transformers version (or branch): latest
* Using GPU ? YES
* Distributed of parallel setup ? No
* Any other relevant information:

## Additional context
The issue reads almost exactly like this - https://github.com/huggingface/pytorch-transformers/issues/924
Except - the problem still appears to persist as I have pytorch 1.2 installed./
To be sure I have to use CODA 9.0 - not sure if that is causing the issue. Note - I can run - XLM, Bert and Roberta in the exact same data and code -jut swapping out the modelname .
<!-- Add any other context about the problem here. -->
Can you post a simple example showing the behavior and a detailed error message?<|||||>-----------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-1-e5cfbf5c4eca> in <module>
    127                     train_df.to_csv('data/train.tsv', sep='\t', index=False, header=False)
    128                     dev_df.to_csv('data/dev.tsv', sep='\t', index=False, header=False)
--> 129                     results= run_model(args,device)
    130                     cv_results.append(results[0])
    131                 r = pa.DataFrame(cv_results)

~/xlm/run_model.py in run_model(args, device)
     65         train_dataset = load_and_cache_examples(task, tokenizer,args,processor,logger,False, undersample_scale_factor=1)
     66         #stop
---> 67         global_step, tr_loss = train(train_dataset, model, tokenizer,args,logger,device)
     68         logger.info(" global_step = %s, average loss = %s", global_step, tr_loss)
     69 

~/xlm/train.py in train(train_dataset, model, tokenizer, args, logger, device)
     64                       'token_type_ids': batch[2] if args['model_type'] in ['bert', 'xlnet'] else None,  # XLM don't use segment_ids
     65                       'labels':         batch[3]}
---> 66             outputs = model(**inputs)
     67             loss = outputs[0]  # model outputs are always tuple in pytorch-transformers (see doc)
     68             print("\r%f" % loss, end='')

~/.local/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    475             result = self._slow_forward(*input, **kwargs)
    476         else:
--> 477             result = self.forward(*input, **kwargs)
    478         for hook in self._forward_hooks.values():
    479             hook_result = hook(self, input, result)

~/.local/lib/python3.5/site-packages/pytorch_transformers/modeling_xlnet.py in forward(self, input_ids, token_type_ids, input_mask, attention_mask, mems, perm_mask, target_mapping, labels, head_mask)
   1120                                                input_mask=input_mask, attention_mask=attention_mask,
   1121                                                mems=mems, perm_mask=perm_mask, target_mapping=target_mapping,
-> 1122                                                head_mask=head_mask)
   1123         output = transformer_outputs[0]
   1124 

~/.local/lib/python3.5/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    475             result = self._slow_forward(*input, **kwargs)
    476         else:
--> 477             result = self.forward(*input, **kwargs)
    478         for hook in self._forward_hooks.values():
    479             hook_result = hook(self, input, result)

~/.local/lib/python3.5/site-packages/pytorch_transformers/modeling_xlnet.py in forward(self, input_ids, token_type_ids, input_mask, attention_mask, mems, perm_mask, target_mapping, head_mask)
    883         if data_mask is not None:
    884             # all mems can be attended to
--> 885             mems_mask = torch.zeros([data_mask.shape[0], mlen, bsz]).to(data_mask)
    886             data_mask = torch.cat([mems_mask, data_mask], dim=1)
    887             if attn_mask is None:

RuntimeError: sizes must be non-negative<|||||>Works perfectly with xlm, bert and roberta<|||||>Ok, this should be fixed on master with 45de034.

You can test it either by installing from source or using `torch.hub` and tell us if it doesn't work.