ValueError: expected sequence of length 133 at dim 1 (got 80)
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.8.0
- Platform:
- Python version:
- PyTorch version (GPU?): 1.5.0
- Tensorflow version (GPU?):
- Using GPU in script?: non
- Using distributed or parallel set-up in script?:

### Who can help
<!-- Your issue will be replied to more quickly if you can figure out the right person to tag with @
 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik


Library:


- tokenizers: @LysandreJik


Documentation: @sgugger

Model hub:

- for issues with a model report at https://discuss.huggingface.co/ and tag the model's creator.

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->
Models:

albert, bert, xlm: @LysandreJik
Library:

tokenizers: @LysandreJik


## Information

Model I am using (Bert, XLNet ...): FlauBert

The problem arises when using:
* [x]my own modified scripts: (give details below)

```
    input_ids = []
    attention_masks = []

    for sent in texte:
        encoded_sent = flaubert_tokenizer.encode_plus(sent, add_special_tokens=True, truncation=True, padding=True, return_attention_mask=True)

        # Add the outputs to the lists
        input_ids.append(encoded_sent.get('input_ids'))
        attention_masks.append(encoded_sent.get('attention_mask'))

        # Convert lists to tensors

    print("len", len(input_ids))

    input_ids = torch.tensor(input_ids)
    attention_mask = torch.tensor(attention_masks)

    hidden_state = flaubert(input_ids=input_ids, attention_mask=attention_mask)

    # Extract the last hidden state of the token `[CLS]` for classification task
    last_hidden_state_cls = outputs[0][:, 0, :]
    print(last_hidden_state_cls)
```

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)  extracting first hidden state/embeddings produce by the model and give it to a classic classifier (smv)


## To reproduce

Steps to reproduce the behavior:

1. install transformers, pandas, numpy and torch (1.5.0 or others )
2.  
3.

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

stacktrace error : 

```
  ---Filename in processed................ corpus_ix_originel_FMC_train
etiquette  : [2 1 0]
Embeddings bert model used.................... :  small_cased

Some weights of the model checkpoint at flaubert/flaubert_small_cased were not used when initializing FlaubertModel: ['pred_layer.proj.weight', 'pred_layer.proj.bias']
- This IS expected if you are initializing FlaubertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing FlaubertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
<class 'numpy.ndarray'>
len 34
Traceback (most recent call last):
  File "/16uw/test/expe_5/train/test.py", line 63, in <module>
    main()
  File "/16uw/test/expe_5/train/test.py", line 46, in main
    dic_acc, dic_report, dic_cm, s = cross_validation(data_train, data_label_train, models_list, name, language_model_dir)
  File "/16uw/test/expe_5/train/../traitements/processin_test.py", line 197, in cross_validation
    features, s = get_flaubert_layer(features, lge_model)
  File "16uw/test/expe_5/train/../traitements/processin_test.py", line 107, in get_flaubert_layer
    input_ids = torch.tensor(input_ids)
ValueError: expected sequence of length 133 at dim 1 (got 80)
 
    
```
## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

I expect to get the inputs_ids and attention_mask to  pass it to the model to get the cls_embedding

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.