New tokenizer code in transformer 3.0.0 is creating error with old code
# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BeRT and GPT2 for Poly-encoder implementation

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)
Ubuntu V2.0 corpus dataset, implementing Poly-encoder pipeline, everything was done, was re-training the model again to verify the results of the first training process.

## To reproduce

Steps to reproduce the behavior:
Happens when using some_tokenizer_fromHF.encode_plus(), below is the eval script to test on custom input text, is exactly same as the one reading from dataset during training(simplified for eval)
```
from transformers import BertTokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
c_text="what is your name. I am Gloid"
context=tokenizer.encode_plus(c_text,
                                      text_pair=None,
                                      add_special_tokens=True,
                                      max_length=max_len,
                                      pad_to_max_length=False)
texts=["Obama", "Trump", "Eminem", "slender man", "Pewdiepie"]
for text in texts:
      tokenized_dict = tokenizer.encode_plus(text,
                                                  text_pair=None,
                                                  add_special_tokens=True,
                                                  max_length=max_len,
                                                  pad_to_max_length=True)
```
The error is - 
"Truncation was not explicitely activated but `max_length` is provided a specific value, please use `truncation=True` to explicitely truncate examples to max length. Defaulting to 'only_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you may want to check this is the right behavior."
Repeated a total of 6 times, i.e.,  for every sequence passed into encode_plus
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

Not to give this error, and return the input ids, segment ids, and input masks
The issue is completely identical to the closed issue - https://github.com/huggingface/transformers/issues/5155

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 3.0.0
- Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.5.1+cu101 (True)
- Tensorflow version (GPU?): 2.2.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

Hi, this is not an error but a warning. If you want to disable warnings, you can use the following:

```py
import logging

logging.basicConfig(level=logging.ERROR)
```<|||||>Oh, I'm sorry for writing "error" everywhere, but I want to know, is this default behaviour correct for BeRT, it says by default it'll use only_first<|||||>You can read the documentation concerning that method [here](https://huggingface.co/transformers/main_classes/tokenizer.html#transformers.PreTrainedTokenizer.__call__).

Here's the part you're probably interested in: 

> ‘only_first’: truncate to a max length specified in max_length or to the max acceptable input length for the model if no length is provided (max_length=None). This will only truncate the first sequence of a pair if a pair of sequences (or a batch of pairs) is provided,

Since you're only using a single sentence, it seems to be what you're looking for?<|||||>I am also concatenating multiple "context" sequences using [SEP] token, I'm just feeding sequences into encode_plus and stripping off the [CLS] token at the beginning and concatenating the rest with the old one, making it
[CLS]seq1[SEP]seq2[SEP]
I assume, even earlier, in older version, when this warning wasnt being logged to the terminal, it still used only_first, did it?<|||||>Is there a reason why you're not using the `encode_plus` method with your pairs? The tokenizer will automatically build the pairs as the model expects them.

If you pass a single sequence and want to build them with the special tokens yourself, you can use the `add_special_tokens=False` flag. No need to strip the special tokens then.

Be careful as since you're building the sequences yourself, if you truncate these some of the special tokens might be cut off.<|||||>This is my snippet, "texts" is a list of strings
```
def __call__(self, texts):
    input_ids_list, segment_ids_list, input_masks_list = [], [], []

    for text in texts[::-1][:self.max_history]:
      tokenized_dict = self.tokenizer.encode_plus(text,
                                                  text_pair=None,
                                                  add_special_tokens=True,
                                                  max_length=self.max_len,
                                                  pad_to_max_length=False)
      input_ids, input_masks = tokenized_dict['input_ids'], tokenized_dict['attention_mask']
      segment_ids = [1] * len(input_ids)
      if len(input_ids_list) > 0:
        input_ids = input_ids[1:]
        segment_ids = segment_ids[1:]
        input_masks = input_masks[1:]
      input_ids_list.extend(input_ids)
      segment_ids_list.extend(segment_ids)
      input_masks_list.extend(input_masks)

      if len(input_ids_list) >= self.max_len:
        input_ids_list = input_ids_list[:self.max_len - 1] + [self.sep_id]
        segment_ids_list = segment_ids_list[:self.max_len]
        input_masks_list = input_masks_list[:self.max_len]
        break
    input_ids_list += [self.pad_id] * (self.max_len - len(input_ids_list))
    segment_ids_list += [0] * (self.max_len - len(segment_ids_list))
    input_masks_list += [0] * (self.max_len - len(input_masks_list))

    assert len(input_ids_list) == self.max_len
    assert len(segment_ids_list) == self.max_len
    assert len(input_masks_list) == self.max_len

    return input_ids_list, segment_ids_list, input_masks_list
```
I'm not truncating anything after I create my full sequence. That suggestion was great for using text_pair, I dont know why I didnt think of that. Thank you
PS- Is this way of creating input ids correct, and in the older version, when this warning wasn't being logged to the terminal, was it using only_first even then?<|||||>In that snippet, are you trying to concatenate a lot of sequences together? If you have 10 sequences in your text, you want to have a giant `input_ids_list` containing all the 10 sequences separated by a separator token?<|||||>Yes, exactly, thats what I am doing, and I am then strippin off earlier later part, coz I am flippin the list too. Basically the list is a conversation where I am making a new token list out of recent N words of conversation<|||||>For anyone stumbling across this issue and having problems with sentence pair classification in v3.0.0:

In v3.0.0, the default truncation strategy was changed, which causes code that used to work in v2.11.0 to break in some cases.
**v2.11.0**: default truncation strategy is `longest_first`  
**v.3.0.0**:  truncation strategy appears to default to `only_first`

For sentence pair classification in v3.0.0, this can result in a failure to truncate sentence pair to the supplied `max_length` parameter, which can break downstream model or other code:
```
W0702 12:56:50.435204 140139424331584 tokenization_utils_base.py:1447] Truncation was not explicitely activated but 
`max_length` is provided a specific value, please use `truncation=True` to explicitely truncate examples to max length.
 Defaulting to 'only_first' truncation strategy. If you encode pairs of sequences (GLUE-style) with the tokenizer you may 
want to check this is the right behavior.
E0702 12:56:50.437675 140139424331584 tokenization_utils.py:784] We need to remove 25 to truncate the input but the first
 sequence has a length 17. Please select another truncation strategy than TruncationStrategy.ONLY_FIRST, for instance 
'longest_first' or 'only_second'.
```

For example, the following code prints  **32** in v2.11.0, but **57** in v3.0.0:

```python
text_a = '''Debunk this: Six Corporations Control $NUMBER$% Of The Media In America'''
text_b = '''
I can't believe people are missing the two obvious flaws in this analysis. 
This infographic doesn't show that $NUMBER$ companies control $NUMBER$% of the media. '''
from transformers import *
t = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
output = t.encode_plus(text_a, text_b, max_length=32)
print(len(output['input_ids']))
```

The solution is to explicitly provide `truncate='longest_first`:, as indicated in the warning:  
```python
output = t.encode_plus(text_a, text_b, max_length=32, truncation='longest_first')
```<|||||>Good point. We will release a patch to fix this breaking change (move back to having `longest_first` as default) plus the one mentioned in #5447 probably tomorrow or early next week. <|||||>@thomwolf:  Thanks - changing the default back to `longest_first` may also address #5460 