Using fast tokenizers with pipelines
# 🚀 Feature request
Currently tokenizers are not working with QA pipeline, because they do not have the tokenize method implemented. Speeding up the tokenization would be really beneficial for my application.
<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

Hi @AlecS12, 

I'm currently working on integrating tokenizers library inside transformers with pipelines support.

It should not be long now before it lang on master / get released.

You can track the development here: https://github.com/huggingface/transformers/pull/2674
If you want to checkout out the branch **tokenizers-v2** and give it a try, I'm more than happy to get your feedback.

Morgan<|||||>Hi @mfuntowicz,

That's great news. I checked out  tokenizers-v2 and tried it in a web server (flask) and jupyterlab. In both cases got the same error. Could you please look into this?

```python
from transformers import pipeline
nlp = pipeline('question-answering',  model='bert-large-uncased-whole-word-masking-finetuned-squad')
nlp({
    'question': 'Where is the cookie?',
    'context': 'I keep cookies in a red plastic container.'
})

...
I0212 15:15:53.546577 140155558979392 modeling_utils.py:456] loading weights file https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased-whole-word-masking-finetuned-squad-pytorch_model.bin from cache at /home/a652726/.cache/torch/transformers/ca2ac20761877486c1e2204d99653106b9adacf9a5eb18ec71b41d2dbef42103.2db7ae79c41a184c87600faabafa1369db2b16457723fd154ca3b436c4172807
convert squad examples to features:   0%|          | 0/1 [00:00<?, ?it/s]
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/data/home/a652726/transformers/src/transformers/data/processors/squad.py", line 141, in squad_convert_example_to_features
    truncation_strategy="only_second" if tokenizer.padding_side == "right" else "only_first",
  File "/data/home/a652726/transformers/src/transformers/tokenization_utils.py", line 1741, in encode_plus
    **kwargs,
  File "/data/home/a652726/transformers/src/transformers/tokenization_utils.py", line 1676, in batch_encode_plus
    tokens = self._tokenizer.encode(*batch_text_or_text_pairs[0])
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/site-packages/tokenizers/implementations/base_tokenizer.py", line 131, in encode
    return self._tokenizer.encode(sequence, pair)
TypeError
"""

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-1-c0bdf7f90854> in <module>
      3 nlp({
      4     'question': 'Where is the cookie?',
----> 5     'context': 'I keep cookies in the red plastic container.'
      6 })

...
```
<|||||>I will definitively have a look, and will keep you posted.

Thanks for reporting<|||||>Hi @mfuntowicz,

I installed the latest 2.5.1 release and the pipeline error is still there. Had to roll back to 2.4.1.<|||||>release 2.5.1 does not produce the error by default anymore, because it changed the default Autotokenizer to slow, but the bug is still there:

```python
import transformers
from transformers import pipeline
tokenizer = transformers.AutoTokenizer.from_pretrained("bert-base-uncased", use_fast=True)
nlp = pipeline('question-answering',  model='bert-large-uncased-whole-word-masking-finetuned-squad', tokenizer=tokenizer)
nlp({
    'question': 'Where is the cookie?',
    'context': 'I keep cookies in the red plastic container.'
})

nlp({
    'question': 'Where is the cookie?',
    'context': 'I keep cookies in the red plastic container.'
})


convert squad examples to features:   0%|          | 0/1 [00:00<?, ?it/s]W0226 10:50:27.573424 140277524375360 tokenization_utils.py:1782] Fast tokenizers add special tokens by default. To remove special tokens, please specify`add_special_tokens=False` during the initialisation rather than when calling `encode`,`encode_plus` or `batch_encode_plus`.
W0226 10:50:27.576760 140277524375360 tokenization_utils.py:1782] Fast tokenizers add special tokens by default. To remove special tokens, please specify`add_special_tokens=False` during the initialisation rather than when calling `encode`,`encode_plus` or `batch_encode_plus`.
---------------------------------------------------------------------------
RemoteTraceback                           Traceback (most recent call last)
RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/multiprocessing/pool.py", line 121, in worker
    result = (True, func(*args, **kwds))
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/data/home/a652726/transformers/src/transformers/data/processors/squad.py", line 141, in squad_convert_example_to_features
    truncation_strategy="only_second" if tokenizer.padding_side == "right" else "only_first",
  File "/data/home/a652726/transformers/src/transformers/tokenization_utils.py", line 1889, in encode_plus
    **kwargs,
  File "/data/home/a652726/transformers/src/transformers/tokenization_utils.py", line 1815, in batch_encode_plus
    tokens = self._tokenizer.encode(*batch_text_or_text_pairs[0])
  File "/home/a652726/miniconda3/envs/nlp2/lib/python3.7/site-packages/tokenizers/implementations/base_tokenizer.py", line 141, in encode
    return self._tokenizer.encode(sequence, pair)
TypeError
"""

'''
<|||||>Hi @AlecS12, 

Thanks for trying out 2.5.1. The issue is still there because for the question-answering pipeline we're relying on a method from the squad data processor `squad_convert_example_to_feature` which is not compatible which the fast tokenizers.

I'll have soon have a look at this to make it compatible with the fast tokenizers. 

Sorry for the inconvenience. <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hi @mfuntowicz,,

The problem is still there in 2.10.1. Could you please reopen the issue and fix it?
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
