Bugs due to design choices in LongformerTokenizer
I am using the transformers library from source (version: 3.0.2). 

```python
import transformers
from transformers import *
longformer_tokenizer = LongformerTokenizer.from_pretrained("allenai/longformer-base-4096")
longformer_tokenizer.tokenize("This is a sample sentence for the tokenizer.")
```
The output I get is 
``` ['This', 'Ġis', 'Ġa', 'Ġsample', 'Ġsentence', 'Ġfor', 'Ġthe', 'Ġtoken', 'izer']```

The design choice here is to use the `Ġ` as a start of every new word (except for the first word). This is in contrast with other tokenizers which insert `##` tokens for suffixes of broken words. 

Due to this slightly different tokenization quirk, many things could break, one of which is the following piece of code in `squad.py`:

https://github.com/huggingface/transformers/blob/02a0b43014ac333a169e99d76aaba023a316e384/src/transformers/data/processors/squad.py#L106-L112

The `doc_tokens` from the processor are whitespace separated tokens, which are to be further tokenized using this code. But since each word is treated individually, and `LongformerTokenizer` doesn't insert `Ġ` for the first token, there is a problem. The resulting `all_doc_tokens` can not be correctly converted to original string using `tokenizer.convert_tokens_to_string` because it is missing the `Ġ`  at the start. 
Thanks for this issue. Longformer was trained on trivia_qa by default and not squad, so the model is not by default compatible with `squad` and needs some special post processing as shown in the example of this model, here:
https://huggingface.co/transformers/model_doc/longformer.html#longformerforquestionanswering

This is also related to: https://github.com/huggingface/transformers/pull/4615 <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
