Tokenizer methods and padding
## ❓ Questions & Help

I wanted to know whether there was a perticular reason why the `get_special_tokens_mask` method of the tokenizer does not also return as mask over padding tokens, only <CLS> and <SEP> tokens, in the case where `already_has_special_tokens=True` ? I had to rewrite a custom function for my usecase, but it seemed off.

Also, I think there should be an additional `padding` kwarg in the method, which if provided would return a longer mask then the sum of lenfgths of `token_ids_0` and `token_ids_1`, in the case where `already_has_special_tokens=False`. The same should be true for `build_inputs_with_special_tokens` IMO. What do yoy think ?

I have no strong opinion about this. Wdyt @LysandreJik?

Related to this though, this is how I'm proposing to mask the padding tokens in Masked language modeling batches in the `run_lm_finetuning` script: https://github.com/huggingface/transformers/pull/2570/commits/55939b5707066f612b0b2390787b325d30af728c#diff-713f433a085810c3d63a417486e56a88R205-R206<|||||>Since you are already caching the encoded examples, I think you can do: `batch_encode_plus.(..., pad_to_max_length=True)` in both Dataset's `__init__`, instead of repeating this for each epoch. This will also get rid of the introduced `collate_fn` logic you introduce.

Regarding the issue, I just think it's surprising `get_special_tokens_mask` does not consider padding tokens as special tokens, requiring them to be handled separately, for instance as you did.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
