Correctly tokenize sentence pairs
Hey,

I saw different ways to tokenize sentence pairs and the intuitive one is not shown here:

https://huggingface.co/transformers/preprocessing.html#preprocessing-pairs-of-sentences

So, I am asking here if I do right.

I encode pairs of sentence using list of lists. Instead handing over two seperate lists for each sentences, I handover a list of list, where each element is a list with  a single sentence pair. So:

pairs=[[sen1, sen2],[sen1,sen2],....]

Is this hopefully right too?
This would be still interesting.

For tokenizing a list of pairs I get 

```
input_ids = tokenizer(pairs, max_length=50, padding="max_length",truncation=True, return_tensors="tf")

 'token_type_ids': <tf.Tensor: shape=(1, 50), dtype=int32, numpy=
array([[0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
        1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0],
....
```
So I wonder I am doing right as it seems the padded values are connetced to the 1. sentences (0 id)
SO token type IDS are 0 for the padded places, is that right?<|||||>Hey @datistiquo ,

As one can see in the following script:

```python
#!/usr/bin/env python3
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")

input_ids = tokenizer([["hey hello there", "what is going on"], ["peter is in the", "what is going on"]], max_length=20, padding="max_length", truncation=True, return_tensors="tf")

print("List of pairs", input_ids)

input_ids = tokenizer("hey hello there", "what is going on", max_length=20, padding="max_length", truncation=True, return_tensors="tf")

print("Pair", input_ids)
```

tokenizing a list of pairs should be done exactly as proposed by you. Regarding the token_type_ids it is also correct that padded places should have a value of 0. In general if a model does not make use of `token_type_ids`, we return a 0 for such a model, see: https://github.com/huggingface/transformers/blob/6b034309ca4ca2ec6e5c3cacda92a448fa10b921/src/transformers/models/roberta/tokenization_roberta.py#L233 . So for padded tokens that should be discarded in the model, 0 seems like the most sensible choice to me.<|||||>This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.