GPT2DoubleHeadsModel documentation example question (error in documentation)?
Hello,
I was reading the documentation for the GPT2DoubleHeadsModel, and I have a question.
In the documentation, the example shown is:
```python
import torch
from transformers import GPT2Tokenizer, GPT2DoubleHeadsModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2DoubleHeadsModel.from_pretrained('gpt2, return_dict=True)

# Add a [CLS] to the vocabulary (we should train it also!)
num_added_tokens = tokenizer.add_special_tokens({'cls_token': '[CLS]'})
embedding_layer = model.resize_token_embeddings(len(tokenizer))  # Update the model embeddings with the new vocabulary size
choices = ["Hello, my dog is cute [CLS]", "Hello, my cat is cute [CLS]"]
encoded_choices = [tokenizer.encode(s) for s in choices]
cls_token_location = [tokens.index(tokenizer.cls_token_id) for tokens in encoded_choices]
input_ids = torch.tensor(encoded_choices).unsqueeze(0)  # Batch size: 1, number of choices: 2
mc_token_ids = torch.tensor([cls_token_location])  # Batch size: 1
outputs = model(input_ids, mc_token_ids=mc_token_ids)
lm_logits = outputs.lm_logits
mc_logits = outputs.mc_logits
```
Now, I don't see any main question statement in this example, although I see two multiple-choice options. The way I used to use `GPT2DoubleHeadsModel` is I first did `tokenizer(question_statement, option_statement)` and do `encoded_dict['input_ids']` to extract `input_ids` and similarly `encoded_dict['token_type_ids']` to extract the `token_type_ids`. Has this changed? I am getting an impression that the example is wrong (maybe the example could apply for BERT, but not GPT2DoubleHeadsModel). Is this an error in the documentation? I thought that, since GPT-2 does causal language modeling, question statement and option statement has to be encoded together, and put the `[CLS]` token at the end (usually), so that GPT-2 can apply the causal language modeling process to solve the multiple-choice problem.

Reading this again, I think the examples for BERTForMultipleChoice and GPT2DoubleHeadsModel are flipped.

 Thanks,
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
