Possible error in the HuggingFace Transformers documentation?
Hello,

According to HuggingFace Transformers documentation website (https://huggingface.co/transformers/model_doc/gpt2.html#gpt2doubleheadsmodel), under the GPT2DoubleHeadsModel, it defines the output lm_prediction_scores as the following:

`lm_prediction_scores: torch.FloatTensor of shape (batch_size, num_choices, sequence_length, config.vocab_size)`

To me this doesn't make sense. Shouldn't the dimension of `lm_prediction_scores` for `GPT2DoubleHeadsModel` be just `(batch_size, sequence_length, config.vocab_size)`? [no `num_choices` in the middle]

Thank you,

The documentation seems correct to me. Have you tried out the example code that's been provided? If you run it and check the resulting `lm_prediction_scores`, you'll see its shape is `torch.Size([1, 2, 7, 50258])`, 2 being the length of `choices`.

This comment, and the linked blog post, explains the *DoubleHeadsModel pretty well:
https://github.com/huggingface/transformers/issues/1794#issuecomment-552627190<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
