Strange error while using the `LongformerForMultipleChoice`
Hello,

I am trying to use `LongformerForMultipleChoice` model, and the code I am using is the following:
```python
# import the pre-trained HuggingFace Longformer tokenizer.
longformer_tokenizer = LongformerTokenizer.from_pretrained('allenai/longformer-base-4096')

# get the pre-trained HuggingFace Longformer
best_model_longformer = LongformerForMultipleChoice.from_pretrained('allenai/longformer-base-4096',
                                                         output_hidden_states = True)

# my multiple choice question has 4 options.
question_list = [main_question, main_question, main_question, main_question]
options_list = [option1, option2, option3, option4]
mc_labels = torch.tensor([my_answer])

encoded_dict = longformer_tokenizer(question_list, options_list,
                             return_tensors = 'pt',
                             add_prefix_space = True,
                             padding = True)
input_hidden_state = best_model_longformer(
                       **{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, 
                        labels = mc_labels)[2][0][:,:,:].detach()
```

and I am getting the error below:
```
/home/ec2-user/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py:71: UserWarning: This overload of nonzero is deprecated:
        nonzero()
Consider using one of the following signatures instead:
        nonzero(*, bool as_tuple) (Triggered internally at  /pytorch/torch/csrc/utils/python_arg_parser.cpp:766.)
  sep_token_indices = (input_ids == sep_token_id).nonzero()
Traceback (most recent call last):
  File "SEED_125_V20_15_LONGFORMER.py", line 427, in <module>
    main_function('/home/ec2-user/G1G2.txt','/home/ec2-user/G1G2_answer_num.txt', num_iter)
  File "SEED_125_V20_15_LONGFORMER.py", line 389, in main_function
    best_model_longformer)
  File "SEED_125_V20_15_LONGFORMER.py", line 198, in fill_MC_loss_accuracy_tensor
    input_hidden_state = best_model_longformer(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:].detach()
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 1808, in forward
    loss = loss_fct(reshaped_logits, labels)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/modules/loss.py", line 948, in forward
    ignore_index=self.ignore_index, reduction=self.reduction)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2422, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "/home/ec2-user/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 2218, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
IndexError: Target 1 is out of bounds.
```
How can I fix this error?

I also tried solving this issue with:
```python
# group question_list and options_list as a single list rather than specifying them seperately
encoded_dict = longformer_tokenizer([question_list, options_list],
                             return_tensors = 'pt',
                             add_prefix_space = True,
                             padding = True)
```
But this generates a different error, saying:
```
ValueError: 2 expected but found 1
```
PS: I don't think my Longformer model is correctly getting that my multiple-choice questions have 4 options...is there any way to make Longformer to take my mutiple-choice questions with the ones with 4 options (instead of 2 options)?

Thank you.

PS: I am interested in extracting hidden embedding more than the error or the logit themselves
The way `xxxForMultipleChoice` models work is actually a bit tricky. It works as follows (based on the [original explanation by the author of BERT](https://github.com/google-research/bert/issues/38)):

Given a question, and several options, the question + options are processed by the model independently. So they will look as follows:  `[CLS] question [SEP] option 1 [SEP]` first, then `[CLS] question [SEP] option 2 [SEP]` second, and so on. 

So when you're using the tokenizer to encode the input, it should be used as follows:
```
# my multiple choice question has 4 options.
question = "this is a question"
option1 = "option 1"
option2 = "option 2"
option3 = "option 3"
option4 = "option 4"

encoded_input = longformer_tokenizer([question, question, question, question], 
                                 [option1, option2, option3, option4], 
                                return_tensors='pt', 
                                padding='max_length')
```
we need to `unsqueeze` the values of that dictionary, so to make sure they are all of shape (batch_size, num_choices, seq_len), or thus (1, 4, 4096). Also, the answer should be a tensor having shape (batch_size,), so in our case this is just a tensor containing a single element, containing the index of the correct option. Suppose the correct option is 3, then `answer` will be `torch.tensor([2])` (since indexing starts at zero). Next, we can run the forward pass as follows:

```
mc_labels = torch.tensor([2])
outputs = model(**{k: v.unsqueeze(0) for k,v in encoding.items()}, labels=mc_labels,
                                 return_dict=True)  # batch size is 1
```

The `outputs` will be a `MultipleChoiceOutput` containing the loss and the logits. The logits are of shape (batch_size, number of choices), so (1, 4). 

If you want to train the model, simply get the loss using `outputs.loss` and perform `loss.backward()`. If you want to get the predictions of the model, convert the logits into predictions by typing `outputs.logits.argmax(-1)`. 

UPDATE: fixed the fact that the answer shouldn't be unsqueezed, since it's just a tensor of shape (batch_size,). 
UPDATE: fix indexing of answer.

<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
