error when using the forward() function of the LongformerLayer class from the LongformerForMultipleChoice model
Hello, 

Sorry if my question sounds a bit silly, but I just have a question:
I am trying to feed in the hidden output of the embedding layer of the `LongformerForMultipleChoice` model directly into the m-th layer of the same model. Each of my multiple-choice question that has 4 options.

When I do:
```Python
my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask=my_attention_mask,output_attention=False)
```
, an this error is generated:
```Python
File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 384, in _sliding_chunks_query_key_matmul
    batch_size, seq_len, num_heads, head_dim = query.size()

ValueError: too many values to unpack (expected 4)
```
Here, `my_attention_mask` is the same attention mask that I would specify under the regular `LongformerForMultipleChoice` command. `my_attention_mask` was generated by:
```Python
# I am using the LongformerForMultipleChoice model, where each multiple choice question has 4 options.
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
my_attention_mask = {k: v.unsqueeze(0) for k,v in encoded_dict.items()}['attention_mask']
my_attention_mask 
>>> tensor([[[1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0],
         [1, 1, 1,  ..., 0, 0, 0]]])

# I can use this my_attention_mask in the regular command without an error, as below:
longformer_output= my_Longformer_multiple_choice_model(input_ids=input_ids,....,attention_mask=my_attention_mask)
```
Also, the `hidden_output` in my command was generated by the following:
```Python
encoded_dict = longformer_tokenizer(question_list, option_list,
                             return_tensors = 'pt',
                             padding ='max_length')
hidden_output = my_Longformer_multiple_choice_model(**{k: v.unsqueeze(0) for k,v in encoded_dict.items()}, labels = mc_labels)[2][0][:,:,:]
hidden_output.size()
>>> torch.Size([4, 4096, 768])
```

I am suspecting the value error is generated because the form of `my_attention_mask` is wrong. What should I pass for the `attention_mask` parameter in the command `my_Longformer_multiple_choice_model.encoder.layer[layer_index].forward(hidden_output, 
                                                 attention_mask,output_attention=False)`?

Thank you,
@LysandreJik 
@NielsRogge 
@sgugger 