LukeForEntitySpanClassification - ValueError: only one element tensors can be converted to Python scalars
## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->

- `transformers` version: 4.6.0.dev0
- Platform: Linux-5.10.25-linuxkit-x86_64-with-debian-10.1
- Python version: 3.7.4
- PyTorch version (GPU?): 1.8.1+cu102 (False)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: none

### Who can help
@LysandreJik 

## Information

Model I am using (Bert, XLNet ...):
- "studio-ousia/luke-large-finetuned-conll-2003"

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

Example script: 
https://huggingface.co/transformers/master/model_doc/luke.html#lukeforentityspanclassification

## To reproduce

Steps to reproduce the behavior:

1. Run [this](https://github.com/loretoparisi/hf-experiments/blob/master/src/luke/run.py) script or the code example below adapted from the documentation [here](https://huggingface.co/transformers/master/model_doc/luke.html#lukeforentityspanclassification)
2. Error:
```
Traceback (most recent call last):
  File "src/luke/run.py", line 71, in <module>
    predicted_class_idx = logits.argmax(-1).item()
ValueError: only one element tensors can be converted to Python scalars

```

```python
import os
from transformers import LukeTokenizer, LukeModel, LukeForEntityPairClassification, LukeForEntitySpanClassification
ner_model = LukeForEntitySpanClassification.from_pretrained("studio-ousia/luke-large-finetuned-conll-2003",
    cache_dir=os.getenv("cache_dir", "../../models"))
ner_tokenizer = LukeTokenizer.from_pretrained("studio-ousia/luke-large-finetuned-conll-2003",
    cache_dir=os.getenv("cache_dir", "../../models"))

# List all possible entity spans in the text
word_start_positions = [0, 8, 14, 17, 21]  # character-based start positions of word tokens
word_end_positions = [7, 13, 16, 20, 28]  # character-based end positions of word tokens
entity_spans = []
for i, start_pos in enumerate(word_start_positions):
     for end_pos in word_end_positions[i:]:
         entity_spans.append((start_pos, end_pos))

inputs = ner_tokenizer(text, entity_spans=entity_spans, return_tensors="pt")
outputs = ner_model(**inputs)
logits = outputs.logits
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", ner_model.config.id2label[predicted_class_idx])
```

## Expected behavior
no errores, print predicted classes

Yes, it should be updated, because `LukeForEntitySpanClassification` classifies each possible entity span independently, so it should instead become this:

```
predicted_class_indices = logits.argmax(-1).squeeze().tolist()
for span, predicted_class_idx in zip(entity_spans, predicted_class_indices):
  if predicted_class_idx != 0:
    print(text[span[0]:span[1]], model.config.id2label[predicted_class_idx])
```

The logits are of shape `(1,15,5)`, because there are 15 possible entity spans and 5 classes.

Thanks for reporting. Will fix this!

cc @ikuyamada <|||||>@NielsRogge cool, updated [my code](https://github.com/loretoparisi/hf-experiments/blob/master/src/luke/run.py#L71) and tested, now it works ok:
```
Beyoncé PER
Los Angeles LOC
```
