Improve the documentation for TrainingArguments.label_names, and if possible raise an error if users misinterpret this attribute like I did
### Original Issue Title: Possible typo in trainer.py: prediction_step(), forgetting to exclude loss item of outputs dict when assigning logits

_**Update**: I determined the root cause of my error to stem from an incorrect assignment of `TrainingArgument.label_names`. **There is not a typo** in `Trainer.prediction_step()`, as I've suggested below. However there is still an issue: see my comment for elaboration._

I was using the `Trainer` trying to fine-tune [KB-Bert-Base-Swedish-Cased](https://huggingface.co/KB/bert-base-swedish-cased) for multi-class SequenceClassification, when I got a `IndexError: tuple index out of range` during the evaluation stage (I set up `Trainer` to evaluate after each Epoch).

I started PDB and paused at this line in the evaluation phase: 

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1805

With the debugger, I saw that `loss=None`, `labels=None`, and `logits` is actually `tuple` with two items. The first item is the prediction loss as, and the second element is the actual output logits from the models forward pass.

I think this strange assignment of the local `logits` variable is coming from here, inside `prediction_step`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1933

As the `outputs `dict includes the loss, and "loss" is not in ignore_keys, the loss value in outputs gets baked into `logits`.

I'm pretty sure it's a typo, as when I'm comparing it to a few lines above, (which is executed when has_labels=True), the similar line is:  

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1922

The above links are all from Version 4.4.2, but this possible typo is still present in master:

https://github.com/huggingface/transformers/blob/9856c9213dfe9f8355fe00dd6cd0fa1ceae4fa5a/src/transformers/trainer.py#L1966

I haven't been able to read and grasp the code too much, but it looks to me like either we're forgetting to ignore the "loss" key in outputs, or the return statement of `prediction_step` should be somehaw unpacking the logits tuple, so the two variables in "logits" tuple are unpacked into `loss` and` logits`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1947



**For clarity, this is the stacktrace of how I encounter the tuple index error from the above typo:**

In the evaluation phase, `prediction_loop` runs over all the batches in my dev dataset. It gets the model output/prediction of each dev batch here:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1805

Later in` prediction_loop`, we, concatenate each prediction batch with the previous predictions here, calling the function `nested_concat`:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer.py#L1810

Inside `nested_concat`, in the line below, `new_tensors` is the above mentioned "logits" tuple. 
https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L95

The above line does a recursive call to `nested_concat`, and we arrive in the line below.

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L97

Which calls this:

https://github.com/huggingface/transformers/blob/6bc89ed9295443e5a3ee236ad544101752563917/src/transformers/trainer_pt_utils.py#L58

And I get a index error, as it's trying to index into what is actually the `loss` tensor.
### Update:

**I was wrong, my error is originally coming from badly assigned TrainingArguments.label_names. However, I strongly recommend fixes.**

Continued investigating and realised my error is appearing because I don't understand the attribute `TrainingArguments.label_names`. I thought that it should be a list of strings, where each string is my specified name of a class.

**Some background on my code/data structure:**

I'm doing Multi-Class classification on sentences and my training data is an ordered list of a string sentences, and an associated list of classes in string form (i.e. the class's name). I.e. `sentences[i]` is a sentence sample, and `classes[i]` is the class of that sentence, more specifically the _name_  of that class, a string.

If I understand the HuggingFace - Transformers documentation correctly, I should be passing these classes to the Model as the `indices` to a One-Hot encoded list of classes instead. Or in another interpretation, class[i] should be a class number.  So in my custom Dataset class, I send my sentences through a Transformers Tokenizer, and I use `MultiLabelBinarizer` from scikit-learn to One-Hot-Encode all my classes, convert it to a tensor, and then call argmax(dim=-1) on the `classes` tensor.

Of course, I don't want the metrics report to just say "class 0 has the f1-score X", so I thought I could pass original class names to Trainer to be used when printing this metrics report. This is what I thought `TrainingArgument.label_names` was for, so I set `TrainingArguments.label_names = MultiLabelBinarizer.classes_`.

**How my error appears**

During the evaluation stage in `prediction_step()`, I saw with pdb that `inputs` indeed has a `labels` item, which makes sense as I'm trying to evaluate how well the model is performing here.

I now understand that `prediction_step()` is used both when **predicting** _(i.e. we have no target labels in `inputs` and thus expect to not obtain a loss value)_, and for **evaluating** _(i.e. we **do** have target labels and should be able to get a loss value)_.

And of course, this is what the `has_labels` variable is used for - indicating the presence of target labels in `input`, and thus that `prediction_step()` should be able to get a loss value using `Trainer.compute_loss`. However if `has_labels=False`, `prediction_step()` assumes that `outputs` variable **will not** have the item `loss`, and so we do not need to ignore this key when converting the `outputs` dict/SequenceClassificationOutput.


However, since I apparently specified `TrainingArguments.label_names` incorrectly, `has_labels` becomes False _when it shouldn't be_, and everything gets messed up. `prediction_step()` thus assumes that we're predicting and doesn't filter out `loss` in outputs, which leads to the error described in my first post.

I still don't understand what `TrainingArguments.label_names` is or should be.
I recommend that two things should be done:

- Improve the documentation regarding `TrainingArguments.label_names`, specifying the expected format, etc. 
- Earlier in the `Trainer` class, check that `TrainingArguments.label_names` is reasonably formatted and raise a specific error if it isn't, so the user doesn't recieve the above extremely confusing rabbit hole of an error that I did.. <|||||>Ping @sgugger <|||||>I'm unsure what you want to improve, the [documentation](https://huggingface.co/transformers/main_classes/trainer.html#trainingarguments) states the following:

"""
label_names (List[str], optional) –

The list of keys in your dictionary of inputs that correspond to the labels.

Will eventually default to ["labels"] except if the model used is one of the XxxForQuestionAnswering in which case it will default to ["start_positions", "end_positions"].
"""

It's clearly indicated it needs to be a list of strings, that have to be the names of the keys for the labels in your input dictionary. I have no idea what your `MultiLabelBinarizer.classes_` contains since you did not share the code of this class.

More generally, please look at the [warning in the Trainer documentation](https://huggingface.co/transformers/main_classes/trainer.html#trainer) since you appear to be using the Trainer with a model that is not a model of the Transformers library.
<|||||>Oh alright, I didn't see that warning. Thank you!

The `MultiLabelBinarizer` from scikit-learn transforms list of class/label strings into a matrix, where each row is a one-hot-encoded version of the label. `MultiLabelBinarizer.classes_` returns the list of all class/label names detected in the original class list, with same ordering as the one-hot-encoded version.

It sounds like I understood `TrainingArguments.label_names` correctly then, but that my usage of a custom model is messing up the behaviour somehow.  Are there any tips/strategies to fix these strange behaviours? Should I just override `prediction_step` and try to fix how has_labels is being assigned?<|||||>The easiest way to have your custom model work with `Trainer` with no strange behavior is to subclass `PreTrainedModel` (for instance by copying the `XXXForSequenceClassification` and tweaking it to your needs). Otherwise, subclassing and overriding the `prediction_step` method is the most straightforward path.<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.<|||||>I ran into exactly the same issue today.
I was also thinking that the parameter `label_names` in `TrainingArguments` refers to `data["train"].features["label"].names`.
The error message `IndexError: tuple index out of range` was not helpful at all and I only found the problem by trial and error.

Actually, I was not able to find the description for `label_names` in the [documentation](https://huggingface.co/docs/transformers/v4.14.1/en/main_classes/trainer#transformers.TrainingArguments) but only in the linked source code.

Besides, I don't even understand what "The list of keys in your dictionary of inputs that correspond to the labels." should mean.

What "dictionary of inputs" and what "list of keys"?

My dataset looks like this
```
DatasetDict({
    train: Dataset({
        features: ['text', 'label'],
        num_rows: 9245
    })
    test: Dataset({
        features: ['text', 'label'],
        num_rows: 1028
    })
})
```
The only dictionaries I see is `DatasetDict` with keys "train" and "test" and each `Dataset` with keys "features" and "num_rows".

It would be really helpful if the description of the parameter `label_names` and the error message could be improved.<|||||>YES I completely agree. This was very confusing in the documentation. I also interpreted it to mean the list of keys in my label2id dictionary, but it turns out that all it wanted for `label_names` in my case was `['labels']`. That is the name of the column in my input that holds the labels. I hope this helps anyone who was still struggling to understand what "a list of the names of the keys for the labels in your input dictionary" means :)