Multilabel Sequence Classification in trainer
# 🚀 Feature request

We need to be able to use the trainer for multilabel classification problems.

## Motivation

Right now we create our models in the old fashioned way, with a sigmoid layer at the end so we can do multilabel. However if we could use the trainer directly, we wouldn't need to maintain different training scripts. Are there any plans for adding this to the trainer?

## Your contribution

I could try to help but I don't even know where to start with.

Thanks you very much for reading

Hi @LysandreJik, 
If no one is working on this can I start on this feature? 
I imagine this will not be that difficult and should be possible by using the sigmoid instead of the softmax, where we're calculating a probability between 0-1 for each class which will be encapsulated in a class similar to [ModelName]ForSequenceClassification. <|||||>Hi @vimarshc, I believe @abhishekkrthakur is working on this in https://github.com/huggingface/transformers/pull/11012<|||||>Thanks for the update! 
Shall try to make myself useful for some other issue. Haha. <|||||>#11012 is merged and 4.6.0 is released, is this feature already there?<|||||>I believe there is now multi-label classification within the models, and changing a model configuration argument (`config.problem_type = "multi_label_classification"`) should enable that out of the box. Have you tried it out?<|||||>I did but I didn't have enough time to try it in depth, I had some problems with the labels format. However I did not find any documentation or examples in the docs.<|||||>This seems to work but I found a weird problem that might be interesting to solve. 

When you binarize the labels, if you are training with pytorch it will throw an error because `BCEWithLogitsLoss` expects floats and not ints. This is counterintuitive for me, I had to cast my labels to floats and then it worked.

Also, there is not a lot of information about the anything of this in the documentation and the notebook for multi-label sequence classification still uses the old training loops, instead of the trainer.<|||||>Indeed, it would be nice to put this in the documentation, it's sparse on this subject right now.<|||||>Sorry for the flood but, is there a  way to instance a multilabel model on a pipeline? I'm trying it but I don't think it is working, the prediction probabilities always sum up to 1.<|||||>I believe this is being taken care of in https://github.com/huggingface/transformers/pull/8328<|||||>Indeed, it looks really good. Thank you<|||||>Is the multilabel pipeline implemented in the last release `4.9.2` @LysandreJik ? If yes, are there examples on how to use it?<|||||>Ssee the docs for the text classification pipeline [here](https://huggingface.co/transformers/master/main_classes/pipelines.html#transformers.TextClassificationPipeline) (See `function_to_apply` argument)

It is currently on the `master` branch and will be in the next release (v4.10.0)<|||||>Great thanks!
However, I don't see a documentation which is specific to multilabel classification (I understand that choosing `function_to_apply='sigmoid'` makes it work though). With the results of the `pipeline` be prettified accordingly?