Option to change loss function for fine tuning
# 🚀 Feature request

## Motivation

I was working in a multi class text classification problem for which I was using `DistilBertForSequenceClassification` and I found out that there is no way for me to change the loss function from CrossEntropyLoss. 

## Your contribution

I can submit a PR, if this feature request is approved. 

You can change the loss function to anything you want. Here's an example:

```
from transformers import BertModel
from transformers.modeling_outputs import SequenceClassifierOutput
import torch.nn as nn

class FancyBertModelWithCustomLossFunction(nn.Module):
    def __init__(self):
        super(FancyBertModelWithCustomLossFunction, self).__init__()
        self.bert = BertModel.from_pretrained("bert-base-uncased")
        self.dropout = nn.Dropout(0.3)
        self.classifier = nn.Linear(768, 1)

    def forward(self, ids, mask, token_type_ids, labels=None):
        outputs = self.bert(ids, attention_mask=mask, token_type_ids=token_type_ids)

        output = self.dropout(outputs.pooler_output)
        logits = self.classifier(output)
        
        loss = None
        if labels is not None:
            if self.num_labels == 1:
                #  We are doing regression
                loss_fct = MSELoss()
                loss = loss_fct(logits.view(-1), labels.view(-1))
            else:
                # you can define any loss function here yourself
                # see https://pytorch.org/docs/stable/nn.html#loss-functions for an overview
                loss_fct = nn.BinaryCrossEntropyLoss()
                # next, compute the loss based on logits + ground-truth labels
                loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))

        return SequenceClassifierOutput(
            loss=loss,
            logits=logits,
            hidden_states=outputs.hidden_states,
            attentions=outputs.attentions,
        )

```<|||||>@NeilsRogge I am aware of this, what I was referring to is an option that can be passed directly to let's say `DistilBertForSequenceClassification` or any other model class without having to write a pytorch model like this. <|||||>This has already been asked before, but we are not planning to do this.

See also [this comment](https://github.com/huggingface/transformers/issues/9625#issuecomment-762167788) in #9625<|||||>Oh ok, got it. Thanks @NeilsRogge. Closing this.