Add loss reduction parameter in forward() method
# Add loss reduction parameter in forward() method

Add possibility to choose reduction method in loss functions used in `forward` method in most of the models (like all BERT based, `BertForMaskedLM`, `BigBirdForMaskedLM` etc.). Currently, models use losses without the possibility to pass additional parameters.

From PyTorch docs [nn.CrossEntropyLoss ](https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html):

> reduction (string, optional) – Specifies the reduction to apply to the output: 'none' | 'mean' | 'sum'. 'none': no reduction will be applied, 'mean': the weighted mean of the output is taken, 'sum': the output will be summed. Note: size_average and reduce are in the process of being deprecated, and in the meantime, specifying either of those two args will override reduction. Default: 'mean'

## Motivation

Especially in models like `BertForMaskedLM` passing a reduction like `none` instead of the default `mean` can be very handy to check individual losses for tokens.

In our project, we handle this by creating our own class inheriting from hugginface model and overwriting `forward()` method.

## Your contribution

A quick demonstration of the idea on the example of base `Bert` models: 
https://github.com/marekrydlewski/transformers/commit/83994b12085b3187f0b80fbb9a6d4a7b5e4bc8de

There are no updated docs, it's just a demonstration.

Then it can be used like:
```
model = BertForMaskedLM(...)
output = model(..., loss_reduction='none')
```

This has been asked before, but I don't think this is on the roadmap. If you want to use a different loss reduction, you can easily overwrite the model and plug in your custom loss function. See also https://github.com/huggingface/transformers/issues/9625#issuecomment-762167788 and https://github.com/huggingface/transformers/issues/7024#issue-696684075. 

Otherwise, the library would become a bit cluttered with all kinds of custom parameters, so simplicity is favored.

<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.