AttributeError: 'DistilBertConfig' object has no attribute 'return_dict'
## Environment info
- `transformers` version: 3.1.0
- Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (False)
- Tensorflow version (GPU?): 2.3.0 (False)
- Using GPU in script?: <True>
- Using distributed or parallel set-up in script?: <False>

### Who can help
 Trainer: @sgugger
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 Bart: @sshleifer
 examples/bert-loses-patience: @JetRunner
 examples/token-classification: @stefan-it

## Information

Model I am using (Bert, XLNet ...):

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

```python
def flair_lstm(text):
    sentence = flair.data.Sentence(text)
    flair_sent.predict(sentences=sentence)
    total_sent = sentence.labels
    for label in total_sent:
        value = label.value
        score = label.score
        return '1' if value == 'POSITIVE' else '-1'
```

The tasks I am working on is:
* I'm working with flair to get classification polarities, but the issue seems refer to transformers

## To reproduce

Steps to reproduce the behavior:

1. write
```python
import flair
flair_sent = flair.models.TextClassifier.load('en-sentiment')

def flair_lstm(text):
    sentence = flair.data.Sentence(text)
    flair_sent.predict(sentences=sentence)
    total_sent = sentence.labels
    for label in total_sent:
        value = label.value
        score = label.score
        return '1' if value == 'POSITIVE' else '-1'

df_test = "some test dataframe"
df_test['flair'] = df_test['word'].apply(lambda x: flair_lstm(x))
```
2. See error:

## Traceback:

```python
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-26-1ee39d7138b3> in <module>()
----> 1 df_test['flair'] = df_test['word'].apply(lambda x: flair_lstm(x))

10 frames
pandas/_libs/lib.pyx in pandas._libs.lib.map_infer()

/usr/local/lib/python3.6/dist-packages/transformers/configuration_utils.py in use_return_dict(self)
    217         """
    218         # If torchscript is set, force `return_dict=False` to avoid jit errors
--> 219         return self.return_dict and not self.torchscript
    220 
    221     @property

AttributeError: 'DistilBertConfig' object has no attribute 'return_dict'
```
I don't see the transformers code that creates this bug. In 3.1.0, `DistilBertConfig` definitely has a 'return_dict' `attribute`. I tried to use your code to investigate the error, but it fails on the line `flair_sent = flair.models.TextClassifier.load('en-sentiment')` for me.

Happy to investigate a code sample that uses transformers and creates the bug, but this looks like a problem to report on the fair GitHub. <|||||>> I don't see the transformers code that creates this bug. In 3.1.0, `DistilBertConfig` definitely has a 'return_dict' `attribute`. I tried to use your code to investigate the error, but it fails on the line `flair_sent = flair.models.TextClassifier.load('en-sentiment')` for me.
> 
> Happy to investigate a code sample that uses transformers and creates the bug, but this looks like a problem to report on the fair GitHub.

I already did, just in case I wanted to report the bug in here. Thank you anyway!<|||||>Don't hesitate to reopen if it ends up being on our side, with a small repro using only transformers ideally.<|||||>It ended being on flair side. Here I'll attached the link for future references [/flairNLP/flair/issues/1841](https://github.com/flairNLP/flair/issues/1841)