Illegal instruction (core dumped) error: PowerPC8
### System Info

```shell
transformers              2.5.1
python3.8
pytorch                   1.10.2 

All packages installed with conda by way of the conda-forge or powerai repositories, all of them are ppc64-le compatible
```


### Who can help?

I am trying to load a BERT pre-trained model. All imports work fine, and loading the tokenizer works fine, but loading the model does not.

I am running on an nvidia-docker2 container for pytorch, namely ibmcom/pytorch-ppc64le  with updated libraries using only powerai repositories. The error occurs here:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import transformers
from transformers import BertTokenizer, BertForSequenceClassification

model = BertForMaskedLM.from_pretrained("dccuchile/bert-base-spanish-wwm-uncased")

@LysandreJik 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import torch
import torch.nn as nn
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report
import transformers
from transformers import BertTokenizer, BertForSequenceClassification

model = BertForMaskedLM.from_pretrained("dccuchile/bert-base-spanish-wwm-uncased")

### Expected behavior

```shell
What is expected is that the model loads and I can use it to further train it.
```

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.