Supported language information by model
Hi there,

Another feature/documentation request. I am evaluating the language support of all pre-trained models distributed via huggingface :)

I have quickly looked into the code and happily found that XLNet and FlauBERT models have that information: https://github.com/huggingface/transformers/search?q=lang&unscoped_q=lang

Do you plan in the short term to add an `available_languages` attribute to all pre-trained models?

If not, happy to do that investigation and share results.

Cheers,

Alex


Hi Alex, language support is described in the metadata in the models' [model cards](https://github.com/huggingface/transformers/tree/master/model_cards) and then rendered in a searchable way on the https://huggingface.co/models website.

The mapping is not exhaustive right now because a lot of the canonical/historical models do not have a model card yet. Feel free to create them for the models you're researching. cc @thomwolf @LysandreJik @clmnt <|||||>Thanks, that's a good place to start. I am more interested in the canonical/historical models as you say. I see that some README.md in model cards have a 
```
---
language:
- bulgarian
- czech
- polish
- russian
- ...
---
```
Are you OK for me to do a pull request to add these?

I would also like to standardize languages using the ISO 693-1 two-letter code (https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes). Do you agree on this standard? <|||||>Sounds good to me!<|||||>Super, I have forked the repo and created a branch. Expect a pull request in the new few weeks :) so we can close this.<|||||>FYI @MobiusLooper<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
