BPE vocab
Do you guys have the functionality to support BPE with the models? 
Not speaking for the core developers, but `pytorch-pretrained-BERT` supports it, because:

* GPT-1 use BPE, see code [here](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/tokenization_openai.py#L73)
* GPT-2 use BPE on byte level, see code [here](https://github.com/huggingface/pytorch-pretrained-BERT/blob/master/pytorch_pretrained_bert/tokenization_gpt2.py#L88)

BERT uses a variant of BPE (word pieces) and the pretrained language model for Transformer-XL was trained on WikiText-103 (so it is a word-based model).

<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
