Optimize XLNet model to generate embedding of long documents
We experiment generating embeddings with TransformerXL and XLnet. Our documents have 5000 to 80000 characters each.

We got an average of 0.8 second per document with TransformerXL and 1.3 second per document with XLNet.

To optimize XLNet we found that using only 200 tokens per call is optimal. 
A ratio around 350 tokens/second with xlnet-base-cased and 100 tokens/second with large.

Any idea if XLNet can be optimize? 

    tokenizer = XLNetTokenizer.from_pretrained("xlnet-base-cased")
    model = XLNetModel.from_pretrained("xlnet-base-cased")

    outputs = model(text_tokens, mems=mems)