TF2 support for Longformer
# 🚀 Feature request

Hi, 

I'm currently working on a project involving long documents (6000+ tokens).  I normally work with Tensorflow and I was wondering if there are any plans for adding a Longformer TF model in the near future? My PyTorch knowledge is fairly limited, but given the potential of the Longformer for my project, I would want to learn the basics if there are no plans for adding TF support. 

Kind regards



Yes, we are planning to add this in ~1 month<|||||>Great, looking forward to it, thanks!<|||||>+1 on this @patrickvonplaten any news? :)<|||||>+2 on this<|||||>Finished by end of the week :-) See https://github.com/huggingface/transformers/pull/5764. It's almost finished :-) <|||||>@patrickvonplaten any reference on how to train unsupervised model for longformer (not fine-tuning)?