Hi,
LongFormer is an encoder-only Transformer (similar to BERT/RoBERTa), it only has a different attention mechanism, allowing it to be used on longer sequences.
The author also released LED (LongFormer Encoder Decoder), which is a seq2seq model (like BART, T5) but with LongFormer as encoder, hence allowing it to be used to summarize long documents for instance, or translate long texts.
Weights are on the hub: Models - Hugging Face 8