Hi,
LongFormer itself is a Transformer encoder, and that’s more than sufficient to perform sentiment analysis. You can just use LongFormerForSequenceClassification, like so:
from transformers import LongformerTokenizer, LongformerForSequenceClassification
import torch

tokenizer = LongformerTokenizer.from_pretrained('allenai/longformer-base-4096')
model = LongformerForSequenceClassification.from_pretrained('allenai/longformer-base-4096')

inputs = tokenizer("This text is positive", return_tensors="pt")
labels = torch.tensor([1]).unsqueeze(0)  # Batch size 1
outputs = model(**inputs, labels=labels)
loss = outputs.loss
logits = outputs.logits

Note that this model will have a classification head that is randomly initialized, so you’ll need to fine-tune it on a custom dataset.