I don’t have a good answer for this, but I have seen people breaking longer documents into chunks, getting the label for each chunk and then aggregate it.
Second approach is, now with recent models like longformer it’s possible to train models with large sequence length so that you won’t need to chunk the document.
@sgugger might have a better answer.