Thanks! Most of the data points that have more than 512 tokens are between 500 and 1000, but it can get up to 2000 or so (which is more “document sentiment analysis” than a “sentence sentiment analysis”).
Regarding Longformer - is there reasonable way to asses how many examples would be needed for fine tuning?