I’m all for usability and have made many suggestions that focus on the user side of things, but decisions need to be made as to what this library should provide and what should be considered user-specific scenarios and how intricate it is for users to implement such functionality themselves. You saying that this library should provide x is your subjective opinion but does not go well with the philosophy of the library. The library cannot implement all small code snippets that individual users want, especially not if those are easy to do yourself.
The library does provide some useful, general help such as the Trainer class, but a sequence summary is more subjective: it can be seen as a final classification module on top of a model, but then the decision needs to be made how to implement it. Dropout? Activation? Mlp? How many layers? Additional skip connections? Perhaps even just ensemble first? Etc. There are so many options and the ‘right’ option will depend on the model and your use-case. So rather than implementing something that is just half-and-half, we can leave that implementation up to the user. As you can see from the code from sequence summary, it isn’t hard to get the position of the CLS token or mean of tokens which, as a basic sentence embedding, works well.
You have to remember that some models are basically extensions of others (Roberta and Bert) but that others are completely different (Bert and gpt2). It doesn’t make sense to force them into the same position especially if it is easy for users to do this themselves.
If you really need this functionality to get sentence embeddings, and you do not have the skill to implement it yourself, you can use the sentence-transformers 2 library which is intended exactly for this purpose.