6

Mistral w/ SWASelfExtendMistral LLM Maybe LongLM: Self-Extend LLM Context Window Without Tuning

Although, a few important tokens, which is related to the
understanding of long context and answering questions, may
not be predicted well.

4.3. Performance on real long context tasks
Self-Extend LLM