• Long-Range Arena (LRA)5 [232] is a benchmark suite speciﬁcally designed for
evaluating the performance of eﬃcient Transformer models on long-context tasks.
It features a variety of tasks that require reasoning over long contexts, ranging from
1,000 to 16,000 tokens in length. These tasks cover diﬀerent modalities like text, nat-
ural language, synthetic images, and mathematical expressions, and involve various
reasoning types like similarity, structural, and visual-spatial reasoning. Evaluations
can be run under controlled resource constraints like memory budget or execution
time limit. This forces models to be eﬃcient within these limitations, highlighting
their ability to optimize performance under real-world constraints.
Beyond Efficiency