LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation
  framework
Long-context processing has become a fundamental capability for large
language models~(LLMs). To assess model's long-context performance, numerous
long-context evaluation benchmarks have been proposed. However, variations in
evaluation settings across these benchmarks lead to inconsistent results,
making it difficult to draw reliable comparisons. Besides, the high
computational cost of long-context evaluation poses a significant barrier for
the community to conduct comprehensive assessments of long-context models. In
this paper, we propose LOOM-Scope, a comprehensive and efficient framework for
long-context evaluation. LOOM-Scope standardizes evaluation settings across
diverse benchmarks, supports deployment of efficient long-context inference
acceleration methods, and introduces a holistic yet lightweight benchmark suite
to evaluate models comprehensively. Homepage: https://loomscope.github.io
686df5fecb5725779c60b4b8
Zecheng Tang
686df5fecb5725779c60b4b9
Haitian Wang
686df5fecb5725779c60b4ba
Quantong Qiu
claimed_verified
2025-07-09T08:49:43.850Z
6732fb1d24b316be87acaafe
https://cdn-avatars.huggingface.co/v1/production/uploads/6732fb1d24b316be87acaafe/BzD8HL4vhh3mfeSF3rm_1.jpeg
Quantong Qiu
user
QQTang1223
686df5fecb5725779c60b4bb
Baibei Ji
686df5fecb5725779c60b4bc
Ruoxi Sun
686df5fecb5725779c60b4bd
Keyan Zhou
686df5fecb5725779c60b4be
Juntao Li
686df5fecb5725779c60b4bf
Min Zhang
LOOM-Scope: a comprehensive and efficient LOng-cOntext Model evaluation
  framework
Long-context processing has become a fundamental capability for large
language models~(LLMs). To assess model's long-context performance, numerous
long-context evaluation benchmarks have been proposed. However, variations in
evaluation settings across these benchmarks lead to inconsistent results,
making it difficult to draw reliable comparisons. Besides, the high
computational cost of long-context evaluation poses a significant barrier for
the community to conduct comprehensive assessments of long-context models. In
this paper, we propose LOOM-Scope, a comprehensive and efficient framework for
long-context evaluation. LOOM-Scope standardizes evaluation settings across
diverse benchmarks, supports deployment of efficient long-context inference
acceleration methods, and introduces a holistic yet lightweight benchmark suite
to evaluate models comprehensively. Homepage: https://loomscope.github.io
LOOM-Scope is a framework that standardizes and streamlines the evaluation of long-context performance in large language models.
long-context processing large language models long-context evaluation benchmarks long-context inference acceleration benchmark suite