title: Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon
summary: Large language models (LLMs) often appear to excel on public benchmarks, but
  these high scores may mask an overreliance on dataset-specific surface cues
  rather than true language understanding. We introduce the Chameleon Benchmark
  Overfit Detector (C-BOD), a meta-evaluation framework that systematically
  distorts benchmark prompts via a parametric transformation and detects
  overfitting of LLMs. By rephrasing inputs while preserving their semantic
  content and labels, C-BOD exposes whether a model's performance is driven by
  memorized patterns. Evaluated on the MMLU benchmark using 26 leading LLMs, our
  method reveals an average performance degradation of 2.15% under modest
  perturbations, with 20 out of 26 models exhibiting statistically significant
  differences. Notably, models with higher baseline accuracy exhibit larger
  performance differences under perturbation, and larger LLMs tend to be more
  sensitive to rephrasings indicating that both cases may overrely on fixed
  prompt patterns. In contrast, the Llama family and models with lower baseline
  accuracy show insignificant degradation, suggesting reduced dependency on
  superficial cues. Moreover, C-BOD's dataset- and model-agnostic design allows
  easy integration into training pipelines to promote more robust language
  understanding. Our findings challenge the community to look beyond leaderboard
  scores and prioritize resilience and generalization in LLM evaluation.
paper_authors._id: 67ac216d602eb9ca8a517be6
paper_authors.name: Nurit Cohen-Inger
paper_authors._id: 67ac216d602eb9ca8a517be7
paper_authors.name: Yehonatan Elisha
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-02-13T20:36:02.389Z
paper_authors._id: 662a80358a721ebd0b4f358b
paper_authors.avatarUrl: /avatars/5be5a3c8b13f6f663206a19d0525c18e.svg
paper_authors.fullname: Yehonatan Elisha
paper_authors.type: user
paper_authors.user: Yoniel
paper_authors._id: 67ac216d602eb9ca8a517be8
paper_authors.name: Bracha Shapira
paper_authors._id: 67ac216d602eb9ca8a517be9
paper_authors.name: Lior Rokach
paper_authors._id: 67ac216d602eb9ca8a517bea
paper_authors.name: Seffi Cohen
paper_title: Forget What You Know about LLMs Evaluations - LLMs are Like a Chameleon
paper_summary: Large language models (LLMs) often appear to excel on public benchmarks, but
  these high scores may mask an overreliance on dataset-specific surface cues
  rather than true language understanding. We introduce the Chameleon Benchmark
  Overfit Detector (C-BOD), a meta-evaluation framework that systematically
  distorts benchmark prompts via a parametric transformation and detects
  overfitting of LLMs. By rephrasing inputs while preserving their semantic
  content and labels, C-BOD exposes whether a model's performance is driven by
  memorized patterns. Evaluated on the MMLU benchmark using 26 leading LLMs, our
  method reveals an average performance degradation of 2.15% under modest
  perturbations, with 20 out of 26 models exhibiting statistically significant
  differences. Notably, models with higher baseline accuracy exhibit larger
  performance differences under perturbation, and larger LLMs tend to be more
  sensitive to rephrasings indicating that both cases may overrely on fixed
  prompt patterns. In contrast, the Llama family and models with lower baseline
  accuracy show insignificant degradation, suggesting reduced dependency on
  superficial cues. Moreover, C-BOD's dataset- and model-agnostic design allows
  easy integration into training pipelines to promote more robust language
  understanding. Our findings challenge the community to look beyond leaderboard
  scores and prioritize resilience and generalization in LLM evaluation.
paper_ai_keywords: large language models (LLMs) Chameleon Benchmark Overfit Detector (C-BOD) parametric transformation semantic content labels performance degradation MMLU benchmark prompt patterns statistically significant baseline accuracy larger LLMs Llama family dataset-agnostic model-agnostic training pipelines resilience generalization