title: LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
    Large Language Models
summary: Scientific equation discovery is a fundamental task in the history of
  scientific progress, enabling the derivation of laws governing natural
  phenomena. Recently, Large Language Models (LLMs) have gained interest for this
  task due to their potential to leverage embedded scientific knowledge for
  hypothesis generation. However, evaluating the true discovery capabilities of
  these methods remains challenging, as existing benchmarks often rely on common
  equations that are susceptible to memorization by LLMs, leading to inflated
  performance metrics that do not reflect discovery. In this paper, we introduce
  LLM-SRBench, a comprehensive benchmark with 239 challenging problems across
  four scientific domains specifically designed to evaluate LLM-based scientific
  equation discovery methods while preventing trivial memorization. Our benchmark
  comprises two main categories: LSR-Transform, which transforms common physical
  models into less common mathematical representations to test reasoning beyond
  memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven
  problems requiring data-driven reasoning. Through extensive evaluation of
  several state-of-the-art methods, using both open and closed LLMs, we find that
  the best-performing system so far achieves only 31.5% symbolic accuracy. These
  findings highlight the challenges of scientific equation discovery, positioning
  LLM-SRBench as a valuable resource for future research.
paper_authors._id: 67fddc276a3c533dc1d3bcbe
paper_authors.name: Parshin Shojaee
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2025-04-15T04:10:17.364Z
paper_authors._id: 6520621836008ecc88699622
paper_authors.avatarUrl: /avatars/b08c00af00f1736a4f4938443e575b0e.svg
paper_authors.fullname: Parshin Shojaee
paper_authors.type: user
paper_authors.user: parshinsh
paper_authors._id: 67fddc276a3c533dc1d3bcbf
paper_authors.name: Ngoc-Hieu Nguyen
paper_authors._id: 67fddc276a3c533dc1d3bcc0
paper_authors.name: Kazem Meidani
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-04-15T07:53:43.431Z
paper_authors._id: 67b4cdee376cfc783f9ec8cf
paper_authors.avatarUrl: /avatars/d9a9f320cd01dc5addfca14feefefef4.svg
paper_authors.fullname: Meidani
paper_authors.type: user
paper_authors.user: mkmeidani
paper_authors._id: 67fddc276a3c533dc1d3bcc1
paper_authors.name: Amir Barati Farimani
paper_authors._id: 67fddc276a3c533dc1d3bcc2
paper_authors.name: Khoa D Doan
paper_authors._id: 67fddc276a3c533dc1d3bcc3
paper_authors.name: Chandan K Reddy
paper_title: LLM-SRBench: A New Benchmark for Scientific Equation Discovery with
    Large Language Models
paper_summary: Scientific equation discovery is a fundamental task in the history of
  scientific progress, enabling the derivation of laws governing natural
  phenomena. Recently, Large Language Models (LLMs) have gained interest for this
  task due to their potential to leverage embedded scientific knowledge for
  hypothesis generation. However, evaluating the true discovery capabilities of
  these methods remains challenging, as existing benchmarks often rely on common
  equations that are susceptible to memorization by LLMs, leading to inflated
  performance metrics that do not reflect discovery. In this paper, we introduce
  LLM-SRBench, a comprehensive benchmark with 239 challenging problems across
  four scientific domains specifically designed to evaluate LLM-based scientific
  equation discovery methods while preventing trivial memorization. Our benchmark
  comprises two main categories: LSR-Transform, which transforms common physical
  models into less common mathematical representations to test reasoning beyond
  memorized forms, and LSR-Synth, which introduces synthetic, discovery-driven
  problems requiring data-driven reasoning. Through extensive evaluation of
  several state-of-the-art methods, using both open and closed LLMs, we find that
  the best-performing system so far achieves only 31.5% symbolic accuracy. These
  findings highlight the challenges of scientific equation discovery, positioning
  LLM-SRBench as a valuable resource for future research.
paper_ai_summary: LLM-SRBench is a comprehensive benchmark designed to evaluate LLMs in scientific equation discovery by preventing memorization and using novel problem representations, indicating significant challenges in this domain.
paper_ai_keywords: Large Language Models LLM-SRBench scientific equation discovery LSR-Transform LSR-Synth symbolic accuracy