title: FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial
    Reasoning
summary: Multi-step symbolic reasoning is critical for advancing downstream
  performance on financial tasks. Yet, benchmarks for systematically evaluating
  this capability are lacking. Existing datasets like FinQA and ConvFinQA
  supervise only final numerical answers, without assessing intermediate
  reasoning steps. To address this, we introduce FinChain, the first symbolic
  benchmark designed for verifiable Chain-of- Thought (CoT) financial reasoning.
  Spanning 54 topics across 12 financial domains, Fin- Chain offers five
  parameterized templates per topic, each varying in reasoning complexity and
  domain expertise required. Each dataset instance includes an executable Python
  trace, enabling automatic generation of extensive training data and easy
  adaptation to other domains. We also introduce ChainEval, a new metric for
  automatic evaluation of both final answers and intermediate reasoning.
  Benchmarking 30 LLMs on our dataset, we find that even state-of-the-art models
  have considerable room for improvement in multi-step financial reasoning. All
  templates and evaluation metrics for FinChain are available at https:
  //github.com/mbzuai-nlp/finchain.
paper_authors._id: 6841704275b84c68e75dd774
paper_authors.name: Zhuohan Xie
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-05T12:41:57.397Z
paper_authors._id: 62d7f921720a579b3bd84a72
paper_authors.avatarUrl: /avatars/eea38e90c3595eab1d7e0fdb7418b0d5.svg
paper_authors.fullname: Zhuohan Xie
paper_authors.type: user
paper_authors.user: Zhuohan
paper_authors._id: 6841704275b84c68e75dd775
paper_authors.name: Dhruv Sahnan
paper_authors._id: 6841704275b84c68e75dd776
paper_authors.name: Debopriyo Banerjee
paper_authors._id: 6841704275b84c68e75dd777
paper_authors.name: Georgi Georgiev
paper_authors._id: 6841704275b84c68e75dd778
paper_authors.name: Rushil Thareja
paper_authors._id: 6841704275b84c68e75dd779
paper_authors.name: Hachem Madmoun
paper_authors._id: 6841704275b84c68e75dd77a
paper_authors.name: Jinyan Su
paper_authors._id: 6841704275b84c68e75dd77b
paper_authors.name: Aaryamonvikram Singh
paper_authors._id: 6841704275b84c68e75dd77c
paper_authors.name: Yuxia Wang
paper_authors._id: 6841704275b84c68e75dd77d
paper_authors.name: Rui Xing
paper_authors._id: 6841704275b84c68e75dd77e
paper_authors.name: Fajri Koto
paper_authors._id: 6841704275b84c68e75dd77f
paper_authors.name: Haonan Li
paper_authors._id: 6841704275b84c68e75dd780
paper_authors.name: Ivan Koychev
paper_authors._id: 6841704275b84c68e75dd781
paper_authors.name: Tanmoy Chakraborty
paper_authors._id: 6841704275b84c68e75dd782
paper_authors.name: Salem Lahlou
paper_authors._id: 6841704275b84c68e75dd783
paper_authors.name: Veselin Stoyanov
paper_authors._id: 6841704275b84c68e75dd784
paper_authors.name: Preslav Nakov
paper_title: FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial
    Reasoning
paper_summary: Multi-step symbolic reasoning is critical for advancing downstream
  performance on financial tasks. Yet, benchmarks for systematically evaluating
  this capability are lacking. Existing datasets like FinQA and ConvFinQA
  supervise only final numerical answers, without assessing intermediate
  reasoning steps. To address this, we introduce FinChain, the first symbolic
  benchmark designed for verifiable Chain-of- Thought (CoT) financial reasoning.
  Spanning 54 topics across 12 financial domains, Fin- Chain offers five
  parameterized templates per topic, each varying in reasoning complexity and
  domain expertise required. Each dataset instance includes an executable Python
  trace, enabling automatic generation of extensive training data and easy
  adaptation to other domains. We also introduce ChainEval, a new metric for
  automatic evaluation of both final answers and intermediate reasoning.
  Benchmarking 30 LLMs on our dataset, we find that even state-of-the-art models
  have considerable room for improvement in multi-step financial reasoning. All
  templates and evaluation metrics for FinChain are available at https:
  //github.com/mbzuai-nlp/finchain.
paper_ai_summary: A new benchmark called FinChain evaluates multi-step symbolic reasoning in financial tasks with a focus on intermediate reasoning steps, introducing ChainEval as a metric for assessing both final answers and reasoning processes.
paper_ai_keywords: symbolic reasoning FinQA ConvFinQA Chain-of-Thought (CoT) FinChain multi-step financial reasoning parameterized templates reasoning complexity domain expertise executable Python trace ChainEval LLMs