FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial
  Reasoning
Multi-step symbolic reasoning is critical for advancing downstream
performance on financial tasks. Yet, benchmarks for systematically evaluating
this capability are lacking. Existing datasets like FinQA and ConvFinQA
supervise only final numerical answers, without assessing intermediate
reasoning steps. To address this, we introduce FinChain, the first symbolic
benchmark designed for verifiable Chain-of- Thought (CoT) financial reasoning.
Spanning 54 topics across 12 financial domains, Fin- Chain offers five
parameterized templates per topic, each varying in reasoning complexity and
domain expertise required. Each dataset instance includes an executable Python
trace, enabling automatic generation of extensive training data and easy
adaptation to other domains. We also introduce ChainEval, a new metric for
automatic evaluation of both final answers and intermediate reasoning.
Benchmarking 30 LLMs on our dataset, we find that even state-of-the-art models
have considerable room for improvement in multi-step financial reasoning. All
templates and evaluation metrics for FinChain are available at https:
//github.com/mbzuai-nlp/finchain.
6841704275b84c68e75dd774
Zhuohan Xie
claimed_verified
2025-06-05T12:41:57.397Z
62d7f921720a579b3bd84a72
/avatars/eea38e90c3595eab1d7e0fdb7418b0d5.svg
Zhuohan Xie
user
Zhuohan
6841704275b84c68e75dd775
Dhruv Sahnan
6841704275b84c68e75dd776
Debopriyo Banerjee
6841704275b84c68e75dd777
Georgi Georgiev
6841704275b84c68e75dd778
Rushil Thareja
6841704275b84c68e75dd779
Hachem Madmoun
6841704275b84c68e75dd77a
Jinyan Su
6841704275b84c68e75dd77b
Aaryamonvikram Singh
6841704275b84c68e75dd77c
Yuxia Wang
6841704275b84c68e75dd77d
Rui Xing
6841704275b84c68e75dd77e
Fajri Koto
6841704275b84c68e75dd77f
Haonan Li
6841704275b84c68e75dd780
Ivan Koychev
6841704275b84c68e75dd781
Tanmoy Chakraborty
6841704275b84c68e75dd782
Salem Lahlou
6841704275b84c68e75dd783
Veselin Stoyanov
6841704275b84c68e75dd784
Preslav Nakov
claimed_verified
2025-07-09T15:10:33.641Z
647f7eb25e1bc4753746bf9f
/avatars/cc9c6210fdc822d8a106937e747dff41.svg
Preslav Nakov
user
preslavnakov
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial
  Reasoning
Multi-step symbolic reasoning is critical for advancing downstream
performance on financial tasks. Yet, benchmarks for systematically evaluating
this capability are lacking. Existing datasets like FinQA and ConvFinQA
supervise only final numerical answers, without assessing intermediate
reasoning steps. To address this, we introduce FinChain, the first symbolic
benchmark designed for verifiable Chain-of- Thought (CoT) financial reasoning.
Spanning 54 topics across 12 financial domains, Fin- Chain offers five
parameterized templates per topic, each varying in reasoning complexity and
domain expertise required. Each dataset instance includes an executable Python
trace, enabling automatic generation of extensive training data and easy
adaptation to other domains. We also introduce ChainEval, a new metric for
automatic evaluation of both final answers and intermediate reasoning.
Benchmarking 30 LLMs on our dataset, we find that even state-of-the-art models
have considerable room for improvement in multi-step financial reasoning. All
templates and evaluation metrics for FinChain are available at https:
//github.com/mbzuai-nlp/finchain.
A new benchmark called FinChain evaluates multi-step symbolic reasoning in financial tasks with a focus on intermediate reasoning steps, introducing ChainEval as a metric for assessing both final answers and reasoning processes.
symbolic reasoning FinQA ConvFinQA Chain-of-Thought (CoT) FinChain multi-step financial reasoning parameterized templates reasoning complexity domain expertise executable Python trace ChainEval LLMs