Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4
  on mock CFA Exams
Large Language Models (LLMs) have demonstrated remarkable performance on a
wide range of Natural Language Processing (NLP) tasks, often matching or even
beating state-of-the-art task-specific models. This study aims at assessing the
financial reasoning capabilities of LLMs. We leverage mock exam questions of
the Chartered Financial Analyst (CFA) Program to conduct a comprehensive
evaluation of ChatGPT and GPT-4 in financial analysis, considering Zero-Shot
(ZS), Chain-of-Thought (CoT), and Few-Shot (FS) scenarios. We present an
in-depth analysis of the models' performance and limitations, and estimate
whether they would have a chance at passing the CFA exams. Finally, we outline
insights into potential strategies and improvements to enhance the
applicability of LLMs in finance. In this perspective, we hope this work paves
the way for future studies to continue enhancing LLMs for financial reasoning
through rigorous evaluation.
652de0de06fab2df13b1da09
Ethan Callanan
admin_assigned
2023-10-17T08:13:01.022Z
6303e6a47373aacccd870f18
/avatars/4b656ce06689b7d910d056e8ecb0702d.svg
Ethan Callanan
user
ecal
652de0de06fab2df13b1da0a
Amarachi Mbakwe
extracted_pending
2025-02-06T18:30:22.371Z
67a4ffa893e9cebec54f20c3
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/X2E_X8XPqjGelF58l_WVH.png
Amarachi Blessing Madu
user
Blessed4ever
652de0de06fab2df13b1da0b
Antony Papadimitriou
claimed_verified
2023-10-17T15:11:43.361Z
652e8ae7da91a2e197967de2
/avatars/181f05167d404cd151d532e2c9c0fe8b.svg
Antony Papadimitriou
user
antonypapa
652de0de06fab2df13b1da0c
Yulong Pei
652de0de06fab2df13b1da0d
Mathieu Sibue
claimed_verified
2023-10-17T15:11:39.875Z
652e446ad2937a53ed7f90e9
/avatars/b33e0af2c34a6430ab719e9e6d7d833b.svg
Mathieu Sibue
user
mathieu-sibue
652de0de06fab2df13b1da0e
Xiaodan Zhu
652de0de06fab2df13b1da0f
Zhiqiang Ma
admin_assigned
2023-10-17T08:13:43.757Z
64b702e8fa7eabaae5f0cbfb
/avatars/512690f60997ae81b79b999a634a1861.svg
Zhiqiang Ma
user
zqmaster
652de0de06fab2df13b1da10
Xiaomo Liu
admin_assigned
2023-10-17T08:13:56.162Z
6365131229941f95ad725910
/avatars/5e709cb8e0f1415aeea38e9904703b25.svg
liu
user
xiaomo
652de0de06fab2df13b1da11
Sameena Shah
Can GPT models be Financial Analysts? An Evaluation of ChatGPT and GPT-4
  on mock CFA Exams
Large Language Models (LLMs) have demonstrated remarkable performance on a
wide range of Natural Language Processing (NLP) tasks, often matching or even
beating state-of-the-art task-specific models. This study aims at assessing the
financial reasoning capabilities of LLMs. We leverage mock exam questions of
the Chartered Financial Analyst (CFA) Program to conduct a comprehensive
evaluation of ChatGPT and GPT-4 in financial analysis, considering Zero-Shot
(ZS), Chain-of-Thought (CoT), and Few-Shot (FS) scenarios. We present an
in-depth analysis of the models' performance and limitations, and estimate
whether they would have a chance at passing the CFA exams. Finally, we outline
insights into potential strategies and improvements to enhance the
applicability of LLMs in finance. In this perspective, we hope this work paves
the way for future studies to continue enhancing LLMs for financial reasoning
through rigorous evaluation.
LLMs, evaluated using CFA exam questions, show mixed performance in financial reasoning across different scenarios, highlighting areas for improvement and future research.
Large Language Models (LLMs) Natural Language Processing (NLP) Chartered Financial Analyst (CFA) Program ChatGPT GPT-4 Zero-Shot (ZS) Chain-of-Thought (CoT) Few-Shot (FS)