title: FinTral: A Family of GPT-4 Level Multimodal Financial Large Language
    Models
summary: We introduce FinTral, a suite of state-of-the-art multimodal large language
  models (LLMs) built upon the Mistral-7b model and tailored for financial
  analysis. FinTral integrates textual, numerical, tabular, and image data. We
  enhance FinTral with domain-specific pretraining, instruction fine-tuning, and
  RLAIF training by exploiting a large collection of textual and visual datasets
  we curate for this work. We also introduce an extensive benchmark featuring
  nine tasks and 25 datasets for evaluation, including hallucinations in the
  financial domain. Our FinTral model trained with direct preference optimization
  employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R,
  demonstrates an exceptional zero-shot performance. It outperforms ChatGPT-3.5
  in all tasks and surpasses GPT-4 in five out of nine tasks, marking a
  significant advancement in AI-driven financial technology. We also demonstrate
  that FinTral has the potential to excel in real-time analysis and
  decision-making in diverse financial contexts.
paper_authors._id: 65d425d4345d14c0c4dfa5d7
paper_authors.name: Gagan Bhatia
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-20T08:38:46.970Z
paper_authors._id: 60394599033b61166496163b
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1614366097007-noauth.jpeg
paper_authors.fullname: Gagan Bhatia
paper_authors.type: user
paper_authors.user: gagan3012
paper_authors._id: 65d425d4345d14c0c4dfa5d8
paper_authors.name: El Moatez Billah Nagoudi
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-20T11:02:55.214Z
paper_authors._id: 5fed19cf1dd5c1d397248f41
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1625528181158-5fed19cf1dd5c1d397248f41.png
paper_authors.fullname: Nagoudi ElMoatez Billah
paper_authors.type: user
paper_authors.user: m-nagoudi
paper_authors._id: 65d425d4345d14c0c4dfa5d9
paper_authors.name: Hasan Cavusoglu
paper_authors._id: 65d425d4345d14c0c4dfa5da
paper_authors.name: Muhammad Abdul-Mageed
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-20T08:38:44.909Z
paper_authors._id: 5feeae8dd8e0b2d24ed258cc
paper_authors.avatarUrl: /avatars/1d66840db4ed6dc12bdee7ba8168ea9f.svg
paper_authors.fullname: Muhammad Abdul-Mageed
paper_authors.type: user
paper_authors.user: mageed
paper_title: FinTral: A Family of GPT-4 Level Multimodal Financial Large Language
    Models
paper_summary: We introduce FinTral, a suite of state-of-the-art multimodal large language
  models (LLMs) built upon the Mistral-7b model and tailored for financial
  analysis. FinTral integrates textual, numerical, tabular, and image data. We
  enhance FinTral with domain-specific pretraining, instruction fine-tuning, and
  RLAIF training by exploiting a large collection of textual and visual datasets
  we curate for this work. We also introduce an extensive benchmark featuring
  nine tasks and 25 datasets for evaluation, including hallucinations in the
  financial domain. Our FinTral model trained with direct preference optimization
  employing advanced Tools and Retrieval methods, dubbed FinTral-DPO-T&R,
  demonstrates an exceptional zero-shot performance. It outperforms ChatGPT-3.5
  in all tasks and surpasses GPT-4 in five out of nine tasks, marking a
  significant advancement in AI-driven financial technology. We also demonstrate
  that FinTral has the potential to excel in real-time analysis and
  decision-making in diverse financial contexts.
paper_ai_summary: FinTral, a multimodal LLM enhanced through domain-specific pretraining, instruction fine-tuning, and RLAIF, outperforms ChatGPT-3.5 and GPT-4 in financial analysis tasks with exceptional zero-shot performance.
paper_ai_keywords: multimodal large language models LLMs Mistral-7b domain-specific pretraining instruction fine-tuning RLAIF training textual datasets visual datasets benchmark hallucinations direct preference optimization zero-shot performance real-time analysis decision-making