The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging
<|reference_start|>The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging: This paper proposes a novel method for constructing instruction-tuned large language models (LLMs) for finance without instruction data. Traditionally, developing such domain-specific LLMs has been resource-intensive, requiring a large dataset and significant computational power for continual pretraining and instruction tuning. Our study proposes a simpler approach that combines domain-specific continual pretraining with model merging. Given that general-purpose pretrained LLMs and their instruction-tuned LLMs are often publicly available, they can be leveraged to obtain the necessary instruction task vector. By merging this with a domain-specific pretrained vector, we can effectively create instruction-tuned LLMs for finance without additional instruction data. Our process involves two steps: first, we perform continual pretraining on financial data; second, we merge the instruction-tuned vector with the domain-specific pretrained vector. Our experiments demonstrate the successful construction of instruction-tuned LLMs for finance. One major advantage of our method is that the instruction-tuned and domain-specific pretrained vectors are nearly independent. This independence makes our approach highly effective. The Japanese financial instruction-tuned LLMs we developed in this study are available at https://huggingface.co/pfnet/nekomata-14b-pfn-qfin-inst-merge.<|reference_end|>
@article{hirano2024the,
    title={The Construction of Instruction-tuned LLMs for Finance without
  Instruction Data Using Continual Pretraining and Model Merging},
    author={Masanori Hirano, Kentaro Imajo},
    journal={arXiv preprint arXiv:2409.19854},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.19854},
    primaryClass={cs.CL econ.GN q-fin.CP q-fin.EC}
}