  Large Language Models (LLMs) have shown the potential to revolutionize
natural language processing tasks in various domains, sparking great interest
in vertical-specific large models. However, unlike proprietary models such as
BloombergGPT and FinGPT, which have leveraged their unique data accumulations
to make strides in the finance domain, there hasn't not many similar large
language models in the Chinese legal domain to facilitate its digital
transformation.
  In this paper, we propose an open-source legal large language model named
ChatLaw. Due to the importance of data quality, we carefully designed a legal
domain fine-tuning dataset. Additionally, to overcome the problem of model
hallucinations in legal data screening during reference data retrieval, we
introduce a method that combines vector database retrieval with keyword
retrieval to effectively reduce the inaccuracy of relying solely on vector
database retrieval. Furthermore, we propose a self-attention method to enhance
the ability of large models to overcome errors present in reference data,
further optimizing the issue of model hallucinations at the model level and
improving the problem-solving capabilities of large models. We also
open-sourced our model and part of the data at
https://github.com/PKU-YuanGroup/ChatLaw.

Computation and Language