Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs
<|reference_start|>Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with LLMs: Detecting anomalies in general ledger data is of utmost importance to ensure trustworthiness of financial records. Financial audits increasingly rely on machine learning (ML) algorithms to identify irregular or potentially fraudulent journal entries, each characterized by a varying number of transactions. In machine learning, heterogeneity in feature dimensions adds significant complexity to data analysis. In this paper, we introduce a novel approach to anomaly detection in financial data using Large Language Models (LLMs) embeddings. To encode non-semantic categorical data from real-world financial records, we tested 3 pre-trained general purpose sentence-transformer models. For the downstream classification task, we implemented and evaluated 5 optimized ML models including Logistic Regression, Random Forest, Gradient Boosting Machines, Support Vector Machines, and Neural Networks. Our experiments demonstrate that LLMs contribute valuable information to anomaly detection as our models outperform the baselines, in selected settings even by a large margin. The findings further underscore the effectiveness of LLMs in enhancing anomaly detection in financial journal entries, particularly by tackling feature sparsity. We discuss a promising perspective on using LLM embeddings for non-semantic data in the financial context and beyond.<|reference_end|>
@article{bakumenko2024advancing,
    title={Advancing Anomaly Detection: Non-Semantic Financial Data Encoding with
  LLMs},
    author={Alexander Bakumenko (1), Katev{r}ina Hlav'av{c}kov'a-Schindler
  (2), Claudia Plant (2), and Nina C. Hubig (1) ((1) Clemson University, USA,
  (2) University of Vienna, Austria)},
    journal={arXiv preprint arXiv:2406.03614},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.03614},
    primaryClass={cs.LG cs.CL q-fin.RM}
}