Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification
<|reference_start|>Retrieval-style In-Context Learning for Few-shot Hierarchical Text Classification: Hierarchical text classification (HTC) is an important task with broad applications, while few-shot HTC has gained increasing interest recently. While in-context learning (ICL) with large language models (LLMs) has achieved significant success in few-shot learning, it is not as effective for HTC because of the expansive hierarchical label sets and extremely-ambiguous labels. In this work, we introduce the first ICL-based framework with LLM for few-shot HTC. We exploit a retrieval database to identify relevant demonstrations, and an iterative policy to manage multi-layer hierarchical labels. Particularly, we equip the retrieval database with HTC label-aware representations for the input texts, which is achieved by continual training on a pretrained language model with masked language modeling (MLM), layer-wise classification (CLS, specifically for HTC), and a novel divergent contrastive learning (DCL, mainly for adjacent semantically-similar labels) objective. Experimental results on three benchmark datasets demonstrate superior performance of our method, and we can achieve state-of-the-art results in few-shot HTC.<|reference_end|>
@article{chen2024retrieval-style,
    title={Retrieval-style In-Context Learning for Few-shot Hierarchical Text
  Classification},
    author={Huiyao Chen, Yu Zhao, Zulong Chen, Mengjia Wang, Liangyue Li, Meishan
  Zhang, Min Zhang},
    journal={arXiv preprint arXiv:2406.17534},
    year={2024},
    archivePrefix={arXiv},
    eprint={2406.17534},
    primaryClass={cs.CL}
}