Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes
<|reference_start|>Efficient Few-shot Learning for Multi-label Classification of Scientific Documents with Many Classes: Scientific document classification is a critical task and often involves many classes. However, collecting human-labeled data for many classes is expensive and usually leads to label-scarce scenarios. Moreover, recent work has shown that sentence embedding model fine-tuning for few-shot classification is efficient, robust, and effective. In this work, we propose FusionSent (Fusion-based Sentence Embedding Fine-tuning), an efficient and prompt-free approach for few-shot classification of scientific documents with many classes. FusionSent uses available training examples and their respective label texts to contrastively fine-tune two different sentence embedding models. Afterward, the parameters of both fine-tuned models are fused to combine the complementary knowledge from the separate fine-tuning steps into a single model. Finally, the resulting sentence embedding model is frozen to embed the training instances, which are then used as input features to train a classification head. Our experiments show that FusionSent significantly outperforms strong baselines by an average of $6.0$ $F_{1}$ points across multiple scientific document classification datasets. In addition, we introduce a new dataset for multi-label classification of scientific documents, which contains 183,565 scientific articles and 130 classes from the arXiv category taxonomy. Code and data are available at https://github.com/sebischair/FusionSent.<|reference_end|>
@article{schopf2024efficient,
    title={Efficient Few-shot Learning for Multi-label Classification of Scientific
  Documents with Many Classes},
    author={Tim Schopf, Alexander Blatzheim, Nektarios Machner, Florian Matthes},
    journal={arXiv preprint arXiv:2410.05770},
    year={2024},
    archivePrefix={arXiv},
    eprint={2410.05770},
    primaryClass={cs.CL}
}