LlamBERT: Large-scale low-cost data annotation in NLP
<|reference_start|>LlamBERT: Large-scale low-cost data annotation in NLP: Large Language Models (LLMs), such as GPT-4 and Llama 2, show remarkable proficiency in a wide range of natural language processing (NLP) tasks. Despite their effectiveness, the high costs associated with their use pose a challenge. We present LlamBERT, a hybrid approach that leverages LLMs to annotate a small subset of large, unlabeled databases and uses the results for fine-tuning transformer encoders like BERT and RoBERTa. This strategy is evaluated on two diverse datasets: the IMDb review dataset and the UMLS Meta-Thesaurus. Our results indicate that the LlamBERT approach slightly compromises on accuracy while offering much greater cost-effectiveness.<|reference_end|>
@article{csanády2024llambert:,
    title={LlamBERT: Large-scale low-cost data annotation in NLP},
    author={B'alint Csan'ady, Lajos Muzsai, P'eter Vedres, Zolt'an N'adasdy,
  Andr'as Luk'acs},
    journal={arXiv preprint arXiv:2403.15938},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.15938},
    primaryClass={cs.CL cs.AI cs.LG}
}