Meta-Task Prompting Elicits Embeddings from Large Language Models
<|reference_start|>Meta-Task Prompting Elicits Embeddings from Large Language Models: We introduce a new unsupervised text embedding method, Meta-Task Prompting with Explicit One-Word Limitation (MetaEOL), for generating high-quality sentence embeddings from Large Language Models (LLMs) without the need for model fine-tuning. Leveraging meta-task prompting, MetaEOL guides LLMs to produce embeddings through a series of carefully designed prompts that address multiple representational aspects. Our comprehensive experiments demonstrate that embeddings averaged from various meta-tasks are versatile embeddings that yield competitive performance on Semantic Textual Similarity (STS) benchmarks and excel in downstream tasks, surpassing contrastive-trained models. Our findings suggest a new scaling law, offering a versatile and resource-efficient approach for embedding generation across diverse scenarios.<|reference_end|>
@article{lei2024meta-task,
    title={Meta-Task Prompting Elicits Embeddings from Large Language Models},
    author={Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, Andrew
  Yates},
    journal={arXiv preprint arXiv:2402.18458},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.18458},
    primaryClass={cs.CL}
}