Prompt Refinement or Fine-tuning? Best Practices for using LLMs in Computational Social Science Tasks
<|reference_start|>Prompt Refinement or Fine-tuning? Best Practices for using LLMs in Computational Social Science Tasks: Large Language Models are expressive tools that enable complex tasks of text understanding within Computational Social Science. Their versatility, while beneficial, poses a barrier for establishing standardized best practices within the field. To bring clarity on the values of different strategies, we present an overview of the performance of modern LLM-based classification methods on a benchmark of 23 social knowledge tasks. Our results point to three best practices: select models with larger vocabulary and pre-training corpora; avoid simple zero-shot in favor of AI-enhanced prompting; fine-tune on task-specific data, and consider more complex forms instruction-tuning on multiple datasets only when only training data is more abundant.<|reference_end|>
@article{møller2024prompt,
    title={Prompt Refinement or Fine-tuning? Best Practices for using LLMs in
  Computational Social Science Tasks},
    author={Anders Giovanni M{o}ller, Luca Maria Aiello},
    journal={arXiv preprint arXiv:2408.01346},
    year={2024},
    archivePrefix={arXiv},
    eprint={2408.01346},
    primaryClass={cs.CY cs.CL physics.soc-ph}
}