Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions
<|reference_start|>Using Pretrained Large Language Model with Prompt Engineering to Answer Biomedical Questions: Our team participated in the BioASQ 2024 Task12b and Synergy tasks to build a system that can answer biomedical questions by retrieving relevant articles and snippets from the PubMed database and generating exact and ideal answers. We propose a two-level information retrieval and question-answering system based on pre-trained large language models (LLM), focused on LLM prompt engineering and response post-processing. We construct prompts with in-context few-shot examples and utilize post-processing techniques like resampling and malformed response detection. We compare the performance of various pre-trained LLM models on this challenge, including Mixtral, OpenAI GPT and Llama2. Our best-performing system achieved 0.14 MAP score on document retrieval, 0.05 MAP score on snippet retrieval, 0.96 F1 score for yes/no questions, 0.38 MRR score for factoid questions and 0.50 F1 score for list questions in Task 12b.<|reference_end|>
@article{zhou2024using,
    title={Using Pretrained Large Language Model with Prompt Engineering to Answer
  Biomedical Questions},
    author={Wenxin Zhou and Thuy Hang Ngo},
    journal={arXiv preprint arXiv:2407.06779},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.06779},
    primaryClass={cs.CL}
}