Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education
<|reference_start|>Leveraging Large Language Models for Concept Graph Recovery and Question Answering in NLP Education: In the domain of Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated promise in text-generation tasks. However, their educational applications, particularly for domain-specific queries, remain underexplored. This study investigates LLMs' capabilities in educational scenarios, focusing on concept graph recovery and question-answering (QA). We assess LLMs' zero-shot performance in creating domain-specific concept graphs and introduce TutorQA, a new expert-verified NLP-focused benchmark for scientific graph reasoning and QA. TutorQA consists of five tasks with 500 QA pairs. To tackle TutorQA queries, we present CGLLM, a pipeline integrating concept graphs with LLMs for answering diverse questions. Our results indicate that LLMs' zero-shot concept graph recovery is competitive with supervised methods, showing an average 3% F1 score improvement. In TutorQA tasks, LLMs achieve up to 26% F1 score enhancement. Moreover, human evaluation and analysis show that CGLLM generates answers with more fine-grained concepts.<|reference_end|>
@article{yang2024leveraging,
    title={Leveraging Large Language Models for Concept Graph Recovery and Question
  Answering in NLP Education},
    author={Rui Yang, Boming Yang, Sixun Ouyang, Tianwei She, Aosong Feng, Yuang
  Jiang, Freddy Lecue, Jinghui Lu, Irene Li},
    journal={arXiv preprint arXiv:2402.14293},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.14293},
    primaryClass={cs.CL}
}