Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation
<|reference_start|>Enhancing Computer Programming Education with LLMs: A Study on Effective Prompt Engineering for Python Code Generation: Large language models (LLMs) and prompt engineering hold significant potential for advancing computer programming education through personalized instruction. This paper explores this potential by investigating three critical research questions: the systematic categorization of prompt engineering strategies tailored to diverse educational needs, the empowerment of LLMs to solve complex problems beyond their inherent capabilities, and the establishment of a robust framework for evaluating and implementing these strategies. Our methodology involves categorizing programming questions based on educational requirements, applying various prompt engineering strategies, and assessing the effectiveness of LLM-generated responses. Experiments with GPT-4, GPT-4o, Llama3-8b, and Mixtral-8x7b models on datasets such as LeetCode and USACO reveal that GPT-4o consistently outperforms others, particularly with the "multi-step" prompt strategy. The results show that tailored prompt strategies significantly enhance LLM performance, with specific strategies recommended for foundational learning, competition preparation, and advanced problem-solving. This study underscores the crucial role of prompt engineering in maximizing the educational benefits of LLMs. By systematically categorizing and testing these strategies, we provide a comprehensive framework for both educators and students to optimize LLM-based learning experiences. Future research should focus on refining these strategies and addressing current LLM limitations to further enhance educational outcomes in computer programming instruction.<|reference_end|>
@article{wang2024enhancing,
    title={Enhancing Computer Programming Education with LLMs: A Study on Effective
  Prompt Engineering for Python Code Generation},
    author={Tianyu Wang, Nianjun Zhou, Zhixiong Chen},
    journal={arXiv preprint arXiv:2407.05437},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.05437},
    primaryClass={cs.AI}
}