Can humans teach machines to code?
<|reference_start|>Can humans teach machines to code?: The goal of inductive program synthesis is for a machine to automatically generate a program from user-supplied examples of the desired behaviour of the program. A key underlying assumption is that humans can provide examples of sufficient quality to teach a concept to a machine. However, as far as we are aware, this assumption lacks both empirical and theoretical support. To address this limitation, we explore the question `Can humans teach machines to code?'. To answer this question, we conduct a study where we ask humans to generate examples for six programming tasks, such as finding the maximum element of a list. We compare the performance of a program synthesis system trained on (i) human-provided examples, (ii) randomly sampled examples, and (iii) expert-provided examples. Our results show that, on most of the tasks, non-expert participants did not provide sufficient examples for a program synthesis system to learn an accurate program. Our results also show that non-experts need to provide more examples than both randomly sampled and expert-provided examples.<|reference_end|>
@article{hocquette2024can,
    title={Can humans teach machines to code?},
    author={C'eline Hocquette and Johannes Langer and Andrew Cropper and Ute
  Schmid},
    journal={arXiv preprint arXiv:2404.19397},
    year={2024},
    archivePrefix={arXiv},
    eprint={2404.19397},
    primaryClass={cs.HC cs.LG}
}