Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
<|reference_start|>Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models: Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of scaffolding on writing quality and productivity (words/time). While low scaffolding did not significantly improve writing quality or productivity, high scaffolding led to significant improvements, especially benefiting non-regular writers and less tech-savvy users. No significant cognitive burden was observed while using the scaffolded writing tools, but a moderate decrease in text ownership and satisfaction was noted. Our results have broad implications for the design of AI-powered writing tools, including the need for personalized scaffolding mechanisms.<|reference_end|>
@article{dhillon2024shaping,
    title={Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing
  with Language Models},
    author={Paramveer S. Dhillon, Somayeh Molaei, Jiaqi Li, Maximilian Golub,
  Shaochun Zheng, Lionel P. Robert},
    journal={arXiv preprint arXiv:2402.11723},
    year={2024},
    archivePrefix={arXiv},
    eprint={2402.11723},
    primaryClass={cs.HC cs.CL}
}