title: Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
summary: Generative AI, particularly Language Models (LMs), has the potential to
  transform real-world domains with societal impact, particularly where access to
  experts is limited. For example, in education, training novice educators with
  expert guidance is important for effectiveness but expensive, creating
  significant barriers to improving education quality at scale. This challenge
  disproportionately harms students from under-served communities, who stand to
  gain the most from high-quality education. We introduce Tutor CoPilot, a novel
  Human-AI approach that leverages a model of expert thinking to provide
  expert-like guidance to tutors as they tutor. This study is the first
  randomized controlled trial of a Human-AI system in live tutoring, involving
  900 tutors and 1,800 K-12 students from historically under-served communities.
  Following a preregistered analysis plan, we find that students working with
  tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more
  likely to master topics (p<0.01). Notably, students of lower-rated tutors
  experienced the greatest benefit, improving mastery by 9 p.p. We find that
  Tutor CoPilot costs only $20 per-tutor annually. We analyze 550,000+ messages
  using classifiers to identify pedagogical strategies, and find that tutors with
  access to Tutor CoPilot are more likely to use high-quality strategies to
  foster student understanding (e.g., asking guiding questions) and less likely
  to give away the answer to the student. Tutor interviews highlight how Tutor
  CoPilot's guidance helps tutors to respond to student needs, though they flag
  issues in Tutor CoPilot, such as generating suggestions that are not
  grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates
  how Human-AI systems can scale expertise in real-world domains, bridge gaps in
  skills and create a future where high-quality education is accessible to all
  students.
paper_authors._id: 6703648177f7e874a2a560be
paper_authors.name: Rose E. Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-07T07:26:30.010Z
paper_authors._id: 651e391852cd39f995249099
paper_authors.avatarUrl: /avatars/59d295863c13f90cb345d5cac303c266.svg
paper_authors.fullname: Rose E Wang
paper_authors.type: user
paper_authors.user: rose-e-wang
paper_authors._id: 6703648177f7e874a2a560bf
paper_authors.name: Ana T. Ribeiro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T08:56:43.863Z
paper_authors._id: 639234b5c1899a08e56a7d2a
paper_authors.avatarUrl: /avatars/34829332a4b10cafab77e396e3c57752.svg
paper_authors.fullname: Ana Ribeiro
paper_authors.type: user
paper_authors.user: Analu
paper_authors._id: 6703648177f7e874a2a560c0
paper_authors.name: Carly D. Robinson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T08:56:52.843Z
paper_authors._id: 660c6f8b0c47aba6600c247e
paper_authors.avatarUrl: /avatars/7f79e19f6c8186cdb40785140cd42578.svg
paper_authors.fullname: Carly Robinson
paper_authors.type: user
paper_authors.user: carlycodes
paper_authors._id: 6703648177f7e874a2a560c1
paper_authors.name: Susanna Loeb
paper_authors._id: 6703648177f7e874a2a560c2
paper_authors.name: Dora Demszky
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-07T08:57:02.827Z
paper_authors._id: 606e3a4c7f3150d181025dd1
paper_authors.avatarUrl: /avatars/224ee859c1df3f6c4ab59910f2a4732b.svg
paper_authors.fullname: Dora Demszky
paper_authors.type: user
paper_authors.user: ddemszky
paper_title: Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
paper_summary: Generative AI, particularly Language Models (LMs), has the potential to
  transform real-world domains with societal impact, particularly where access to
  experts is limited. For example, in education, training novice educators with
  expert guidance is important for effectiveness but expensive, creating
  significant barriers to improving education quality at scale. This challenge
  disproportionately harms students from under-served communities, who stand to
  gain the most from high-quality education. We introduce Tutor CoPilot, a novel
  Human-AI approach that leverages a model of expert thinking to provide
  expert-like guidance to tutors as they tutor. This study is the first
  randomized controlled trial of a Human-AI system in live tutoring, involving
  900 tutors and 1,800 K-12 students from historically under-served communities.
  Following a preregistered analysis plan, we find that students working with
  tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more
  likely to master topics (p<0.01). Notably, students of lower-rated tutors
  experienced the greatest benefit, improving mastery by 9 p.p. We find that
  Tutor CoPilot costs only $20 per-tutor annually. We analyze 550,000+ messages
  using classifiers to identify pedagogical strategies, and find that tutors with
  access to Tutor CoPilot are more likely to use high-quality strategies to
  foster student understanding (e.g., asking guiding questions) and less likely
  to give away the answer to the student. Tutor interviews highlight how Tutor
  CoPilot's guidance helps tutors to respond to student needs, though they flag
  issues in Tutor CoPilot, such as generating suggestions that are not
  grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates
  how Human-AI systems can scale expertise in real-world domains, bridge gaps in
  skills and create a future where high-quality education is accessible to all
  students.
paper_ai_summary: Tutor CoPilot, a Human-AI system, enhances tutoring effectiveness by providing expert-like guidance, resulting in better student mastery and improved pedagogical strategies.