Generative AI, particularly Language Models (LMs), has the potential to transform real-world domains with societal impact, particularly where access to experts is limited. For example, in education, training novice educators with expert guidance is important for effectiveness but expensive, creating significant barriers to improving education quality at scale. This challenge disproportionately harms students from under-served communities, who stand to gain the most from high-quality education. We introduce Tutor CoPilot, a novel Human-AI approach that leverages a model of expert thinking to provide expert-like guidance to tutors as they tutor. This study is the first randomized controlled trial of a Human-AI system in live tutoring, involving 900 tutors and 1,800 K-12 students from historically under-served communities. Following a preregistered analysis plan, we find that students working with tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more likely to master topics (p<0.01). Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by 9 p.p. We find that Tutor CoPilot costs only 20per−tutorannually.Weanalyze550,000+messagesusingclassifierstoidentifypedagogicalstrategies,andfindthattutorswithaccesstoTutorCoPilotaremorelikelytousehigh−qualitystrategiestofosterstudentunderstanding(e.g.,askingguidingquestions)andlesslikelytogiveawaytheanswertothestudent.TutorinterviewshighlighthowTutorCoPilot′sguidancehelpstutorstorespondtostudentneeds,thoughtheyflagissuesinTutorCoPilot,suchasgeneratingsuggestionsthatarenotgrade−levelappropriate.Altogether,ourstudyofTutorCoPilotdemonstrateshowHuman−AIsystemscanscaleexpertiseinreal−worlddomains,bridgegapsinskillsandcreateafuturewherehigh−qualityeducationisaccessibletoallstudents.
