AI Boosted Homework Scores by 18%—But Exam Performance Crashed by 24%. Here's Why It Took Two Years to Show Up

Economists David Strömberg, Victor Lei, and Yanhui Wu (CEPR Discussion Paper DP21577) analyzed panel data from students in grades 7–12 in a Chinese county of over one million residents. The dataset spanned monthly closed-book exams, homework scores and completion times, and high-stakes entrance exams across nine subjects. Self-reported AI usage rose from near zero to about 80 percent over the study period, with a major spike coinciding with the releases of DeepSeek V2.5 and DeepSeek R1. The most popular tools were Doubao, DeepSeek, ChatGLM, Ernie Bot, and Qwen. Using a difference-in-differences design, the researchers compared students before and after they started using AI against classmates who had not yet adopted it—isolating a causal effect rather than a mere correlation.

Within six months of first using AI, homework scores rose by 18 percent and average completion time fell from 64 to 45 minutes. At the same time, scores on monthly closed-book exams dropped by 20 percent. High-stakes entrance exams—the Zhongkao (high school) and Gaokao (college)—saw declines of 18 to 24 percent, but the full magnitude of that damage took approximately two years to fully emerge. This delay is a critical finding: short-term studies of AI in education systematically underestimate the harm because they simply do not wait long enough. Regular exam losses hit full force within half a year; entrance exam losses compound slowly and invisibly.

About 81 percent of students who used AI for more than five months finished homework faster than even the quickest non-AI users—the hallmark of outsourcing. These students earned high homework grades but bombed closed-book exams. By contrast, AI users who spent the same amount of time on homework as non-users suffered minimal learning losses while still earning better homework grades, suggesting that AI used as a thinking aid rather than a thinking replacement can be neutral or even helpful.

The learning losses were sharpest in social science subjects (politics, geography: −27%), followed by STEM (−22%), English (−17%), and Chinese (−9%)—a finding that challenges prior research focused almost exclusively on math and programming. Top-performing students suffered the most: the top third saw a −24% effect versus −16% for the bottom third. Younger students and boys were also disproportionately affected. A clear dose-response pattern emerged: students using AI up to one hour per week lost about 5 percent; those using it five or more hours per week lost 30 percent.

The study offers a sobering explanation for why schools and policymakers have been slow to react. Teachers typically observe students in only one subject, so a 20-percent grade drop is not unusual on its own and raises no alarms. The aggregate county-level effect didn't even reach −10 percent until June 2025, because few students had been using AI long enough for losses to fully accumulate. Students themselves often mistake the mental struggle of independent learning for a sign they are performing poorly—making AI's relief feel like an improvement rather than a crutch.

The researchers recommend three concrete countermeasures: give students credible information about the long-term costs of AI outsourcing; put greater weight on in-person, closed-book exams; and track homework completion time alongside grades, since AI erodes homework's value as a signal of actual learning. World Bank economist Gabriel Demombynes, who highlighted the study, notes that the scale of uncontrolled AI adoption in the wild may dwarf the learning gains from carefully designed AI tutoring interventions. Anthropic researcher Andrej Karpathy has argued schools should stop policing AI-generated homework and instead shift most grading to in-class work—a recommendation that aligns closely with the study's findings.