How people learn to program9 min read

“It's Weird That it Knows What I Want”: Usability and Interactions with Copilot for Novice Programmers

James Prather, Brent N. Reeves, Paul Denny, Brett A. Becker, Juho Leinonen, Andrew Luxton-Reilly, et al. · 2023 · ACM Transactions on Computer-Human Interaction 31(1), Article 4, 1–31

Nineteen beginners were watched using Copilot on a real assignment; the problem was rarely that the code was wrong — it was that they could not tell.

Read the paper ↗doi:10.1145/3617367 ↗

The short version

→Nineteen students in an introductory C++ course each spent an observed session solving a real Minesweeper-style assignment with GitHub Copilot, thinking aloud, followed by interviews.
→Two interaction patterns get named: shepherding — spending your effort steering the tool toward the code you already had in mind — and drifting — accepting, adapting, deleting and re-accepting with no plan underneath.
→The recurring difficulty is metacognitive. Students could not judge whether a suggestion was correct, several could not say what the code they had just accepted did, and some expected Copilot to tell them whether their solution was right.
→Suggestions were also experienced as an interruption: constant, too long, and arriving mid-thought.

Subscription

The Library

Everything running a software business asks of you and none of it is code: onboarding that activates, churn you can actually see coming, marketing with no budget, SEO that finds buyers, and the AI-plus-Ahrefs workflow for writing the blog that feeds it. One new playbook a week, plus the whole archive from day one.

$9/ month · cancel any time

✓A new playbook every week — onboarding, churn, growth, SEO, content
✓The full back catalogue the moment you join, not drip-fed
✓Vendor explainers: Stripe, Supabase, Resend, Vercel, the AI APIs
✓The research shelf — real papers, decoded, on learning and on LLMs
✓One email when a new one lands, off by default in one click
✓Cancel any time; you keep reading to the end of the month

first — it takes ten seconds.

Billed monthly through Whop's secure checkout · cancel from Settings any time. See all plans →