How LLMs and coding tools work9 min read

Attention Is All You Need

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin · 2017 · NeurIPS 2017 · Advances in Neural Information Processing Systems 30

The architecture every model you call is built on: throw away recurrence, keep attention, and the whole sequence can be processed at once.

Read the paper ↗

The short version

→Before this, sequence models read left to right, one step at a time. The Transformer reads the whole input in parallel and lets every position look at every other position directly.
→The mechanism is attention: for each position, score how relevant every other position is, then take a weighted average of their representations.
→Because attention has no built-in sense of order, position has to be added to the input explicitly — the model does not know that word 3 comes after word 2 unless you tell it.
→Every cost, limit and failure mode you meet in an AI API — the token, the context window, why long prompts cost quadratically more to attend over — starts here.

Subscription

The Library

Everything running a software business asks of you and none of it is code: onboarding that activates, churn you can actually see coming, marketing with no budget, SEO that finds buyers, and the AI-plus-Ahrefs workflow for writing the blog that feeds it. One new playbook a week, plus the whole archive from day one.

$9/ month · cancel any time

✓A new playbook every week — onboarding, churn, growth, SEO, content
✓The full back catalogue the moment you join, not drip-fed
✓Vendor explainers: Stripe, Supabase, Resend, Vercel, the AI APIs
✓The research shelf — real papers, decoded, on learning and on LLMs
✓One email when a new one lands, off by default in one click
✓Cancel any time; you keep reading to the end of the month

first — it takes ten seconds.

No card is asked for yet — the subscription is recorded and really opens your account. See all plans →