TL;DR
Large language models excel at finding statistical regularities in the world, but it is the surprises beyond those regularities that define understanding, creativity and human relationships — and that is precisely what AI can never learn.
The author comes right out and says it: statistical extrapolation is fine. Measuring what happened in the past, finding the correlations, and using them to predict what comes next is a reliable method, and it genuinely helps people understand the world. The catch is that working for some things does not mean it works for everything.
How good a word-guessing program can get
What is most surprising about large language models is how far they have pushed the single act of guessing the next word. Before LLMs, almost everyone overestimated how random ordinary sentences are. People assumed the world was textured and rough; it turned out that natural phenomena — including actions people think of as free will — are distributed far more smoothly than expected. Calling an LLM a word-guessing program is not a dismissal; it is an acknowledgment that the guessing has exceeded every expectation.
Put a smooth thing under a microscope
Glass, stainless steel, ice, polished wood — all smooth to the naked eye. Under high magnification, all of them are a world of tiny irregularities. Most of the time you can treat them as smooth, but the roughness is what matters: it is where the glass cracks, where the ice starts to melt, where the wood begins to warp. Forget that, and you will only know how the thing works — and be completely lost when it fails.
The surprise is everything
The author tells a story about a radio host: he claimed that since he could predict what his wife would say next, and the AI-powered autocomplete on her phone could predict it too, her phone understood her the way he did. The author calls the idea repellent and feels sorry for the wife. The real problem is that when she suddenly says something like 'I want a divorce', someone armed only with a statistical lookup table cannot cope. What is needed is an actual theory of how this person feels and why it might change. Surprises are everything: Dylan going electric, Miles Davis choosing not to play a note, Picasso's cubism and Kahlo's mustache.
Put another way, an LLM is trained on what people already know. And anything already known is not a surprise.
Curated from high-quality sources, with concise summaries and key takeaways.
