What's actually happening

When you type a question, it doesn't look up an answer in a database. It works out, one tiny piece at a time, which word is most likely to come next — based on patterns it learned from an enormous amount of writing.

String enough good guesses together and you get a paragraph that sounds like it was written by someone who knows exactly what they're talking about.

The strange part: “predict the next word” sounds far too simple to produce something that can write code, explain physics, or argue with you. But past a certain scale, getting genuinely good at prediction seems to require something that looks a lot like understanding. Whether it really is understanding is one of the biggest open questions in the field — we come back to it in Chapter 7.

“Training” means roughly this: the model makes a guess, checks how wrong it was, and nudges billions of tiny internal dials a fraction to do better next time. Repeat that an astronomical number of times and the dials settle into patterns that capture how language — and a surprising amount of the world — actually works.

Those dials have a name: parameters, or weights. A big model has hundreds of billions of them, and nobody sets them by hand — the training process does.

• Tokens — it doesn't read letters or whole words — it reads “tokens,” little chunks (often part of a word). Everything it costs, remembers, and processes is measured in tokens.
• The loop — data in → guess → measure how wrong it was (the “loss”) → nudge the weights a hair in the direction that would’ve reduced it (“gradient descent”) → repeat, across batch after batch of examples. No one writes the rules of grammar or fact; the model discovers them.

Nothing inside it fact-checks. If it doesn't “know” (it doesn't really know anything), it predicts the most plausible-sounding answer and delivers it with total confidence.

It's most likely to be wrong about things that are recent, obscure, very specific, or involve exact numbers.

Here's the uncomfortable part: hallucination isn't a simple bug someone forgot to fix. It's the flip side of the very prediction engine that makes the model useful. You can reduce it — mostly by giving the model real sources to work from (“grounding”) — but you can't fully delete it without breaking what makes it powerful. So don't wait for a version you can fully trust — it isn't coming. Check the things that matter.

These are called reasoning models, and the extra thinking time genuinely helps — mostly with maths, logic, code, and planning, the jobs where the first guess is usually a bad one. The cost: they're slower, and they're more expensive to run.

Two things worth knowing. The app usually shows you a tidied summary of the scratchpad, not the whole messy thing. And when a dropdown offers you a “fast” model or a “smart” one, this is largely what you're choosing between — answer now, or think first.

Here's the twist: the “thinking” is generated text too. It comes out of the same prediction engine as everything else, which means it can be wrong — and, stranger, it can be beside the point. Researchers have caught models writing out a tidy chain of reasoning for an answer they'd have given anyway. The working looked like the reason. It wasn't. So visible reasoning is not a guarantee of logic — it's the model showing you something that reads like its work, which is not quite the same as showing its work. The rule doesn't change: if the answer matters, check it.

• Context window — the model's own working memory is just what's in the current conversation. When a long chat starts forgetting what you said way earlier, that's this window filling up. (Some apps add a separate long-term memory that carries a few facts across chats — handy, but it's a feature layered on, not the model itself.)
• Knowledge cutoff — it only learned from data up to a certain date. Ask about something newer and it's guessing — unless it can search the web in the moment.

Hold that picture and most things click into place: why clear instructions help (you're briefing the intern), why you double-check its facts (it won't admit doubt), and why your judgment is the thing that turns its output into something actually good.