Catching it lying

We’re wired to trust confidence. When a person speaks clearly and without hesitation, we assume they know their stuff. The model triggers that same instinct — but it has no idea whether it’s right. It’s optimised to sound plausible, and plausible is not the same as true.

So the polish is exactly what you can’t rely on. A wrong answer and a right answer look identical.

Why no wobble? Because nothing inside it measures its own truthfulness. It can sometimes report a vague “confidence,” but that number reflects how strongly the patterns it learned point to those particular words — not whether the fact behind them is actually real. Sometimes that lines up with the truth. Often enough, it doesn’t — and the number won’t tell you which time is which. There’s active research into getting models to say “I don’t know” more honestly — it’s genuinely hard, partly because for years we trained them, with thumbs-up and thumbs-down, that a confident answer pleases people more than an honest shrug. We rewarded the bluffing.

• Recent things — events after its training cut-off, or anything that changed lately. Its knowledge has a date stamp, and the world kept moving.
• Obscure things — niche topics, small towns, lesser-known people. The fewer times something appeared in its training, the shakier it gets — and the more likely it is to invent.
• Very specific things — exact quotes, dates, page numbers, names, web links. It’ll happily generate a real-looking source that doesn’t exist.
• Numbers and maths — it predicts what a calculation looks like rather than actually computing it, so arithmetic and statistics are a classic weak spot.

Notice the pattern: it’s strongest on the common and the general, weakest on the rare and the exact. That’s prediction showing its hand — it’s averaging over patterns it saw a lot, so well-trodden ground is solid and the edges crumble. The cruel twist is that the edges are often exactly what you came for: the one specific fact, the precise figure, the real citation.

• Triangulate — check the claim against a second, independent source you trust. If one real source agrees, you’re probably fine. If you can’t find any, be suspicious.
• Ask for sources, then open them — request where a claim comes from — but click through. Sometimes the source is real and says something different; sometimes the source is invented outright.
• Test it on what you already know — ask it about a topic you know well. Watch how it handles the bits you can check. That tells you how much to trust it on the bits you can’t.
• Notice the danger zones — if the answer is recent, obscure, specific, or numerical, raise your guard automatically.

The deeper habit is keeping the burden of proof on the machine, not on yourself. The natural pull is to believe it unless something feels off — but feelings are exactly what fluent writing is built to soothe. Flip it: for anything that matters, assume “unverified” until you’ve seen it confirmed somewhere that isn’t the chatbot. Treat its answer as a first draft. The checking is what turns it into something you can actually use.

If most of the text it read assumed a “nurse” is a woman and an “engineer” is a man, it will lean that way too, quietly, even when nobody asked. It’s not just jobs. It also leans toward the stories, languages, and points of view that showed up most in what it read — and quietly treats those as the normal ones.

It isn’t plotting anything. It’s just reflecting the average of everything it was fed — which means it can dress up old, unfair ideas in clean, neutral-sounding language.

• Skewed sources — more of its training is English, online, and recent — so other languages, places, and eras are under-represented.
• The default-person problem — ask for “a person” or “a CEO” and watch which assumptions show up uninvited.
• Confident averages — it can state a contested or one-sided view as if it were settled fact.

Companies try to correct this after the fact, with extra training and rules about what the model should and shouldn’t say. That helps, but it’s a second layer of human judgment stacked on top — which raises a sharper question: whose idea of “fair” and “balanced” got encoded? Nobody gets to stand outside their own point of view — not the training data, not the tuners, not you. There’s no perfectly unbiased AI to find. What you can do is remember that every answer arrives with a slant, from the data underneath and from the people who shaped it.

Everything in these chapters points one way: the model is a fast, fluent, occasionally brilliant, occasionally wrong assistant — and you are the editor. You set the task, supply the context, catch the mistakes, and make the call about what to actually use.

Get comfortable in that editor’s seat. The more these tools can do, the more it matters that someone with judgment is sitting in it.

Under all of this sits an old, serious question: how do you actually know something is true? Not “it sounds right,” not “a confident voice said so,” not “most sources online agree” — but the harder work of evidence, reasoning, and admitting when you’re not sure. Philosophers call this epistemics, and AI has suddenly made it matter outside the classroom. A machine that produces unlimited fluent, confident, sometimes-false text raises the value of one stubborn human habit: the willingness to ask “how do I know this?” and not look away from the answer. Keep asking that question and no chatbot ever gets to do your believing for you.