HyperTeamsWhat a language model actually does
What you will learn One layer deeper into "a program that guesses what word comes next" from what AI is. Knowing this lets you predict what it will be good at and what it will not.
It really only does one thing
What a language model does is literally this one thing:
Look at the text so far and pick the next fragment.
And then repeat.
You have probably seen an answer appear one character at a time. That is not a visual effect — it is genuinely being produced that way.
So how does that become summarising and translating?
This is the key part. If it only guesses the next word, how does it summarise?
You are not commanding it to summarise; you are creating a situation where a summary is the only thing that can come next.
In that state, what comes next? In human writing, what follows "Summary of the above:" is a summary. The model has seen that pattern countless times.
Translation is the same:
The natural continuation after "English:" is a translation.
So it is less "it understood the instruction" and more "it produced the natural continuation for that situation." The results are similar, but this distinction explains the limits.
What follows from this structure
1. What comes before is everything
The model picks the next fragment based only on the text it is looking at. So:
- Describe the situation in detail → you get a continuation fitted to it
- Write nothing → you get the most generic continuation
That is why getting the answer you wanted says to give the situation. It is not a trick — it follows directly from how it works.
2. The same question gives different answers
We said it "picks the next fragment," but there are several candidates and it picks probabilistically.
Always picking first place makes writing stilted, so a little randomness is built in. That is why asking the same question twice gives slightly different answers.
This is design, not malfunction. It also means it is unsuited to work requiring identical results (same input must always give the same output).
3. It is bad at arithmetic
Ask for 237 × 481 and it does not calculate — it produces a number that
looks like a plausible answer. That is where "right number of digits, wrong
value" comes from.
So in practice you make a calculator do the calculating. Rather than the model doing it, you have it call a tool — the subject of the next chapters.
What it is good and bad at
Knowing the structure makes this predictable.
| Good at | Why |
|---|---|
| Summarising, translating, reformatting | patterns common in human writing |
| Drafting | rich patterns for "writing in this situation" |
| Classification and tagging | give a few examples and it continues the pattern |
| Smoothing awkward sentences | producing natural continuations is its actual job |
| Bad at | Why |
|---|---|
| Exact arithmetic | it generates plausible numbers, it does not calculate |
| Current facts | it does not know past its training cutoff |
| Your company's internal information | it has never seen it |
| Identical output every time | it picks probabilistically |
All four in the second table are solved with "tools." Attach a calculator, attach search, attach the internal database. That is tool use.
Common misconceptions
"The model understands my question"
The word "understand" causes confusion. From the results it looks like understanding; in reality it produced the most natural continuation in that context.
This distinction matters in practice because natural and correct are not the same thing. A plausible but non-existent book title is the example — the subject of the next chapter.
"Won't a bigger model solve everything?"
Size increases the richness of patterns. But it does not make the model know facts it never saw. No model, however large, knows your company's revenue. That is not a size problem but a connection problem.
Check yourself
1. How does "summarise this" lead to a summary?
Answer
Because the natural continuation after "Summary of the above:" is a summary. The model has seen that pattern countless times in human writing. It is less understanding and executing an instruction, and more producing what naturally comes next in that situation.
2. Why does the same question give slightly different answers each time?
Answer
Because the next fragment is picked probabilistically. Always picking first place makes writing stilted, so some randomness is built in. That is design, not malfunction — but it is unsuited to work that needs identical output for identical input.
3. What is the common fix for the four things models are bad at?
Answer
Attaching tools. Arithmetic to a calculator, current information to search, internal information to a database. Making the model bigger does not fix it.
Next, the problem that follows inevitably from this structure → Why it is wrong so plausibly