HyperTeamsThe context window
What you will learn The real structure behind what feels like the model "remembering." Knowing it explains why, in long jobs, it starts acting as though it forgot your earlier instructions.
It starts from a blank page every time
First, a surprising fact.
The model remembers nothing.
Conversations appear to continue because the whole prior conversation is fed back in every time.
That is why "tell me more about the third one" worked. Nothing was remembered — the earlier conversation was inside that request.
Context = everything you just fed in
That "everything you just fed in" is called the context. It is not only the conversation.
graph TD
A["System instructions<br/>role · rules · tone"] --> E["Context"]
B["The whole prior conversation"] --> E
C["Pasted material<br/>documents · tables · code"] --> E
D["Tool results"] --> E
F["This request"] --> E
E --> G["Model"]Tool results go into the context too. This matters later — a large lookup result takes up correspondingly large space.
The window has a size
A context window has a fixed maximum size. It varies by model, but the key point is that whatever the size, it is finite.
The unit is the token. Roughly:
| Approximately | |
|---|---|
| One English word | 1–2 tokens |
| One page of A4 | 1,000–1,500 tokens |
You do not need to know this precisely. A sense of "long documents take up a lot of space" is enough.
Why this matters
Three practical consequences follow.
1. What you put in costs money
Cost is generally calculated from the volume of tokens exchanged. As a conversation grows, the whole of it is resent with every request, so each request gets more expensive the further in you are.
The same question costs more at the end of a long conversation.
2. When the window fills, things get pushed out
At the limit, the oldest goes first. That is the next chapter.
3. More is not better
"Surely more material is better" — sometimes the opposite. With a lot of irrelevant material, the important part gets buried. People also find it harder to locate the key point in a 100-page pack.
Only what is needed, only when it is needed. That is the principle for handling context.
When to start a new conversation
| Situation | Verdict |
|---|---|
| The topic changed | new conversation. Unhelpful context is baggage |
| An earlier attempt failed and you are changing direction | new conversation. The failed attempt keeps influencing things |
| Same topic, continuing | keep it |
| Answers are getting strange | new conversation. The window may be full |
The last is a common signal. When answer quality drops in a long conversation it is often not a model problem but a context that got messy.
Common misconceptions
"It remembers earlier conversation, so it remembers personal data?"
Two things to separate.
- In-conversation memory: only within that conversation. It does not carry to a new one
- What the service stores: this varies by service
As covered in things to watch out for, what gets stored where is a matter of the service's policy. The model's memory structure and the service's storage policy are separate questions.
"Does a bigger window solve everything?"
A bigger window does give you headroom. But cost keeps rising, and the quality drop from irrelevant material remains. Size is mitigation, not a solution.
Check yourself
1. What is the real structure behind the model "remembering" earlier conversation?
Answer
It does not remember. The whole prior conversation is fed back in on every request. So the longer the conversation, the more goes into each request.
2. Why does the same question cost more at the end of a long conversation?
Answer
Because the entire prior conversation is sent with it. Request 20 includes all 19 previous turns.
3. What should you suspect when answers get strange in a long conversation?
Answer
A full or messy context. It is usually not a model performance problem but accumulated irrelevant material, so starting a new conversation is faster.
What actually happens when the window fills → When context gets pushed out