Choosing a model, and cost

What you will learn Why models come in tiers, how billing actually works, and three ways to cut cost without cutting quality.

There is more than one model

Even within one provider, models usually come in three tiers.

"Why not just run everything on the best one?" is the first reaction, and it is where the money starts leaking.

How billing works

The unit is the token — roughly one to two tokens per English word. Three things matter:

Point 3 is the one people miss. As we saw in the context window, the model re-reads the entire conversation each turn — so the twentieth question in a conversation costs far more than the first.

As of August 2026. These figures change. Take the ratios and the principle, not the numbers.

The gap between tiers looks roughly like this (per million tokens, list price — it changes over time).

Run everything on the top tier and you pay 5x for work the light tier handles. Put hard judgement on the light tier and you pay more in human rework. The trick is matching the tier to the job.

What goes where

graph TD
  A["What kind of task is it?"] --> B{"Does a mistake mean human rework?"}
  B -->|"yes — expensive"| C["Top"]
  B -->|"no"| D{"Fixed format,<br/>almost no judgement?"}
  D -->|"yes"| E["Light"]
  D -->|"no"| F["Start at Mid"]

A real deployment usually ends up looking like this.

The last two rows are the point. Do the bulk on a cheap model and put only the review step on an expensive one — that combination gives the best quality per unit of cost.

Three ways to save

1. Cache the identical prefix

Most requests share an identical opening: the instructions, the company background, the policies, the tool list. You re-compute and re-pay for the same text every single time.

Caching stores that repeated prefix and reuses it on the next request.

You come out ahead from the second request onward — 1.25x + 0.1x = 1.35x beats 2x (two uncached requests). The longer the instructions and the more repetitive the workload, the bigger the effect.

But the cache only hits on an exact match from the beginning. Put "today's date: ..." at the very top of the instructions and every request becomes a different string, breaking the cache entirely.

2. Dial the depth of thinking

Current models let you set how hard to think before answering. Lower is fast and cheap; higher is slower and pricier but solves harder problems.

Most teams leave it at maximum and never touch it. Dropping day-to-day work one notch frequently cuts cost noticeably with no perceptible quality change.

3. Put less in

The surest method. Use retrieval to pass relevant chunks instead of whole documents, and wrap up long conversations to start fresh.

A big context window is not a goal

Current models hold up to a million tokens — several books' worth. But being able to hold it is not the same as it being a good idea.

Why the middle line happens is covered in when context gets pushed out.

A large window is headroom, not a target.

Common misconceptions

"Isn't using only the best model the safe choice?"

Quality gets safer and cost and latency get riskier. And on high-volume repetitive work, the top tier's advantages mostly don't show — a smarter model makes little difference where there is no judgement to make.

"Doesn't juggling several models get complicated?"

Start with two: one for day-to-day work, one for review and hard judgement. That two-model setup alone changes the cost structure substantially. Three or more is a question for after evaluation is in place.

Check yourself

1. Why does the last question in a 20-turn conversation cost far more than the first?

2. Why does putting today's date at the top of the prompt raise cost?

3. What combination gives the best quality per unit of cost?

That is the technical basics. Next, the standard that connects all of this to real systems → MCP — why a standard was needed