# Choosing a model, and cost > **What you will learn** > Why models come in tiers, how billing actually works, and three ways to cut > cost without cutting quality. ## There is more than one model Even within one provider, models usually come in three tiers. | Tier | Character | Fits | |---|---|---| | **Top** | smartest, most expensive | hard judgement, long autonomous runs, complex code | | **Mid** | good enough for most things | most day-to-day work | | **Light** | fast and cheap | classification, extraction, high-volume repetition | **"Why not just run everything on the best one?"** is the first reaction, and it is where the money starts leaking. ## How billing works The unit is the **token** — roughly one to two tokens per English word. Three things matter: ``` 1. Tokens in (input) and tokens out (output) are priced separately 2. Output costs about 5x more than input 3. Every turn re-charges the whole conversation so far as input ``` Point 3 is the one people miss. As we saw in [the context window](/guide/ai-context-window), the model re-reads the entire conversation each turn — so the twentieth question in a conversation costs far more than the first. > **As of August 2026.** These figures change. Take the **ratios and the > principle**, not the numbers. The gap between tiers looks roughly like this (per million tokens, list price — it changes over time). | Tier | Input | Output | Relative | |---|---|---|---| | Top | $5 | $25 | 5x | | Mid | $3 | $15 | 3x | | Light | $1 | $5 | 1x | > **Run everything on the top tier and you pay 5x for work the light tier > handles.** Put hard judgement on the light tier and you pay more in human > rework. The trick is **matching the tier to the job.** ## What goes where ```mermaid graph TD A["What kind of task is it?"] --> B{"Does a mistake mean human rework?"} B -->|"yes — expensive"| C["Top"] B -->|"no"| D{"Fixed format,
almost no judgement?"} D -->|"yes"| E["Light"] D -->|"no"| F["Start at Mid"] ``` A real deployment usually ends up looking like this. | Task | Tier | Why | |---|---|---| | Document classification, tagging | Light | narrow judgement, high volume | | Email drafts, summaries, translation | Mid | more than sufficient | | Multi-step research and write-up | Mid | top tier is overkill | | Complex code changes, design decisions | Top | rework cost is high when wrong | | Final review and sign-off | Top | an error here collapses everything | **The last two rows are the point.** Do the bulk on a cheap model and put **only the review step** on an expensive one — that combination gives the best quality per unit of cost. ## Three ways to save ### 1. Cache the identical prefix Most requests share an **identical opening**: the instructions, the company background, the policies, the tool list. You re-compute and re-pay for the same text every single time. **Caching** stores that repeated prefix and reuses it on the next request. ``` Read from cache → about 10% of list price Write to cache → about 1.25x list price (once) ``` **You come out ahead from the second request onward** — 1.25x + 0.1x = 1.35x beats 2x (two uncached requests). The longer the instructions and the more repetitive the workload, the bigger the effect. But **the cache only hits on an exact match from the beginning.** Put "today's date: ..." at the very top of the instructions and every request becomes a different string, breaking the cache entirely. ``` ✗ top of the prompt: "Today is 16 August 2026..." → cache breaks daily ✓ fixed instructions first → changing content after ``` ### 2. Dial the depth of thinking Current models let you set how hard to think before answering. Lower is fast and cheap; higher is slower and pricier but solves harder problems. **Most teams leave it at maximum and never touch it.** Dropping day-to-day work one notch frequently cuts cost noticeably with no perceptible quality change. ### 3. Put less in The surest method. Use [retrieval](/guide/ai-grounding) to pass relevant chunks instead of whole documents, and wrap up long conversations to start fresh. ## A big context window is not a goal Current models hold up to a million tokens — several books' worth. But **being able to hold it is not the same as it being a good idea.** ``` Fill a million tokens every turn and you get: → input cost at that volume, every turn → irrelevant content diluting accuracy → slower responses ``` Why the middle line happens is covered in [when context gets pushed out](/guide/ai-context-overflow). **A large window is headroom, not a target.** ## Common misconceptions ### "Isn't using only the best model the safe choice?" Quality gets safer and **cost and latency get riskier.** And on high-volume repetitive work, the top tier's advantages mostly don't show — a smarter model makes little difference where there is no judgement to make. ### "Doesn't juggling several models get complicated?" Start with two: **one for day-to-day work, one for review and hard judgement.** That two-model setup alone changes the cost structure substantially. Three or more is a question for after [evaluation](/guide/ai-evaluation) is in place. --- ## Check yourself **1. Why does the last question in a 20-turn conversation cost far more than the first?**
Answer **Because every turn re-charges the entire conversation so far as input.** The model does not remember the conversation — it re-reads it. Wrapping up and starting fresh is the right call for cost as well as quality.
**2. Why does putting today's date at the top of the prompt raise cost?**
Answer **It breaks the cache.** Caching only hits on an exact match from the start, so a value that changes at the very top forces everything after it to be re-computed each time. Fixed content first, changing content after.
**3. What combination gives the best quality per unit of cost?**
Answer **Bulk processing on a cheap model, with only review and final sign-off on an expensive one.** All-top-tier pays 5x for work with no judgement in it; all-light-tier costs more in human rework.
--- That is the technical basics. Next, the standard that connects all of this to real systems → [MCP — why a standard was needed](/guide/mcp-why-standard)