Which kind of learning?

What you will learn What each of the four ways to make an AI "better at this" changes and where that change persists — and the order to try them in so you don't burn money and months.

Four options

One sentence turns up in almost every executive meeting.

"Can't we just train the model on our own data?"

You can. But that sentence means picking the most expensive of four options. To see why, separate what each one changes.

Only the bottom row behaves differently.

Fine-tuning is not a tool for putting facts in.

Fine-tune "our return window is 30 days" into a model and, when the policy changes to 14, you have to retrain. Until then the model states 30 with full confidence. Attaching material, by contrast, needs one line edited in one document.

All four are learning — they differ in where it persists

Reserve the word "training" for fine-tuning and the other three start sounding like stopgaps. They are not. All four make the next attempt better; what differs is what persists, and where.

What persists in your material is learning too

The paper that first proposed RAG calls this non-parametric memory — memory kept outside the parameters and fetched when needed, rather than baked into the weights. Add one document and from the next question onwards the system knows more. That is learning; it is just that the store learns rather than the model.

The same goes for instruction. Changing behaviour by attaching examples is called in-context learning, and the GPT-3 paper is titled Language Models are Few-Shot Learners. This one, though, is gone when the conversation ends.

Tools and procedure — the row organisations accumulate most

Connecting tools, capturing a procedure as a skill, building criteria through evaluation, placing approval points as guardrails. It goes by names like context engineering and harness engineering, but none has settled yet.

And this row survives a change of model. Most of what people mean on the ground by "our AI got smarter" happens here — and what improved is not the model but what has accumulated around it.

What persists in the weights

Of the three in that last row, the first two are already done by the model vendor. Pre-training in particular settles most of the ability, and you do not repeat it. "Let's train it on our data" means the fine-tuning slot and nothing else.

Which is why what fine-tuning can change is thin. Voice and output format shift; how the model understands the world was settled during pre-training. It is less "re-teach it as our company's model" and more showing someone who already knows the subject what your document format looks like.

What a mistake costs is what really differs

This is the practical reason to keep the four apart.

That bottom row is what this whole chapter is about. Reaching first for the row that is hardest to undo is the most common mistake there is.

The order to choose in

graph TD
  A["What is the problem?"] --> T{"Can it not reach something?<br/>(internal systems · live values)"}
  T -->|"yes"| TT["Connect a tool"]
  T -->|"no"| B{"Does it not know the material?"}
  B -->|"yes"| C{"Is there little of it?<br/>(a handful of documents)"}
  C -->|"yes"| D["Just paste it into the prompt"]
  C -->|"no"| E["Attach material (RAG)"]
  B -->|"no"| F{"Is the voice or format wrong?"}
  F -->|"yes"| G{"Can instruction fix it?"}
  G -->|"yes"| H["Fix the instruction"]
  G -->|"it keeps drifting"| I["Consider fine-tuning"]
  F -->|"no"| J["Build evaluation first"]

Working top-down is the rule. Cost and irreversibility rise sharply at each step down.

Check "can instruction fix it?" first

The most common reason a fine-tuning project starts is that nobody wrote a proper instruction. Try the four elements of a good instruction plus three to five examples. Only when the format still keeps drifting do you move down a step.

The test: if adding five examples visibly improves the result, this is an instruction problem, not a fine-tuning one.

Two cases where fine-tuning really is right

1. Locking a voice or format so it never slips

Some deliverables — legal review memos, regulatory filings — get rejected if the format is slightly off. Instruction holds it eight or nine times out of ten and wobbles on the rest. If that remainder is expensive, it is worth putting into the weights.

2. Moving a big model's ability into a small one

Here the goal is cost and speed. If a task works well on a large model but has to run tens of thousands of times a day, use those outputs as training data for a smaller model. Same quality, far cheaper.

Note that in both cases, the goal is not to insert facts.

In practice you combine all four

Systems that work well did not pick one of the four — they gave each a role.

Common misconceptions

"Isn't RAG a stopgap and fine-tuning the proper solution?"

Closer to the opposite. Most knowledge-dependent work is solved by retrieval. Fine-tuning is what you reach for when there is a specific, demonstrated gap the other three cannot close. A fine-tuning project started without naming that gap almost always fails.

"How much training data do we need?"

For locking a voice or format, a few hundred well-chosen examples is a starting point. But the cost here is not GPUs — it is the human time to produce and review those examples. Spend that same time on retrieval quality and you usually get a bigger effect.

Check yourself

1. Why is fine-tuning a bad way to teach "our return policy"?

2. What separates the four options?

3. What must you try before considering fine-tuning?

4. Which row do organisations accumulate most, and why is it invisible?

That covers what it knows. Next, the structure that loops on top of it → Agents — the loop that runs itself