Evaluation (evals)

What you will learn An individual can judge by eye; an organisation cannot. What to measure, how, and what happens without it.

The difference between individual and organisation

An individual needs no evaluation. Look at the answer, and if you do not like it, say so again. Judgement happens instantly and for free.

An organisation is different.

	Individual	Organisation
Who judges	you	a different person each time
When	immediately	later, or never
Standard	your instinct	different per person
Volume per day	a few	hundreds to thousands

If people look at hundreds of items, the gain from automating disappears. So judgement itself has to be automated, and that is evaluation.

What happens without it

graph TD
  A["Edit the prompt"] --> B["Seems better"]
  B --> C["Deploy"]
  C --> D["A different case gets worse"]
  D --> E["Nobody notices"]
  E --> A

The most common symptom: "I edited the prompt, this case got better but that one got worse." Except nobody looks at that one, so nobody knows. A few iterations of this and nobody can say which version is best.

Software already went through this problem, and the answer is the same — regression tests.

What you build — a golden dataset

The core is a set of examples with correct answers attached.

text

Case 1  Input:    "When's my order from yesterday arriving? Order A-12345"  Expected: category=delivery_status, order_number=A-12345Case 2  Input:    "Can I get a refund on this"  Expected: category=refund, order_number=none → must ask backCase 3  Input:    "It was late last time and it's late again, seriously"  Expected: category=complaint, sentiment=negative, escalate to a person...

How many do you need?

Twenty to thirty is enough to start. You do not need thousands.

What matters is not the count but what is in it.

Must include	Why
The most common cases	they determine overall quality
Cases that actually went wrong	prevents the same mistake recurring
Ambiguous edge cases	where judgement diverges
Cases where it must refuse	e.g. it should refuse requests for personal data

The second is the key one. When something goes wrong in production, add it to the dataset. Then the dataset improves over time.

How to score

Three approaches, usually mixed.

1. Exact match (most reliable)

For classification or extraction, where there is one right answer.

text

expected: category=delivery      actual: category=delivery      → passexpected: order_number=A-12345   actual: order_number=A-1234    → fail

Build it this way when you can. There is nothing to argue about in scoring.

2. Rule checks

Where there is not one right answer but there are conditions to satisfy.

text

□ is the reply three sentences or fewer□ if an amount is mentioned, does it match the lookup result□ did it avoid asking for personal data□ did it follow the required format (JSON)

3. Model scoring (LLM-as-judge)

For things you cannot write as rules — tone, appropriateness — have another model score it.

text

Evaluate the customer support reply below.Criteria: is it polite / did it actually answer the question / did itavoid inventing information it did not have.Answer pass or fail for each criterion only.

Caution: the scoring model is also wrong sometimes. So a person must spot check the scoring results occasionally. Without that, nobody knows when the scorer is wrong.

When to run it

text

□ every time you edit a prompt          ← the most important□ when you change models□ when you add or change tools□ periodically (weekly)

The first line is the key one. Edit a prompt without running evals and you are back in the diagram above.

How to start in practice

There is no need to start elaborately.

text

Week 1: gather 20 real cases in a spreadsheet        (two columns — input and expected — is enough to start)Week 2: run all 20 and count how many pass → this is your baselineWeek 3: rerun all 20 every time you edit the promptAfter:  when a case goes wrong, add it to the dataset

Week 2's baseline matters. If you do not know your current score, you do not know whether it improved. Same principle as recording the current value in Phase 1.

Connection to hyper-agile

Hyper-agile says "tests are the only judge of done." For code that test is a unit test; for AI output it is this evaluation.

They are different things.

	Subject	Judgement
Unit test	code behaviour	pass/fail is unambiguous
Evals	AI output	statistical. Something like 90% pass

AI output differs slightly every time, so "100% pass" is not the goal. "Better than the baseline" is the judgement.

Common misconceptions

"We're too small to need this."

Size is irrelevant. A spreadsheet with 20 cases is an eval. Without one, every prompt edit becomes a judgement by instinct, and that judgement is often wrong.

"Isn't building the evals more work than the thing itself?"

Gathering 20 cases takes half a day. Without that half day you can never answer "is this version better?" — and that question comes up dozens of times ahead of you.

Check yourself

1. Why does an individual not need evals but an organisation does?

Answer

An individual judges instantly and for free, while in an organisation the judge differs each time, the standard differs, and the volume is hundreds to thousands. If people review it all, the gain from automating disappears — so judgement itself must be automated.

2. Which of the must-includes in a golden dataset matters most?

Answer

Cases that actually went wrong. Adding production errors to the dataset stops the same mistake recurring, and the dataset improves over time.

3. How do evals differ from unit tests?

Answer

Unit tests are unambiguously pass or fail, while AI output varies slightly every time, so it is viewed statistically. The standard is not "100% pass" but "better than the baseline."

Next: what to pick, and what it costs → Choosing a model, and cost