# Agreeing with you, and leaning
> **What you will learn**
> [Hallucination](/guide/ai-hallucination) invents what is not there. The two in
> this chapter are different — **they tilt among things that are there.** So
> fact-checking does not catch them. How sycophancy and bias show up, and where
> they are dangerous.
## Why these two together
| | Hallucination | Sycophancy | Bias |
|---|---|---|---|
| What it does | Invents what is absent | Tilts toward you | Tilts toward the training data |
| Caught by | **Fact-checking** | Not caught | Not caught |
| What does catch it | Checking sources | **Asking the reverse** | **Asking with attributes swapped** |
**Hallucination you can verify. These two survive verification,** because each
individual answer may be perfectly true. The problem is not the answer; it is
**how it was selected.**
## Sycophancy — it tilts toward you
### Where it comes from
Models are tuned toward the answers people rated better. And people mostly rate
**the agreeing answer, the eager-to-help answer, the one that accepts my
premise** as better. That preference hardens into a property.
> **"Trying to be helpful" and "trying to agree" are not cleanly separated
> during training.**
### What it looks like
```
You: What do you think of this proposal? Took me three weeks.
AI: The overall structure is very solid. Section 2 is especially strong.
You: Honestly I think it's weak. What do you think?
AI: As you say, there are some weaknesses. Section 2 is thinly supported.
```
**Same document, reversed answer.** What changed was not the document but the
preference you leaked.
Four symptoms:
```
□ State an opinion and it organises itself around that opinion
□ One "really?" and it retracts what it just said
□ It answers on top of your premise without testing the premise
□ It generally praises drafts, plans, and diagnoses
```
### Where it is dangerous
**When you use AI as a second opinion.** One person judges, then has the AI
confirm it. It feels like review happened, but what you actually got was **your
own judgement returned as an echo.** You did not add a review step; you added a
step you believe exists.
```
✗ "There's no problem with these contract terms, right?"
→ an answer saying there is no problem comes easily
✓ "Find three clauses in this contract that could put us at a
disadvantage"
→ you gave it a job of finding
```
### What works
| Method | How |
|---|---|
| **Do not state your view first** | Get the judgement, then say what you thought |
| **Assign the other side** | "If this plan fails, why?" |
| **Assign both sides** | Three arguments for, three against |
| **Make it test the premise** | "Is any premise in my question wrong?" |
| **Split the roles** | Draft in one conversation, review in another |
That last line matters most in an organisation. **Put the maker and the reviewer
in the same conversation and no review happens.** It is the same reason human
organisations separate the author from the approver.
To see it once for yourself →
[Ask for the same thing three times](/guide/ai-try-compare)
## Bias — it tilts toward the training data
### Where it comes from
Training data comes from the world, and **the world's writing was not produced
evenly.** Some countries, occupations, names, and phrasings appear far more
often; others barely at all. The model inherits that distribution.
This is not malice, it is **an average.** But an average applied to an individual
becomes discrimination.
### What it looks like
```
You: Write a short example passage featuring a developer and a nurse.
AI: (produces sentences that assign particular genders by default)
You: Score these candidate CVs.
AI: (names, schools, and hometowns quietly move the scores)
```
The first one is visible. **The second is not** — you get a score and no reason.
### Where it is dangerous
**When you automate a judgement about people.**
| Use | Risk | Principle |
|---|---|---|
| Polishing prose, summarising | Low | Use as is |
| Classifying customer enquiries | Medium | Sample review |
| **Hiring, appraisal, screening scores** | **High** | **A person decides; AI only assembles evidence** |
| **Differential pricing or limits** | **High** | Check the regulation first |
At the high tier, what the model may do is **assemble the evidence a person will
read, not produce the score.** The moment a decision is made, a duty to explain
it follows — and the model cannot explain why it gave that score. For the
regulatory side →
[Regulation and compliance](/guide/ax-compliance)
### What works
**Asking with attributes swapped** is the cheapest reliable test.
```
Take one CV, change only the name, run it ten times,
and see whether the score moves.
```
If it moves, do not use it for that purpose. Build the test once and you can keep
running it — that is what [evaluation (evals)](/guide/ai-evaluation) is. Defining
quality follows in [Quality](/guide/ax-quality).
## Summary — which question to ask
```
Suspect hallucination: "Does this source actually exist?" → verify
Suspect sycophancy: "Same answer if I ask the reverse?" → invert
Suspect bias: "Same answer if I swap attributes?" → substitute
```
The three do not substitute for one another. **No amount of source-checking
catches sycophancy, and asking the reverse does not catch bias.**
---
## Check
**1. Why is sycophancy harder to filter out than hallucination?**
Answer
**Because the answer itself may be true.** Hallucination surfaces when you check
the source; sycophancy is a selection among true points, keeping the ones on your
side, so verification finds nothing. Only asking the reverse exposes it.
**2. Why is using AI as a "second opinion" dangerous?**
Answer
Stating your judgement and then asking for confirmation returns **an echo, not a
review.** That is worse than adding nothing, because **you now believe a review
step exists.** Get the judgement before you state your view, and assign the
opposite side too.
**3. What is the principle for scoring CVs with a model?**
Answer
**Do not have it produce the score; have it assemble the evidence a person will
read.** A decision carries a duty to explain, and the model cannot explain its
own score. And test first by swapping only attributes such as the name to see
whether the score moves.
---
On to keeping it from answering out of memory →
[Tool use](/guide/ai-tool-use)