Try attaching sources

What you will learn Not a chapter to read but one to do. See for yourself how it differs with and without material attached — and what happens when the material has no answer.

What you need

text

□ One company document — a policy, a manual, a report, anything   · roughly 10-30 pages   · better if it contains numbers (amounts, periods, rates)   · start with something not confidential□ Three questions, written down first, that only this document can answer

That third line is the point of the exercise. Write the questions afterwards and you will end up fitting the questions to the answers.

If you cannot get hold of a company document right now, expand the sample below and copy it. Exercises 1-4 are written against this document, so you can check every answer against a known key.

Sample document — Hanbit Trading Co., travel expense policy (fictional)

text

[page 1]Hanbit Trading Co. — Domestic Travel Expense PolicyEnacted 2023-01-01 / Last revised 2025-03-15Article 1 (Purpose) This policy sets out the standards and procedures forreimbursing costs incurred by employees on business travel.Article 2 (Scope) This policy applies to all permanent and contractemployees. Seconded staff and partner-company personnel are covered byseparate agreements.[page 2]Article 3 (Grades) Travel allowances are paid by grade.  Grade A — executives and division heads  Grade B — team leads and senior staff  Grade C — associate level and belowArticle 4 (Accommodation) The per-night accommodation limit for domestictravel is set by grade.  Grade A  KRW 150,000  Grade B  KRW 100,000  Grade C  KRW  80,000For stays in a metropolitan city, the limit above is increased by 20%.[page 3]Article 5 (Meals) The daily meal allowance is KRW 30,000 regardless ofgrade. Where breakfast is included in the accommodation, KRW 8,000 isdeducted from the daily meal allowance.Article 6 (Transport) Actual costs are reimbursed. Air travel is limited todomestic routes in economy class, except for Grade A.Article 7 (Settlement) Claims must be filed with receipts within 10 businessdays of the last day of travel. Late claims may be withheld.[page 4]Article 8 (Overseas travel) Overseas travel expenses are governed by theseparate Overseas Travel Policy.Article 9 (Prior approval) All travel requires the approval of the directsupervisor at least 3 business days before departure. In urgent cases,approval may be obtained retroactively.Addendum This policy takes effect on 2025-04-01.

The three questions for exercises 1-2 — do not read the key first.

#	Question	Answer	Source
1	What is the per-night accommodation limit for a Grade B employee?	KRW 100,000 (KRW 120,000 in a metropolitan city)	p.2, Art. 4
2	What is the daily meal allowance when breakfast is included?	KRW 22,000	p.3, Art. 5
3	By when must a claim be filed?	Within 10 business days of the last day of travel	p.3, Art. 7

The trap question for exercise 3

text

Does the company cover the cost of a visa for overseas travel?

Article 8 only says "governed by a separate policy" and visas appear nowhere in this document. If the model reaches for Article 8 and answers plausibly, that is exactly the failure exercise 3 is built to catch.

Exercise 1 — Ask with nothing attached (5 min)

Do not attach the document. Ask your three questions as they are.

text

e.g. "What is the domestic accommodation limit in our travel policy?"

What to look for

The answer you get	What it means
"I don't have that information"	an honest model. Good sign
A plausible-looking number	it was invented. Your policy cannot be in the training data
A generality	"it varies by company..." — an evasion

If you got the middle row, screenshot it. It explains hallucination to your team far better than someone else's example.

Exercise 2 — Attach it and ask again (10 min)

Now attach or paste the document and ask the same questions. Add one more sentence.

text

Answer only from the document below. Put the page number at the end of the answer.If the document does not support an answer, say "not in the document."<paste or attach the document>Question: what is the domestic accommodation limit?

What to look for

text

□ Was the answer right?□ Was a page number attached?□ Does that page actually say it?   ← always check this one

Do not skip the third. Page numbers get invented too. A citation being present and a citation being correct are different things.

Exercise 3 — The trap question (10 min)

The most important exercise. Ask something the document does not answer.

text

e.g. with the travel policy attached     "Does the company cover visa fees for overseas trips?"     (assuming visas are not mentioned)

What to look for

The answer you get	Verdict
"Not in the document"	✓ working correctly
It stretches a nearby clause into an answer	the most dangerous failure
It answers from general knowledge	the instruction was ignored

The second row is what causes incidents. It even carries a citation, so it looks correct — but that clause is about something else.

If you get it, strengthen the instruction.

text

If it is not stated explicitly in the document, do not infer — reply only"not in the document." Do not substitute a similar-looking clause.

Run it again and see whether it changes. That one line often catches it.

Exercise 4 — Chunking changes everything (5 min)

Instead of the document, paste one paragraph cut out of it and ask the same question.

text

"  Grade B  KRW 100,000"Question: "What is the grade B accommodation limit?"

If you are using the sample document, cut exactly that line out of Article 4. A limit on what, per night or per trip, and whether the metropolitan uplift applies — all of it was cut away. If the answer wobbles or it asks you back, you have just felt why attaching context to each chunk cuts the retrieval failure rate by a third.

After the exercise

text

□ I saw it invent an answer with no material attached□ I know what asking for a citation does□ I know a citation still has to be opened and checked□ I know "say so if it isn't there" has to be stated explicitly

All four checked means you are ready for Drive or document conversion.

Check yourself

1. In exercise 3, which result is most dangerous, and why?

Answer

Stretching a nearby clause into an answer. It even carries a citation, so it looks correct while actually being about something else — which fools the reviewer too. Adding "if it is not stated explicitly, reply only 'not in the document'" blocks most of it.

2. Why must a cited answer still be opened and checked?

Answer

Because page numbers themselves get invented. A citation being present is not the same as a citation being correct — which is why the third check in exercise 2 cannot be skipped.

3. Why does the answer wobble when you paste only one paragraph?

Answer

A chunk torn out loses its context. What "this limit" refers to was in the paragraph above, which got cut. That is why production systems prepend a line of context — which document, which section — before storing each chunk.

Now for "why not just train it on our data?" → Which kind of learning?