# Try attaching sources > **What you will learn** > Not a chapter to read but one to **do**. See for yourself how it differs with > and without material attached — and what happens when the material has no > answer. ## What you need ``` □ One company document — a policy, a manual, a report, anything · roughly 10-30 pages · better if it contains numbers (amounts, periods, rates) · start with something not confidential □ Three questions, written down first, that only this document can answer ``` **That third line is the point of the exercise.** Write the questions afterwards and you will end up fitting the questions to the answers. > **If you cannot get hold of a company document right now,** expand the sample > below and copy it. Exercises 1-4 are written **against this document**, so you > can check every answer against a known key.
Sample document — Hanbit Trading Co., travel expense policy (fictional) ``` [page 1] Hanbit Trading Co. — Domestic Travel Expense Policy Enacted 2023-01-01 / Last revised 2025-03-15 Article 1 (Purpose) This policy sets out the standards and procedures for reimbursing costs incurred by employees on business travel. Article 2 (Scope) This policy applies to all permanent and contract employees. Seconded staff and partner-company personnel are covered by separate agreements. [page 2] Article 3 (Grades) Travel allowances are paid by grade. Grade A — executives and division heads Grade B — team leads and senior staff Grade C — associate level and below Article 4 (Accommodation) The per-night accommodation limit for domestic travel is set by grade. Grade A KRW 150,000 Grade B KRW 100,000 Grade C KRW 80,000 For stays in a metropolitan city, the limit above is increased by 20%. [page 3] Article 5 (Meals) The daily meal allowance is KRW 30,000 regardless of grade. Where breakfast is included in the accommodation, KRW 8,000 is deducted from the daily meal allowance. Article 6 (Transport) Actual costs are reimbursed. Air travel is limited to domestic routes in economy class, except for Grade A. Article 7 (Settlement) Claims must be filed with receipts within 10 business days of the last day of travel. Late claims may be withheld. [page 4] Article 8 (Overseas travel) Overseas travel expenses are governed by the separate Overseas Travel Policy. Article 9 (Prior approval) All travel requires the approval of the direct supervisor at least 3 business days before departure. In urgent cases, approval may be obtained retroactively. Addendum This policy takes effect on 2025-04-01. ``` **The three questions for exercises 1-2** — do not read the key first. | # | Question | Answer | Source | |---|---|---|---| | 1 | What is the per-night accommodation limit for a Grade B employee? | KRW 100,000 (KRW 120,000 in a metropolitan city) | p.2, Art. 4 | | 2 | What is the daily meal allowance when breakfast is included? | KRW 22,000 | p.3, Art. 5 | | 3 | By when must a claim be filed? | Within 10 business days of the last day of travel | p.3, Art. 7 | **The trap question for exercise 3** ``` Does the company cover the cost of a visa for overseas travel? ``` Article 8 only says "governed by a separate policy" and **visas appear nowhere in this document.** If the model reaches for Article 8 and answers plausibly, that is exactly the failure exercise 3 is built to catch.
--- ## Exercise 1 — Ask with nothing attached (5 min) **Do not attach the document.** Ask your three questions as they are. ``` e.g. "What is the domestic accommodation limit in our travel policy?" ``` ### What to look for | The answer you get | What it means | |---|---| | "I don't have that information" | an honest model. Good sign | | A plausible-looking number | **it was invented.** Your policy cannot be in the training data | | A generality | "it varies by company..." — an evasion | > **If you got the middle row, screenshot it.** It explains > [hallucination](/guide/ai-hallucination) to your team far better than someone > else's example. --- ## Exercise 2 — Attach it and ask again (10 min) Now attach or paste the document and ask the same questions. **Add one more sentence.** ``` Answer only from the document below. Put the page number at the end of the answer. If the document does not support an answer, say "not in the document." Question: what is the domestic accommodation limit? ``` ### What to look for ``` □ Was the answer right? □ Was a page number attached? □ Does that page actually say it? ← always check this one ``` **Do not skip the third.** Page numbers get invented too. A citation being present and a citation being correct are different things. --- ## Exercise 3 — The trap question (10 min) The most important exercise. Ask something the document **does not** answer. ``` e.g. with the travel policy attached "Does the company cover visa fees for overseas trips?" (assuming visas are not mentioned) ``` ### What to look for | The answer you get | Verdict | |---|---| | "Not in the document" | ✓ working correctly | | It stretches a nearby clause into an answer | **the most dangerous failure** | | It answers from general knowledge | the instruction was ignored | **The second row is what causes incidents.** It even carries a citation, so it looks correct — but that clause is about something else. If you get it, strengthen the instruction. ``` If it is not stated explicitly in the document, do not infer — reply only "not in the document." Do not substitute a similar-looking clause. ``` **Run it again and see whether it changes.** That one line often catches it. --- ## Exercise 4 — Chunking changes everything (5 min) Instead of the document, paste **one paragraph** cut out of it and ask the same question. ``` " Grade B KRW 100,000" Question: "What is the grade B accommodation limit?" ``` If you are using the sample document, cut exactly that line out of Article 4. **A limit on what, per night or per trip, and whether the metropolitan uplift applies — all of it was cut away.** If the answer wobbles or it asks you back, you have just felt why [attaching context to each chunk](/guide/ai-grounding) cuts the retrieval failure rate by a third. --- ## After the exercise ``` □ I saw it invent an answer with no material attached □ I know what asking for a citation does □ I know a citation still has to be opened and checked □ I know "say so if it isn't there" has to be stated explicitly ``` All four checked means you are ready for [Drive](/guide/cn-drive) or [document conversion](/guide/ht-documents). --- ## Check yourself **1. In exercise 3, which result is most dangerous, and why?**
Answer **Stretching a nearby clause into an answer.** It even carries a citation, so it looks correct while actually being about something else — which fools the reviewer too. Adding "if it is not stated explicitly, reply only 'not in the document'" blocks most of it.
**2. Why must a cited answer still be opened and checked?**
Answer **Because page numbers themselves get invented.** A citation being present is not the same as a citation being correct — which is why the third check in exercise 2 cannot be skipped.
**3. Why does the answer wobble when you paste only one paragraph?**
Answer **A chunk torn out loses its context.** What "this limit" refers to was in the paragraph above, which got cut. That is why production systems prepend a line of context — which document, which section — before storing each chunk.
--- Now for "why not just train it on our data?" → [Which kind of learning?](/guide/ai-rag-vs-finetune)