# Build one pipeline end to end
> **What you will learn**
> Not a chapter to read but one to **do**. Build a **three-stage** pipeline, then
> **break the middle stage on purpose** and see what the later stages do.
## What you need
```
□ 60 minutes
□ Schedules working
□ Auto-start enabled — schedules do not fire if it is down
□ One working directory that is easy to restore
□ Do not include a stage that goes outside this time
```
If you have never set one up, read [scheduled runs](/guide/ht-schedule) first.
**Keep that last line.** Put a send or a post in your first pipeline and step 4,
where you break it deliberately, becomes a real incident.
---
## Step 1 — Draw the three stages (10 min)
Draw it on paper first. The key is writing down **what each stage leaves
behind.**
```
[1 collect] -> what it leaves:
[2 shape] -> what it leaves:
[3 summarise] -> what it leaves:
```
```
Example:
1 collect every day at 09:00, write yesterday's changed files to a file
2 shape read that list, group by type, write a table
3 summarise read the table, write a three-line summary
```
**Every stage must leave a file.** A stage that holds its result only in its head
cannot be re-run from the middle — this is what "stages must be re-runnable" in
[pipeline design](/guide/ht-pipelines) actually looks like.
---
## Step 2 — Build stage 1 alone and run it (15 min)
Do not build all three at once. **Stand one up and verify it.**
```
□ You created the stage 1 schedule
□ You ran it once by hand
□ The output file actually appeared
□ You opened that file and checked the contents
```
**Do not skip the fourth.** A file existing and a file being right are different
things. Skip it and, when stage 3 comes out wrong, you cannot tell where it went
wrong.
---
## Step 3 — Chain stages 2 and 3 (15 min)
Build each later stage so that it **reads the previous stage's output.**
```
□ Stage 2 reads stage 1's output file
□ Stage 3 reads stage 2's output file
□ Each stage runs later than the one before <- check the gaps
```
### How to set the gaps
```
Stage 1 09:00 -> Stage 2 09:20 -> Stage 3 09:40
```
**Do not put stage 2 at 09:05 because stage 1 usually takes three minutes.** There
will be slow days. Leave **at least three times the usual duration**, or write the
stage so it stops when the previous output is missing.
```
The line to put in stages 2 and 3:
"If there is no file to read, produce nothing and leave only 'no previous result'."
```
---
## Step 4 — Break the middle (15 min)
**This is the point of the exercise.** **Rename or delete** stage 1's output file,
then run stages 2 and 3.
### What to look for
| What stage 2 did | Verdict |
|---|---|
| Left "no previous result" and stopped | ✓ properly designed |
| Produced an empty table and completed | **stage 3 now summarises that empty table** |
| Found and used an older file | **the worst — today's report goes out on yesterday's data** |
**That third row is the classic production incident.** No error is raised, so
nobody notices. The numbers just quietly slip by a day.
### Check all the way to stage 3
```
□ You opened stage 3's output
□ Did "none" propagate into it, or did you get a plausible summary?
```
**If it did not propagate, fix the stage 2 instruction and do step 4 again.** One
stage's failure being laundered into success by the next is the most common way a
pipeline fails.
---
## Step 5 — Cost and cleanup (5 min)
```
□ You counted how many times a day each of the three runs
□ You turned off the practice schedules
□ For anything you kept, the failure behaviour is in the instruction
```
**Do count the first one.** Every stage carries cost, and running daily makes it
[accumulate](/guide/ht-pipelines). Three stages every day is ninety runs a month.
---
## Self-check
```
□ Every stage leaves a file
□ You stood the stages up one at a time and verified each
□ Gaps are at least three times the usual duration
□ You broke the middle and checked the later stages <- skip this and you did not do it
□ "No previous result" propagated all the way to stage 3
```
---
## Check yourself
**1. Why must every stage leave a file?**
Answer
**So you can re-run from the middle.** Without the result on disk you cannot re-run
stage 2 alone; you have to redo everything from the start.
**2. Why is "found and used an older file" the most dangerous outcome?**
Answer
**No error is raised, so nobody notices.** The pipeline reports success and today's
report goes out built on yesterday's data.
**3. Why leave generous gaps between stages?**
Answer
**There will always be slow days.** If the next stage runs before the previous
finished, it proceeds on a missing or stale result. Widen the gap, or write the
stage to stop when there is no result.
---
Next, turning the repeated procedure itself into a reusable unit →
[Capturing procedures as skills](/guide/ht-skills)