Chunking Strategies
Chunking is how you split documents into retrievable pieces.
If chunks are bad, retrieval is bad. If retrieval is bad, the final answer is bad.
The tradeoff
| Chunk too small | Chunk too large |
|---|---|
| missing context | noisy retrieval |
| broken meaning | expensive prompts |
| weak citations | irrelevant text |
Common strategies
| Strategy | Use when |
|---|---|
| fixed-size | quick baseline |
| recursive | markdown, docs, headings |
| semantic | topic boundaries matter |
| sliding window | local context matters |
| parent-child | precise search plus larger answer context |
| table-aware | structured documents |
Good chunk metadata
Store:
- document ID
- title
- section heading
- page or line number
- source URL
- timestamp
- permissions
Practical default
Start with:
- split by headings
- keep 300-800 token chunks
- add 10-20 percent overlap only when needed
- evaluate with real user queries
Knowledge check
Q1: Why does chunking affect citations?
The chunk is often the evidence unit the answer cites.
Q2: What should you measure?
Whether expected evidence appears in retrieved top results.