Advanced Chunking Strategies
Chunking decides what the retriever can find.
Bad chunks make good models look bad.
Chunking goals
Good chunks are:
- small enough to retrieve precisely
- large enough to preserve meaning
- attached to source metadata
- aligned to document structure
- easy to cite
Strategies
| Strategy | Best for |
|---|---|
| fixed token chunks | simple baseline |
| recursive chunks | markdown/docs with headings |
| semantic chunks | topic-based boundaries |
| parent document retrieval | small search chunk, larger answer context |
| sentence window retrieval | exact sentence plus nearby context |
| table-aware chunking | forms, CSVs, financial docs |
Metadata matters
Store:
- document ID
- title
- section heading
- page
- date/version
- permission scope
Without metadata, citations and debugging are weak.
Evaluation
Test chunking with real questions. Measure whether expected evidence appears in top results.
Knowledge check
Q1: Why not use one giant chunk per document?
It retrieves too broadly and adds noise to the model context.
Q2: Why use parent document retrieval?
To search precisely but answer with enough surrounding context.