Context Compression and Long-Context RAG
Context compression means reducing text while preserving the information needed to answer.
The goal is not shorter text. The goal is higher signal.
Why compression helps
Long context can still fail when:
- evidence is buried
- chunks repeat each other
- irrelevant text distracts the model
- token cost is too high
- latency is too slow
Compression strategies
| Strategy | Use when |
|---|---|
| extractive compression | keep exact source sentences |
| abstractive summary | compress long background context |
| metadata filtering | remove irrelevant documents early |
| reranking | keep only strongest chunks |
| section selection | include headings around evidence |
| conversation summary | compress old chat history |
Safety rule
For factual RAG, prefer extractive compression when possible. Summaries can accidentally change meaning.
Good compressed context
Good context keeps:
- source ID
- exact quote or close paraphrase
- section title
- date/version
- enough surrounding context
Knowledge check
Q1: What is the risk of abstractive compression?
It can change or invent meaning.
Q2: What metadata should compressed evidence keep?
Source ID, section, date/version, and enough text to verify the claim.