Fine-Tuning Methods: SFT, PEFT, RLHF, and DPO
Fine-tuning changes model behavior. It should be used only after prompting, tools, and RAG are not enough.
Method map
| Method | Best for |
|---|---|
| SFT | teaching examples of ideal behavior |
| LoRA/PEFT | low-cost specialization |
| QLoRA | fine-tuning larger models with less memory |
| RLHF | learning from human preferences |
| DPO | simpler preference optimization |
| distillation | moving behavior into a smaller model |
Choose by goal
| Goal | Method |
|---|---|
| consistent format | SFT or LoRA |
| cheaper repeated task | distillation or LoRA |
| better preference behavior | DPO/RLHF |
| new factual knowledge | usually RAG |
| tool-call style | SFT/LoRA with tool traces |
Evals first
Before training:
- collect representative examples
- define success criteria
- run baseline
- estimate deployment cost
- plan rollback
Knowledge check
Q1: What is PEFT?
Parameter-efficient fine-tuning, where only a small set of parameters is trained.
Q2: Which method should handle changing facts?
Usually RAG, not fine-tuning.