Back
advanced
Fine-Tuning Techniques

Transfer Learning for LLMs

Understand how pretrained models transfer to downstream tasks through prompting, RAG, adapters, and fine-tuning

18 min read· Transfer Learning· Fine-Tuning· Pre-training· LLMs

Transfer Learning for LLMs

Transfer learning means using knowledge learned from one broad training process on new tasks.

LLMs transfer through:

  • prompting
  • in-context examples
  • retrieval
  • adapters
  • fine-tuning
  • distillation

Transfer options

MethodChanges weights?Best for
promptingnoquick task adaptation
RAGnoprivate/current facts
LoRA/adapterssmall added weightsrepeated behavior
full fine-tuneyesdeep specialization
distillationtrains smaller modelcost and latency reduction

Decision rule

Start with the lightest method that passes evals.

Do not train a model when a better prompt, retrieval system, or tool call solves the problem.

Knowledge check

Q1: What is transfer learning?

Reusing broad learned capability for a new task.

Q2: Why is RAG often better than fine-tuning for facts?

Facts can change without retraining the model.