“Should I use SFT or LoRA?” combines two different decisions. One describes what training rewards; the other describes which updates can be learned. Separate these axes before comparing methods.
Before you begin: You understand full fine-tuning, LoRA, QLoRA, and the role of training examples.
Those are separate decisions. Supervised fine-tuning describes a training objective and data format. LoRA describes a parameterization of updates. QLoRA adds a way to store the frozen base compactly. You can combine a supervised objective with LoRA or QLoRA.
Compare methods on the right axis
| Choice | Main change |
|---|---|
| Full fine-tuning | Update the model's main parameters. |
| LoRA | Learn low-rank updates to selected linear layers. |
| QLoRA | Train adapters with a quantized frozen base. |
| Prompt or prefix tuning | Learn continuous task-specific representations. |
| Preference optimization | Learn from comparisons between responses. |
Continuous prompt tuning learns vectors; it is not the same as editing a text prompt. Prefix-tuning variants inject learned representations into the model in specific ways. They can have deployment implications that differ from an adapter that can be merged into a linear weight.
Preference methods such as DPO are not interchangeable alternatives to LoRA on a single axis. A preference objective can train a full model or an adapter. Record both the objective and the trainable parameter set.
Build a two-axis experiment table
Draw rows for a supervised objective and a preference objective. Draw columns for full parameter updates and LoRA updates. Each cell describes a possible combination, provided the trainer supports it. QLoRA adds another storage choice for the frozen base; it does not replace the need to choose an objective.
Suppose supervised LoRA improves extraction, while preference optimization on a full model improves writing style. Can you attribute the difference to LoRA versus full fine-tuning?
Identify the confounded comparison
No. The objective, likely data, and parameterization all changed. A useful follow-up holds the objective and data fixed while changing the update method. You can still compare complete systems for deployment, but report what differs and avoid claiming you isolated one mechanism.
Add a column for the observed limiting resource: reviewed examples, weight memory, activation memory, training time, or serving complexity. The table should make your next experiment easier to justify, not merely list every available method.
Which constraint is the method supposed to relieve?
Adaptation methods trade different resources: training memory, stored customer-specific state, serving complexity and the effort needed to maintain variants. Name the binding constraint before selecting a technique. A method that fits training hardware may still complicate deployment or fail the task evaluation.
Use the adapter calculator to inspect one storage relationship, then build the wider decision in the challenge. Keep a prompting or retrieval baseline where it addresses the task. The goal is a method that improves the required behavior within the whole operating budget, not the most elaborate training setup available.
Several customers need related behavior changes under a limited GPU budget.
Compare full copies, small adapters and a non-training baseline with the same evaluation.
You can weigh quality against training, storage and serving requirements.
Start with the limiting resource
If optimizer state dominates memory, training fewer parameters may help substantially. If frozen weights dominate, compact storage or a smaller model may be necessary. If long sequences dominate, activation checkpointing or sequence handling may matter more than adapter rank.
If the bottleneck is reviewed data, a more complex optimizer will not manufacture trustworthy targets. A small clean dataset with representative hard cases may teach the intended behavior more reliably than a much larger set of unverified outputs.
Serving constraints also count. Many small task adapters may be easier to store than many full model copies. Switching adapters can complicate batching and cache handling. Merged adapters can simplify some serving paths but lose the same convenient switching behavior.
Design a fair comparison
Hold the base revision, training data, evaluation cases, and deployment prompt constant where possible. Allocate a reasonable tuning budget to each method. Record peak memory, training time, task quality, and regressions.
A comparison where one method receives much more tuning is still an experiment, but it does not isolate the method alone. Report the difference rather than presenting the winner as universal.
Make an experiment table before launching runs. Include the hypothesis for each change. For example: “Broader adapter targets may improve field extraction because attention-only updates plateaued on the same validation cases.” This is more informative than trying ranks at random and selecting the nicest chart.
Choose for a concrete task
Exercise: you need ten writing-style variants of one model. Each uses a small reviewed dataset, and the base weights already fit in serving memory. What is a sensible first comparison?
Compare your reasoning
Compare strong text prompts against separate small adapters using held-out style and factual-preservation checks. LoRA is a plausible storage-efficient experiment. QLoRA may help the training environment if base storage is tight, but it is not automatically necessary for the serving requirement. Full fine-tuning needs evidence that its added cost improves the goal.
The next lesson focuses on the demonstrations themselves, which often matter more than the method label.
Practice with feedback
See what rank changes in an adapter
One frozen d × d matrix gets two trainable matrices with shapes d × r and r × d. The adapter therefore has 2dr entries; the base has d².
Adapter/base ratio: 0.39%
The base bar stays fixed when rank changes. The adapter bar grows linearly with rank. The ratio describes this matrix only; a real run may adapt many matrices and keep optimizer states and activations.
What this experiment assumes. Parameter counting, not a quality prediction or a complete training-memory estimator. No assumption is made that a larger rank always helps. Notes and recorded results here last until you leave this page.
Choose an adaptation method by your limiting resource
You have 1,800 labelled examples, one 24 GB GPU, six customers each needing slightly different behaviour, and a two-week deadline.
Compare methods on the axis that actually binds you.
Check your understanding
Your task
Build the two-axis table and pick a method from your binding constraint.
These notes stay on this page. Download them before leaving.
What to include
- Constraints are ranked, not listed
- Memory figures are estimated with a stated rule of thumb
- The serving column reflects six variants, not one
- The first experiment addresses data quality or a prompted baseline
Compare with a worked answer
Here is one way to answer. Check how it uses the information in the task.
My constraints, ranked by which binds first: 1. Serving shape: six behaviours must share one 24 GB card. 2. Training memory: one 24 GB card. 3. Data: 1,800 examples, and I have not yet checked label agreement. The deadline is real but is not what rules anything out.
| Method | trainable params | peak train memory | serving for 6 customers | fits deadline | |---|---|---|---|---| | full FT | 7B | ~90 GB (16 bytes/param) | 6 x 14 GB = 84 GB | no | | LoRA | 2-80M | ~20 GB at bf16 base | 1 base + 6 adapters, ~15 GB | yes | | QLoRA | 2-80M | ~12 GB, 4-bit base | 1 base + 6 adapters, ~5 GB | yes | | prompt/prefix | <1M | ~10 GB | 1 base + 6 prefixes | yes, weakest capacity |
Choice: QLoRA, rank 16 on the attention projections, six adapters over one 4-bit base. The deciding constraint is the serving shape, not training: even if full fine-tuning fit on a bigger training machine, six 14 GB models cannot be served from this card, and adapters are the only shape that can.
The experiment I run first, which is not a training run: have a second person relabel 100 of the 1,800 examples and measure agreement. If agreement is below about 90%, the ceiling on any trained model is set by the labels, and cleaning them is worth more than every hyperparameter I could sweep in two weeks. I would also run the best prompted baseline on the held-out set, so I know what training has to beat before I start it.
When you are signed in, opening the challenge carries your edited working notes into its draft in this browser. The challenge has its own completion record. Practising here does not award points or mark it complete.
Sources
See LoRA, QLoRA, and Prefix-Tuning for distinct mechanisms. The PEFT documentation describes supported implementations.