Project: adapt a workshop assistant and prove what changed

Can your adapter preserve a missing field more reliably than a strong prompt baseline? Build one small adaptation project around that question, then make the deployment decision from held-out behavior rather than training loss alone.

Before you begin: You understand instruction data, LoRA, evaluation splits, and basic Python.

Keep current records outside the weights and supply them at request time. This lets the project separate a stable answering behavior from facts that change.

Define the deliverables

Produce a dataset with source groups, a baseline evaluation, an adapter experiment, and a short comparison report. The report must include gains, regressions, runtime requirements, and a decision to keep or reject the adapter.

Begin with reviewed examples covering complete records, missing fields, conflicting versions, paraphrases, and out-of-scope requests. Use fictional workshop details while developing the pipeline. Do not upload private conversations to a training service merely because they are convenient.

Store each example as one JSONL record with group_id, versioned source_id, and messages. Keep related examples from one workshop or source group in the same split. Create train.jsonl, validation.jsonl, and a separate final test.jsonl.

Validate the split before training

The following local check uses a tiny in-memory example. Apply the same invariant to your actual files.

# Runnable: Python 3, standard library.
train = [{'group_id': 'pottery'}, {'group_id': 'coding'}]
validation = [{'group_id': 'drawing'}]
test = [{'group_id': 'music'}]
groups = [{row['group_id'] for row in split}
          for split in [train, validation, test]]
assert groups[0].isdisjoint(groups[1])
assert groups[0].isdisjoint(groups[2])
assert groups[1].isdisjoint(groups[2])
print('Source groups are disjoint')

This catches exact group overlap, not every kind of near-duplicate leakage. Review similar prompts and records too.

Keep revisions of one source together

Imagine pottery-v1 in training and pottery-v2 in testing. The IDs differ, but most text may be identical. A disjoint set of versioned source IDs would miss this leakage. Group both under a stable family such as pottery, then keep that family in one split.

For practice, deliberately insert a new revision of a training source into validation and confirm that the group check fails. Restore a clean split before running the trainer. The final test should ask whether the behavior transfers to genuinely new cases, not whether the adapter recognizes a familiar notice.

Run a small adapter experiment

Use a compatible Python environment with PyTorch, Transformers, Datasets, TRL, and PEFT. Choose a causal chat model you are permitted to use and that fits your hardware. Set TRAINING_MODEL to its local path or model ID. For remote IDs, pin a revision in your loading setup and record it.

This integration example reads the files you create. It is not executed by the lesson and does not include a GPU, model download, or prepared dataset. Start with a short smoke run before increasing the training budget.

# Integration example: requires the ML environment and JSONL files above.
import os
from datasets import load_dataset
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

data = load_dataset('json', data_files={
    'train': 'train.jsonl',
    'validation': 'validation.jsonl',
})
trainer = SFTTrainer(
    model=os.environ['TRAINING_MODEL'],
    train_dataset=data['train'],
    eval_dataset=data['validation'],
    peft_config=LoraConfig(
        r=8, lora_alpha=16, target_modules='all-linear',
        task_type='CAUSAL_LM',
    ),
    args=SFTConfig(
        output_dir='workshop-adapter',
        max_steps=10,
        max_length=512,
        per_device_train_batch_size=1,
        gradient_accumulation_steps=4,
        learning_rate=1e-4,
        assistant_only_loss=True,
        report_to='none',
    ),
)
trainer.train()
print(trainer.evaluate())
trainer.save_model('workshop-adapter/final')

Assistant-only loss requires a compatible chat template with generation spans. Inspect processed labels first. Ten steps are a pipeline smoke test, not a recommended training duration or evidence of learning. The sequence limit is also a pilot setting; verify it does not remove the desired responses.

Record installed package versions and lock the environment after a successful smoke test. Inspect trainable modules, peak memory, and a decoded batch. If the base weights do not fit, use the separately documented quantized preparation path rather than adding unrelated settings at random.

Decide whether the adapter earned deployment

Compare the fixed baseline and adapter on the same held-out inputs and supplied records. Score correct fields, unsupported claims, appropriate uncertainty, format validity, and response length separately. Inspect every case that changed from correct to incorrect.

Then load the saved adapter in the intended serving path and repeat the checks. Preserve the base revision, tokenizer, template, data version, and configuration. A lower validation loss is useful diagnostic evidence, but the release decision depends on the task outcomes.

Completion exercise: write a short decision: keep, revise, or reject the adapter. Support it with observed counts and examples. If the prompt baseline already performs as well, rejecting the adapter is a valid successful project outcome.

Next, investigate how a stronger teacher can help create training signals for a smaller student, and why those signals still need checking.

Sources

The integration follows the current TRL SFT Trainer interface and PEFT LoRA options. Check those references against the versions installed in your environment.

Continue: Distillation and Synthetic Data.