When is showing an example better than adding a rule?

You want short support messages labeled “question,” “problem,” or “feedback.” You can describe those labels, but what happens when a message both asks a question and reports a failure? Examples reveal the decisions hidden inside a label definition.

Before you begin: Know how to define a task and test a prompt against held-out inputs.

Show the mapping you want

Zero-shot prompting gives the task without worked task examples. Few-shot prompting includes a small number of input-output demonstrations in the context. In this setting, “learning” refers to behavior shaped by the provided context; it does not mean the model weights are being updated during the request.

Start with these demonstrations:

Label the message with one category.
Use problem when a current failure is reported, even if phrased as a question.

Message: What time does the session begin?
Label: question

Message: The joining link returns an error.
Label: problem

Message: I enjoyed the hands-on exercise.
Label: feedback

Message: Why does my joining link return an error?
Label:

The final message should be “problem” under the stated rule. The question mark does not override the reported failure. Without a clear precedence rule, another label may be defensible. Do not call a model wrong when the task definition itself is ambiguous.

Choose coverage over quantity

Three nearly identical compliments do little to clarify a boundary between labels. A useful example set shows the intended output format and the distinctions the model is likely to miss. Include an ordinary case, a boundary case, and a missing-information or out-of-scope case when those occur in the real task.

Demonstrations can also teach unintended shortcuts. If every “problem” example is long and every “question” is short, the model may pick up length as a cue. If examples contain personal details unrelated to the task, you add exposure without educational value. Use representative, non-sensitive examples and inspect their labels.

The number and order of examples can affect behavior. More examples consume context and may introduce contradictions. Measure whether an added example repairs the intended failure and whether it harms cases that previously worked.

Keep demonstrations and tests separate

Suppose you add a difficult message to the prompt and then count its correct label as an evaluation success. That checks whether the system can follow a displayed answer pattern. It is weak evidence about new messages.

Keep a held-out collection with different wording. For the joining-link example, test “I cannot enter the room; is there another address?” and “Where will you send the joining link?” The first reports a problem; the second asks about the process. Both concern links, so topic alone is insufficient.

Use examples for style too

For editing, show a before-and-after pair that preserves facts while changing style. A useful demonstration might convert a long event notice into a concise reminder without adding promises. An example that silently fixes a missing date teaches the wrong behavior if your actual requirement is to ask for missing dates.

The example is part of the specification. Read it as carefully as the written rules. A mismatch between the two leaves the assistant with inconsistent evidence about what you want.

Repair a demonstration set

Your prompt labels “The teacher was excellent” as feedback and “Was the teacher excellent?” as feedback. You intended the second message to be a question. What should you change, and how should you test the repair?

Next we will examine worked reasoning examples: when intermediate steps help, what they can reveal, and why a long explanation is not proof of a correct answer.

Further reading

Language Models are Few-Shot Learners studies in-context task demonstrations. Rethinking the Role of Demonstrations investigates how demonstrations communicate task structure and input distributions.

Continue to the next lesson.