Design five small AI tools

A code explainer, an email drafter, a document helper, an image brief writer, and a study partner look like five products. What stays the same when you strip away their interfaces? Each needs a clear input, a transformation, and a way to judge the result.

Before you begin: Know the basic API flow and how to check a response against its task.

Compare the jobs

For a code explainer, the evidence is the supplied program and its environment. The response should distinguish observed behavior from assumptions. For an email draft, the evidence is the sender's facts and intent; the tool should not invent promises or send the message without authorization.

For document questions, answers need passages that support them. For an image brief, the output is a creative instruction rather than factual evidence about a real scene. For a study partner, a useful question and feedback matter more than flattering the learner's answer.

These differences belong in the task definition, not only in five different button labels.

Work one task end to end

Choose a document helper. Supply a fictional notice and a question. The application prepares the prompt, calls a model if configured, receives a response, validates the structure, and displays the answer with its evidence.

Start by testing validation with a fixed response. A fixed response is a fixture, known data used to test application behavior. It is not a live model answer, and the interface should never claim that inference happened when it did not.

What boundary can all five tools share?

Run a small validator

This complete Python 3 program checks that a proposed quotation appears in the source. It does not call a model. Its purpose is to reveal what a simple check can and cannot establish.

# Runnable: Python 3, standard library.
notice = 'The class starts at 10:00. Bring a notebook.'

def quote_is_present(source, response):
    quote = response.get('quote')
    return isinstance(quote, str) and bool(quote.strip()) and quote in source

good = {'answer': 'Bring a notebook.', 'quote': 'Bring a notebook.'}
invented = {'answer': 'Bring paint.', 'quote': 'Paint is provided.'}
misleading = {'answer': 'The class starts at 12:00.',
              'quote': 'The class starts at 10:00.'}
assert quote_is_present(notice, good)
assert not quote_is_present(notice, invented)
assert quote_is_present(notice, misleading)
print('A present quote can still fail to support the answer')

The final assertion is intentional. A quotation-presence check verifies the quotation, not the relationship between the quotation and the answer. You still need an entailment or human evidence check. Naming the limitation prevents a small useful validator from becoming a false guarantee.

Adapt the validation to the other tools

For an email draft, compare names, dates, and commitments against supplied facts. For a code explanation, run a small example in the documented environment when safe and appropriate. For a study question, verify that the answer key follows from the lesson. For an image brief, check required subject and constraints rather than grading it as a factual report.

Do not reuse one generic “AI quality score” for all five. A score based on response length or the presence of certain words can look objective while measuring little about the actual task.

Add the live boundary deliberately

Once the local flow works, replace the fixture with an actual provider response using the earlier API lesson. Keep the credential on the server or in the local process environment. Show loading, failure, and incomplete-response states honestly. Record which examples were tested locally and which require external services.

Build one tool before expanding to five. The comparison teaches reusable design questions; it does not require five unfinished applications in a single sitting.

Choose the next check

Your document helper always returns valid JSON with a quotation copied from the notice. Is it ready to answer users' questions?

Practice with feedback

import re
from dataclasses import dataclass, field

@dataclass
class Result:
    ok: bool
    problems: list = field(default_factory=list)


def validate_extraction(source, output, required):
    problems = []
    for key in required:
        if key not in output:
            problems.append(f'missing key: {key}')
    for key, value in output.items():
        if isinstance(value, str) and value != 'NOT STATED' and value not in source:
            problems.append(f'{key}={value!r} does not appear in the source')
    return Result(not problems, problems)


def validate_translation(source, output):
    problems = []
    for number in re.findall(r'\d[\d:.,]*', source):
        if number not in output:
            problems.append(f'lost number: {number}')
    for name in re.findall(r'\b[A-Z][a-z]{2,}\b', source):
        if name not in output:
            problems.append(f'possibly lost name: {name}')
    return Result(not problems, problems)


src = 'Advanced Sewing runs Tuesdays 18:30-20:00 in Room B.'
print(validate_extraction(src, {'day': 'Tuesdays', 'start': '18:30'}, ['day', 'start']))
# Result(ok=True, problems=[])
print(validate_extraction(src, {'day': 'Wednesdays', 'start': '18:30'}, ['day', 'start']))
# Result(ok=False, problems=["day='Wednesdays' does not appear in the source"])
print(validate_translation(src, 'Couture avancee le mardi de 18:30 a 20:00 en Salle B.'))
# Result(ok=False, problems=['possibly lost name: Advanced', ...])
#
# The name check is deliberately noisy: it flags candidates for a human
# rather than claiming certainty. A validator that is quietly wrong is
# worse than one that is loudly unsure.

Next, we will put these pieces into an application design with durable state and explicit responsibility for each boundary.

Further reading

The original RAG paper motivates combining retrieval with generation. JSON Schema's getting-started guide explains structural validation, which is distinct from factual validation.