Build a multi-step research agent

A research helper finds one page and immediately announces a finished brief. Another keeps searching forever. Your project needs a middle ground: a stated evidence requirement, bounded work, and an honest incomplete outcome when the requirement is not met.

Before you begin: Understand tools, state, and stopping conditions. Python 3 is enough for the local exercise.

Define a narrow research task

Use a small fictional collection of workshop notices. The goal is to find a venue and a start time, each supported by a source, then prepare a draft brief. This local project does not search the web or publish anything. It isolates the control loop before adding a model or external tools.

State should include the question, observations, remaining requirements, step count, and outcome. A plan such as “find the time” is not an observation. Only a tool result should add evidence, and the final check should inspect the actual evidence requirements.

What would count as useful progress?

Run a bounded reference loop

This complete program uses fixed tool results. The policy chooses which missing field to look up. It is a deterministic reference for an agent runtime, not an LLM agent or a benchmark.

# Runnable: Python 3, standard library.
def collect(tools, limit=3):
    evidence = {}
    trace = []
    for step in range(limit):
        missing = [key for key in ('venue', 'time') if key not in evidence]
        if not missing:
            return {'status': 'ready_for_review', 'evidence': evidence, 'trace': trace}
        field = missing[0]
        result = tools[field]()
        trace.append({'step': step + 1, 'field': field, 'result': result})
        if not result or not result.get('source') or not result.get('value'):
            return {'status': 'incomplete', 'evidence': evidence, 'trace': trace}
        evidence[field] = result
    status = 'ready_for_review' if len(evidence) == 2 else 'budget_exhausted'
    return {'status': status, 'evidence': evidence, 'trace': trace}

tools = {
    'venue': lambda: {'value': 'Room 4', 'source': 'N1'},
    'time': lambda: {'value': '10:00', 'source': 'N2'},
}
assert collect(tools)['status'] == 'ready_for_review'
assert collect(tools, limit=1)['status'] == 'budget_exhausted'
assert collect({**tools, 'time': lambda: None})['status'] == 'incomplete'
print(collect(tools))

The status says “ready for review” because source IDs and values are necessary but not sufficient for factual correctness. A real evidence checker must confirm that the sources support the values and are appropriate for the event.

Introduce a model at one decision point

If you extend the project, let a model propose the next allowed lookup or draft a brief from collected evidence. Keep the action schema narrow. Validate that a proposed tool exists and that its arguments fit the current task before executing it.

Do not give the model a general shell or unrestricted browser merely to retrieve two fields. Add capabilities only when the task requires them. Keep step and time budgets in application code so an untrusted response cannot remove them.

Test trajectories, not just final prose

Inspect the trace for repeated lookups, unsupported assumptions, and premature completion. A correct-looking final sentence can hide that the model never retrieved a required source. Conversely, an incomplete result can be appropriate when the source collection lacks a fact.

Add a conflicting-time fixture and decide how the policy handles it. It should preserve the conflict for review or resolve it using explicit source authority and version rules. Silently choosing the first value is not a general solution.

Extend one boundary at a time

Replace the in-memory fixtures with read-only retrieval before adding persistence or external actions. Then test restart behavior and state ownership. Publishing the brief, emailing it, or editing a calendar belongs to a later explicitly authorized stage.

Practice with feedback

MAX_STEPS = 12

def run(question, sub_questions):
    trace, claims, open_q = [], [], list(sub_questions)
    for _ in range(MAX_STEPS):
        if not open_q:
            break
        step = decide_next(question, open_q, trace)
        result = call_tool(step.tool, step.args)
        trace.append({'tool': step.tool, 'args': step.args,
                      'result_summary': summarise(result), 'ids': ids_of(result)})
        for c in extract_claims(result, open_q):
            claims.append(c)
            if c['answers'] in open_q:
                open_q.remove(c['answers'])

    status = ('answered' if not open_q else
              'partial' if claims else 'not_finished')
    return Outcome(status, claims, open_q, trace)


def test_every_claim_is_grounded():
    out = run(QUESTION, SUBS)
    seen = {i for step in out.trace for i in step['ids']}
    for c in out.claims:
        assert c['source_id'] in seen, f"ungrounded claim: {c['text']}"

def test_budget_exhaustion_is_honest():
    out = run(UNANSWERABLE_QUESTION, SUBS)
    assert out.status in ('partial', 'not_finished')
    assert out.open_questions, 'must say what it could not answer'

# The grounding test found the real bug in my first version: the summariser
# was merging two notices into one claim and attributing it to whichever id
# came first. The prose was fine. The trace was not.

Next, the RAG module will replace the tiny source collection with retrieval and show how to evaluate whether the right evidence reaches the model.

Sources

ReAct studies interaction between reasoning and actions. The LangChain agent guide documents a current runtime implementation; the local loop here is intentionally framework-independent.