The Agent Loop

Think, act, observe, repeat. The cycle is trivial to implement and the hard parts are all about when to stop.

The loop at the center of every agent is about fifteen lines:

messages = [system_prompt, user_request]

while True:
    response = model(messages, tools=available_tools)
    messages.append(response)

    if not response.tool_calls:
        return response          # model produced an answer

    for call in response.tool_calls:
        result = execute(call)
        messages.append(result)

Call the model. If it wants tools, run them and append results. Repeat until it answers instead.

That is genuinely all of it. Every difficulty is in the details around it.

Think, act, observe

Each iteration has three parts.

Think — the model examines the conversation and decides. Reasoning may be explicit, and asking for it helps for the same reason chain-of-thought helps: each token is another forward pass, and the loop makes that reasoning visible for the next iteration.

Act — a tool call is emitted and your code executes it.

Observe — the result is appended. The model now knows something it did not.

The accumulating message list is the agent’s memory. There is no other state. Everything the agent knows about its own progress lives in that transcript, re-sent in full on every iteration.

Stopping is the hard part

The loop as written above has no exit besides the model choosing to answer. That is not sufficient in practice.

Iteration cap. A hard maximum, always. Without it, a confused agent loops until you notice. Ten to twenty is typical.

Token budget. Context grows every iteration. Track it and stop before overflow rather than crashing into it.

Wall-clock timeout. Independent of iteration count, because individual tool calls can hang.

Repetition detection. The most common real failure: the same tool called with the same arguments repeatedly, because the result did not help and the model has no better idea. Detect and break — a loop-detection check on recent calls is cheap and catches a lot.

Explicit completion signal. Rather than inferring completion from absence of tool calls, give the model a finish tool with the answer as an argument. Now completion is a deliberate act you can validate.

The last one is worth adopting. “No tool call” as a completion signal conflates done with stuck, and those need different handling.

Context growth

Every iteration appends the model’s response plus every tool result. Ten iterations of verbose tool output can consume an entire context window.

Worse, the whole transcript is resent every iteration, so total token cost grows with the square of loop length. A twenty-step run is not twice a ten-step run.

Mitigations, roughly in order of effort:

Return compact tool results. The cheapest and most effective fix. A tool returning 10,000 tokens of JSON when 200 would do poisons everything downstream.

Truncate old results. Keep recent observations in full, compress older ones to a line.

Summarize periodically. Every n steps, replace history with a summary of progress and findings. Loses detail, extends the horizon.

Externalize state. Have the agent write findings to a file or scratchpad it can re-read, keeping the transcript short. More engineering, and the only approach that genuinely scales to long tasks.

Why reliability compounds downward

If each step succeeds 95% of the time, ten steps succeed about 60% of the time. Twenty steps, about 36%.

This arithmetic is the central constraint on agent design, and it explains why long autonomous runs remain unreliable regardless of prompt quality. Nothing at the prompt layer changes an exponent.

The responses that actually work: keep loops short, make individual steps more reliable rather than adding more of them, and checkpoint so a failure at step 8 does not discard steps 1 through 7.

Structuring around it

Include the goal every iteration. Long loops drift. The original objective sitting in the system prompt helps, and restating it explicitly helps more.

Log every iteration. The transcript of thoughts, calls, and results is your only debugging artifact. Agents fail in ways that are incomprehensible without it.

Make progress inspectable. For anything user-facing, stream what the agent is doing. Silence during a twelve-step run is unacceptable UX.

Gate the consequential steps. Read-only tools can run freely. Anything irreversible warrants confirmation — see How Agents Fail.

What to remember

• The loop is trivial: call model, execute tools, append results, repeat until it answers.
• The message list is the entire memory, resent every iteration.
• Stopping needs explicit machinery: iteration cap, token budget, timeout, repetition detection, and ideally a finish tool.
• Context grows quadratically with loop length; compact tool results are the highest-value fix.
• Reliability compounds downward — 95% per step is 60% over ten. Prefer short loops and reliable steps.

Next: Writing Good Tool Definitions