Coding Agents

The agent application that works best in practice, and the specific properties of software work that make it work.

Coding agents work better than most other agent applications. That is not because code is easy — it is because software development happens to have four properties that agents need.

Understanding those properties tells you where else agents will work, and where they will not.

Why code is favourable

Verification is automatic. Tests pass or fail. Code compiles or does not. The agent gets a ground-truth signal after every action, without a human in the loop. This is the big one — most domains have no equivalent.

Actions are reversible. Version control means a wrong edit costs a revert. Compare with sending an email or issuing a refund, where confident wrong action is unrecoverable.

Feedback is fast. Seconds, not days. Tight loops make multi-step work viable.

Training data was abundant. Public repositories, issues, and reviews are enormous, and code has consistent structure.

Where a domain lacks these — no automatic verification, irreversible actions, slow feedback — agents degrade sharply. That is the transferable lesson.

What the tools are

A coding agent is an agent loop over a small set of tools: read a file, search the codebase, edit a file, run a command, run tests.

Two details matter more than the list.

Search quality drives everything. An agent that cannot find the relevant code cannot fix it. Good search — by symbol, by pattern, across the repository — is the difference between an agent that works and one that flails. This is the most underrated component.

Running tests is the highest-value tool. It converts guessing into verification. An agent that can run the test suite self-corrects; one that cannot is writing code blind.

Where they do well

Well-specified changes. Rename this, add this parameter everywhere, extend this pattern to these cases. Mechanical work across many files.

Test-covered bug fixes. A failing test is a perfect objective — the agent knows exactly when it is done.

Boilerplate. New endpoints, migrations, test scaffolding matching existing patterns.

Codebase questions. Where is this implemented, what calls this, how does this flow work. Search plus reading, no editing.

Incremental refactoring within one clear pattern.

Where they do badly

Ambiguous requirements. “Make this faster” without a target produces plausible changes with unknown effect.

Architectural decisions. Tradeoffs depending on context the agent does not have — team conventions, roadmap, operational constraints.

Untested code. Without verification, the loop’s main advantage disappears. The agent produces code that looks right, which is the dangerous middle.

Large cross-cutting changes. Coordinated modification across many files exceeds what reliability compounding allows in one run.

Subtle correctness. Concurrency, numerical precision, security-sensitive logic. Code that passes tests and is wrong.

Working with them effectively

Specify verification up front. “Make the failing test in auth_test.py pass” is a far better instruction than “fix authentication.” Give the agent a way to know it succeeded.

Keep scope narrow. One well-defined change per run. Reliability degrades with step count, so short runs succeed more.

Review the diff, not the explanation. The agent’s account of what it did is generated text and not a reliable audit trail. Read the actual changes.

Provide context explicitly. Conventions, patterns to follow, files to avoid. Many project-level tools support a persistent instructions file for exactly this — it is the highest-value setup step.

Expect the plausible-but-wrong failure. Code that compiles, passes tests, and misses an edge case is the characteristic output. Off-by-one errors, unhandled nulls, and missed cases survive verification.

Gate the consequential commands. Reading and editing are cheap to undo. rm -rf, force pushes, and deploys are not. See Human-in-the-Loop Design.

What to remember

• Coding agents work because software has automatic verification, reversible actions, fast feedback, and abundant training data.
• Domains lacking those properties are where agents degrade — that is the transferable lesson.
• Search quality and test execution are the two components that matter most.
• Strong on specified changes, test-covered fixes, boilerplate, and codebase questions.
• Weak on ambiguity, architecture, untested code, large cross-cutting changes, and subtle correctness.
• Give an explicit success criterion, keep scope narrow, and review the diff rather than the explanation.

Next: Computer-Use Agents