{/* banner prompt: High-key cinematic shot in a vast, minimalist white space with atmospheric haze; a single, compact, pristine cube on the floor projects a focused, coherent beam of light that forms a perfect code structure on a far wall, while much larger, ghostly, and fragmented machines dissolve into the bright background, 16:9, volumetric lighting, sense of scale and clarity. */}

The first five posts in this series were the telescope and the microscope: what agents [are](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/), what a [harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), how they [learn](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/), which one [wins](/blog/2026/06/13-the-coding-agent-shootout-and-the-best-harness-we-could-not-read/), and how one [actually writes a line of code](/blog/2026/06/13-how-a-coding-agent-actually-writes-code/). All of it was *reading*, of code and behaviour. This post is different: we stopped reading and **ran the experiment.**

The question: on cheap, local hardware, specifically a 4GB laptop GPU, **what actually makes a small model a good coding agent?** Not "which frontier model wins" (you can't run those at home), but: given a model that fits in 4 GB, which *harness choices* matter? Edit format? Retry? Task decomposition? A planner model? We built a measurement harness, wrote a suite of coding tasks with automated pass/fail oracles, and ran roughly **900 trials**. The results overturned most of our intuitions.

## The setup

A gaming laptop with an **RTX 3050 (4 GB VRAM)** serving models via Ollama; a separate machine running the agent and holding the code. The harness runs each task in a fresh copy of its workspace, lets the model attempt it, then grades it with a hidden `pytest` oracle, so the test is the only judge, so every result is automated and reproducible.

The **task suite** is 28 self-contained coding tasks: add a function, fix a bug, small refactors, implement-from-spec, multi-file changes, and a deliberately hard tier (a bowling scorer, a circular buffer, rational-number arithmetic, a 24-hour clock, a book-store discount optimiser, variable-length-quantity encoding). Each oracle was checked two ways. It must *fail* on the starting code and *pass* on a correct solution. Calibrated this way, the simplest configuration solves about 71%, leaving real room to measure whether the fancy techniques help.

We varied **one lever at a time** around that baseline (`qwen2.5-coder:3b`, whole-file edits, single-shot), ran each task/config combination three times to smooth out the noise of a non-deterministic model, and recorded success rate, tokens, and wall-time.

## The results

<Table data={
    {
        headers: ['Config', 'Success', 'Sec/run', 'What changed vs baseline'],
        rows: [
            ['baseline (coder-3b, whole-file)', '71%', '2.6', 'reference'],
            ['retry (run test, feed failure back)', '70%', '8.6', 'no change'],
            ['decomposition (plan then execute)', '64%', '5.3', 'hurt'],
            ['role-split (planner + editor)', '66%', '134', 'hurt + 50x slower'],
            ['model: qwen3:1.7b', '43%', '50', 'much worse coder'],
            ['model: granite3.3:2b', '39%', '2.2', 'much worse coder'],
            ['edit format: diff', '24%', '5.9', 'far worse'],
            ['edit format: native tool-calls', '0%', '2.1', 'impossible'],
        ]
    }
} />

Read that table for a minute, because almost every row is a surprise.

## Finding 1: the edit format is the dominant lever

How the model is asked to express a change matters more than anything else. Whole-file rewrite: **71%**. Search/replace diff: **24%**. Native function-calling: **0%**.

That **0%** deserves a pause. We gave the model proper OpenAI-style tools and asked it to call them, and across 84 attempts it never once produced a usable tool call. It *knew what to do*: it emitted perfectly correct JSON like `{"name": "read_file", "arguments": {"path": "config.py"}}`, but as a markdown code block in its **message text**, never in the structured `tool_calls` field the protocol requires. A native-tool agent (Codex, Pi, anything function-calling-based) sees an empty tool-call list and does nothing. **The small coder model literally cannot drive a native-tool agent.**

Diff (24%) is a softer version of the same lesson: the model produces well-formed search/replace blocks, but on longer files the "search" text drifts from the source by a character and the edit won't apply. Whole-file editing sidesteps both problems: there's nothing to match, nothing to format. For a small model, **make the easy thing the only thing.**

## Finding 2: "best coder" is not "best tool-caller"

We tried three models that fit in 4 GB. The coding-specialised `qwen2.5-coder:3b` (71%) crushed the newer general models `qwen3:1.7b` (43%) and `granite3.3:2b` (39%) at producing correct edits.

Here's the twist: **qwen3:1.7b is the *better tool-caller***, able to emit native `tool_calls` and drive an agent loop where the coder model can't, yet it's the *worse coder*. Tool-calling ability and coding ability are different axes, and a model can be strong on one and weak on the other. If your harness leans on native tool-calling, you're implicitly selecting for the tool-caller and possibly against the coder. (qwen3:1.7b is also strikingly slow and verbose here, at ~50 s/run and ten times the output tokens, despite being told not to "think.")

## Finding 3: the clever agentic scaffolding didn't help, and mostly hurt

This is the one that stings, because it's the stuff everyone reaches for.

- **Retry** (run the test, feed the failure back, let it fix): **neutral** (70% vs 71%). Retry recovers *near-misses*. But this suite's failures are the hard tier (bowling, VLQ) which are *capability-bound*: the 3B can't write a correct bowling scorer no matter how many times you show it the failing test. **Retry can't buy capability the model doesn't have.**
- **Decomposition** (plan the task into steps, then execute): **−7 points**. The planning step adds tokens and structure a small model handles poorly; a slightly wrong plan actively misleads it.
- **Role-split** (a stronger `qwen3:4b` planner directing the coder): **−5 points and ~50× slower**. On 4 GB the two models can't co-reside, so the GPU thrashes swapping them in and out. A separate planner didn't help and cost two minutes a task.
- **Stacking** the "improvements" together regressed *below* baseline, because the negative levers compound.

The pattern: orchestration tricks that help a frontier model coordinate **do not rescue a small model whose failures are a capability ceiling, and several make it worse** by adding surface area for a limited model to trip over.

## The follow-up: can we craft tool-calls the small model *can* do?

Finding 1 left a loose end. Native tool-calling scored 0% only because the model emits tool intent as *text*, not in the `tool_calls` field. So we asked: what if we meet the model where it is, building our own tool-call protocol that parses the JSON-in-content it naturally produces, and give it a real agent loop (read → edit → run → fix)?

We did. A lenient parser (accepting fenced JSON, `<tool_call>` tags, or bare JSON), a small tool set, and a loop that hides the grading test until the end. Two results:

1. **A custom text protocol rescues tool-calling: 0% → ~56%.** Meeting the model where it is *works* where the rigid native protocol is impossible. The 3B genuinely drives a multi-step loop.
2. **But the loop still loses to single-shot, and its value depends on the model.** On the strong coder it *hurt* (71% single-shot → 56% in the loop: more turns, more chances to go wrong). On the weaker-but-tool-capable qwen3:1.7b it *helped* (43% → 58%). Neither beat plain coder-3b doing one whole-file edit.

So you *can* engineer a small model into being an agent at all, and that's a genuinely useful capability, but the engineering doesn't beat the simplest possible setup.

## What it adds up to

For a small model on scoped coding tasks, the high-leverage choices are the **editor model** and the **edit format**, not the agentic scaffolding. The recommended 4 GB configuration is almost embarrassingly plain: **`qwen2.5-coder:3b`, whole-file edits, single-shot.** 71% at under three seconds a task. Everything fancier was neutral, slower, or worse.

This both confirms and sharpens the series' running thesis. We've said all along that *the harness, not the model, is where the engineering lives*, and that's still true: the harness's edit-format and model choices made the difference between 0% and 71%. But there's a hard limit the experiment made vivid. A harness can fix problems of **interface**, giving the model an edit format and a tool-call protocol it can actually use, and the payoff is enormous. It cannot fix problems of **capability**: when the model simply can't write the bowling scorer, no amount of retry, planning, or multi-model orchestration conjures the answer, and the cleverer the scaffolding, the more often it backfires. With a small model you feel that ceiling directly, and the winning move is to stop adding machinery and get out of the model's way.

There's a nice corollary for anyone running local AI on modest hardware: you don't need an elaborate agent framework to get real value from a 4 GB GPU. You need the right small coder model, whole-file editing, and the discipline to keep it simple.

*The harness for this study (the task suite, the levers, the custom tool-call loop, and the full results) is open source. This experiment grew out of trying to give a real coding agent a local brain; the surprise was how little the brain needed around it.*

616338c19d78232f96d954bbd1e72488e9bee452987f2c1f76d9ea6f9d3c94a9

An empirical study of small-model coding agents on a 4 GB RTX 3050: edit format, model choice, retry, decomposition, role-split, and a custom text tool-call loop, measured across ~900 automated trials.

We stopped theorizing and ran ~900 trials on a 4 GB laptop GPU to measure what makes a small local model a good coding agent. The result overturns intuition: the edit format and the editor model decide almost everything, the clever agentic scaffolding mostly hurts, and the simplest setup won.

The 4 GB Experiment: What Actually Makes a Small Model a Good Coding Agent

{/* banner prompt: Warm, realistic analog photography of a cluttered but organized workshop desk, where a long ticker tape of computer output unspools from a vintage telegraph machine, with sections circled in red ink and handwritten notes in the margins, representing an AI agent debugging code through text logs, all in soft, natural window light, 16:9. */}

We called the [previous post](/blog/2026/06/13-the-coding-agent-shootout-and-the-best-harness-we-could-not-read/) the last in the series. That was the telescope view: what agents *are*, what a [harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), how they [learn](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/), which one wins. But a developer doesn't experience an agent as an architecture diagram. You experience it one edit at a time: it changes three lines, runs the tests, reads the failure, tries again. So here is the microscope view: how a coding agent actually turns an instruction into a working, tested change, held up against how a human programmer does the same thing.

## The human inner loop

Picture your own setup. You have an editor where you type, a terminal where you compile and run, and some way to *see the result*: a test runner's green bar, a printed value, a web UI you reload in a browser. Around it sits the big lifecycle everyone draws on whiteboards: idea → plan → architecture → design → write. But that's the outer loop. The thing you actually do all day is the **inner loop**, and it's tight: write a little, run it, look at what happened, adjust. Seconds per cycle.

Two habits define how humans fill that loop. First, we mostly **build small and grow** (a few lines, run, a few more), keeping a working program at almost every step, rather than generating a whole finished file and hoping. Second, we choose a direction: **vertically** (one feature sliced end-to-end through every layer until it runs) or **horizontally** (one layer built out across features). Either way the program is *alive* in front of us the whole time. We glance at the screen and instantly know if it's broken.

That last point is the one that matters most for what follows, because it's exactly the sense an agent doesn't have.

## The agent inner loop

An agent runs the same shape of loop (instruction in, change out, observe, repeat), but through a fundamentally narrower keyhole. Its loop is: the harness sends the model the context, the model emits a **tool call** (edit this file, run this command), the harness executes it and feeds the **text result** back, and the model decides what to do next. That's it. The agent has no eyes. It cannot glance at a screen. **Its only senses are the text that tools return**: file contents, stdout, stderr, an exit code. Everything a human takes in visually (the rendered web page, the debugger paused on a line, the red squiggle under a typo) has to be converted into text and handed back, or it doesn't exist for the agent at all.

This single fact explains almost everything distinctive about how agents code. It's why output truncation matters (an 80,000-line log blinds it as surely as a wall of noise), why edit-acceptance is fiddly, why verification has to be deliberate, and why debugging, the most *visual* and interactive thing a human does, is the hardest part to translate.

It also shifts the economics of "small and grow." Each model round-trip costs real tokens and seconds, so an agent can't afford the human's keystroke-level incrementalism; it tends to write a whole function or file in one shot, then verify. The good ones pull back toward small steps anyway (write, run, read, fix), because a 200-line blob that fails gives the model almost nothing to reason about, whereas a 20-line change that fails points right at the problem. The tension between "big chunks are cheaper per round-trip" and "small chunks are debuggable" is the central rhythm of agentic coding.

## The crux: turning intent into a changed file

Here is the most under-appreciated micro-mechanic in the whole stack: how does the model's *intention* to change code become an actually-changed file on disk? This step, call it **output acceptance**, is where coding agents most visibly differ, and where they most often fail. The five harnesses we read take three distinct approaches:

- **Exact string replacement.** The model supplies the old text and the new text; the tool finds the old verbatim and swaps it. This is what Claude Code's `Edit` and Pi's `edit` do. It's surgical and the resulting diff is trivially reviewable, but it's brittle: if the model's remembered snippet is off by a space, the match fails. Pi softens this with a *fuzzy fallback* that normalises smart quotes, Unicode dashes and trailing whitespace before giving up.
- **A patch grammar.** The model emits a structured diff in a format it was trained on, such as Codex's `*** Begin Patch … *** End Patch` `apply_patch` syntax. This handles multi-hunk edits cleanly and reads like a real patch, at the cost of the model having to produce exactly-valid patch syntax.
- **Self-correcting replace.** Gemini CLI's `replace` tool, on a failed match, fires a follow-up model call to *repair its own edit* rather than just erroring, trading tokens for a higher landing rate.

A failed edit is the single most common micro-failure in agentic coding, and each of those designs is one harness's answer to it. It's also exactly where your "native programmer versus scripter" observation enters, but that deserves its own section below.

## Calling the tools: compile, run, build, spawn

Once a file is changed, the agent needs hands to *do* something with it, and those hands are a single tool: shell execution (`Bash` in Claude Code and Pi, `exec`/unified-exec in Codex, a PTY shell in Gemini and Kimi Code). Through that one door it compiles, runs the test suite, starts a build, installs a dependency, spawns a server.

The interesting wrinkles are all about state and blindness:

- **Statefulness.** A naive `subprocess.run` can't hold a REPL open or answer an interactive prompt. Codex's "unified exec" gives the model a persistent PTY it can write stdin to; the mature shells track background processes so the agent can start a dev server and keep working while it runs.
- **Truncation.** Every harness caps tool output (Kimi Code's read tool stops at 1,000 lines or 100 KB; Pi spills overflow to a temp file). Without it, one chatty build destroys the context budget.
- **The blind spot, literally.** The agent can `npm run dev` and start a web UI, but it cannot *look* at the page. So it `curl`s the endpoint, greps the logs, or, increasingly, drives a headless browser and feeds itself a screenshot through a vision model. Watching an agent start a server it fundamentally cannot see is the clearest reminder of the text-keyhole problem.

## Tests and verification

Because the agent can't eyeball correctness, **running things is its substitute for seeing**, which makes tests and verification not a virtue but a necessity.

On creating tests, agents are comfortable both ways: writing the test first and coding to green (some skills enforce test-first explicitly), or writing tests after to lock in behaviour. Their real edge is that they are tireless about *running* the full suite, the linter and the type-checker on every change, the chores humans skip when tired.

Their matching weakness is that a language model will cheerfully announce "done" without having run anything. This is why mature harnesses and skills bolt on a **verification discipline**: re-read the file you just edited, run the build, run the tests, and only then claim success: evidence before assertion. The agent's superpower (re-run the world twenty times without complaint) and its failure mode (declare victory unverified) are two sides of the same coin, and verification gates exist to keep it on the right side.

## Debugging: the widest gap of all

Now the part of the inner loop that translates worst. Here is the canonical human bug hunt, in \*nix muscle memory:

1. `grep` the error string across the tree.
2. Open the file at the offending line, read the surrounding context.
3. Set a **breakpoint**.
4. Run, and when execution pauses, **step** through it (step over, step into, step out), watching variables change.
5. Catch the moment the state goes wrong, and you've found the cause.

An agent does steps 1 and 2 almost exactly as a human does: grep the error, read the context around the line. Then the methods fork hard, because **step 3 is mostly unavailable to it.** Agents rarely drive an interactive debugger; they can't easily sit at a paused frame and inspect live state. (Some can speak the Debug Adapter Protocol through an MCP server, but it's the exception, not the daily path.) So instead of stepping, the agent debugs the way we all did before debuggers existed, just faster: it inserts `print`/log statements, re-runs, reads the output, reasons about it, removes them. Printf debugging at machine speed.

The deep difference is one of *substrate*. A human debugs a **live, paused, inspectable** program. You can poke it while it's frozen mid-thought. An agent debugs a **dead transcript** of what the program already emitted. It loses the ability to interrogate a running frame, and it compensates with two things humans can't match: it will happily re-run the failing case fifty times with different instrumentation, and it can hold the entire call path in context at once rather than stepping through it line by line. Slower per insight, but relentless and wide.

## The observation: native programmer vs. scripter

Which brings us to the thing you actually notice after watching these agents work for a while: **they have different programming temperaments, and Claude Code behaves the most like a native \*nix programmer.**

By "native" we mean it reaches, by default, for the same surgical tools a seasoned terminal user does: `grep` to locate, a targeted read of the exact line range, a precise string-replace edit, the right small command for the job. And it leaves behind a clean, reviewable trail of minimal diffs. Watching it is like watching someone who has `grep`, `sed`, `cut`, `head` and `tail` in their fingers and uses them with intent.

The contrast: when faced with a bulk change (rename this symbol across forty files, rewrite every import), some agents (Codex and Gemini among them, in our observation) are more inclined to **write a throwaway script to do it**: a `python3 -c "..."`, a `perl -pi -e`, a `sed -i` one-liner run through the shell, rather than a series of surgical edits. It works, and for genuinely mechanical mass-edits it's even the *right* call. But it has a cost: a thrown-away script's effect is harder to review than a diff, since you have to reconstruct what it did, and a slightly wrong regex can silently mangle forty files at once, where a string-replace edit simply fails loudly on the one it couldn't match.

Two honest caveats, because this is the part most easily overstated. First, it's a *tendency*, not a wall: every one of these agents can use native tools, and every one can write a script; the difference is which it reaches for first, and that's shaped as much by the model's temperament as by the harness's tool design (a patch-grammar editor nudges toward batch thinking; a string-replace editor nudges toward surgical thinking). Second, these behaviours drift from model version to model version, so any snapshot like this one has a short shelf life. The durable point isn't "Claude Code wins the bug hunt." It's that **there is a real, observable spectrum between surgical and scripted code-editing, and it changes how reviewable and how safe the agent's work is.** When you pick an agent, that temperament is something you can feel within an hour, and it matters more to the day-to-day than any benchmark.

There's a fitting irony to end on: this post about how agents wield `grep` and surgical edits was itself drafted by one, reaching for exactly those tools as it went. The microscope, it turns out, works on the thing holding it.

## Where micro meets macro

Across five posts the same theme keeps surfacing from different distances. The harness (macro) decides which tools exist and how their results come back; the model (micro) decides which tool to reach for and how big a bite to take. A great coding experience is when those align: a native-feeling toolset in the harness, and a model with the judgement to be surgical when a scalpel is called for and to script when a bulldozer is. The model sets the ceiling; the harness and the editing temperament decide how close to it you actually get to live, one edit, one test, one bug at a time.

*This is a micro-level companion to our four-part series on AI agent architecture. All six harnesses behind these observations are open source; the best way to see the difference between a surgical edit and a thrown-away script is to watch two agents solve the same task and read what each left behind.*

37e17e97233f7397bb5ccb79b71f8fd079867b511dcd41109dbbb81fe5f3bcc2

A micro-level look at how AI coding agents write, run, test, verify and debug code: the edit loop, output acceptance, the exec tool, and why some agents feel like native command-line programmers while others feel like scripters.

The first four posts in this series took the telescope view of AI agents. This one takes the microscope: how a coding agent turns an instruction into a working, tested change (how it edits, compiles, runs, tests and debugs), and why Claude Code behaves like a native *nix programmer while others reach for a throwaway script.

How a Coding Agent Actually Writes Code

{/* banner prompt: A retro technical poster in the style of a 1970s engineering manual, diagramming five different abstract 'coding agent' mechanisms, with bold sans-serif typography labeling 'THE HARNESS' and 'MODEL CORE', using a limited-print color palette of navy blue and burnt orange on a cream paper texture, 16:9. */}

This is the fourth and final post in our series on AI agent architecture. The first three were about *how* agents are built: we [compared three of them](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/), dissected [what a harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), and looked at [skills and memory](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/). This one answers the question everybody actually asks first: **which coding agent should I use?**

We read the source of five open coding agents end to end: OpenAI's [Codex CLI](https://github.com/openai/codex) (the Rust harness is open even though the models aren't), [Pi](https://pi.dev/), [OpenCode](https://github.com/sst/opencode), Moonshot's [Kimi Code](https://github.com/MoonshotAI/kimi-code), and Google's [Gemini CLI](https://github.com/google-gemini/gemini-cli). Below is the honest shootout: strengths, weaknesses, and a ranking. But there's an asterisk we owe you up front, so here it is in plain sight. **The best harness in the field belongs to a tool we deliberately did not put under the microscope: Claude Code.** We'll get to why at the end.

(We're leaving [Hermes Agent](https://github.com/NousResearch/hermes-agent) mostly out of this one. It can code, but it's an assistant-first agent, and judging it as a coding tool would be judging a Swiss Army knife by its corkscrew.)

## How to judge a coding agent

A coding agent's quality is the product of two things that multiply rather than add: **the model** doing the reasoning, and **the harness** that decides what the model sees, what it can touch, and what happens when it's wrong. A great model in a sloppy harness flails on long tasks; a great harness can't make a weak model reason past its ceiling. So the axes that matter are:

- **Model strength on hard tasks:** can it actually fix the gnarly bug, not just the toy one?
- **Harness reliability:** does it survive a forty-minute run: compaction, recovery from loops, checkpoints?
- **Safety:** what stops it running `rm -rf` or leaking a secret?
- **Extensibility & ownership:** can you bend it, script it, run your own models?
- **Cost and friction:** what does a day of heavy use actually cost?

No single agent wins all five. That's the whole point.

## The contenders

**OpenAI Codex CLI**: *the strongest engineer in the room.* Codex pairs the best coding models available (the GPT-5.x-Codex family) with a harness that takes the hard problems seriously. It's the only one of the five where **sandboxing is architecture, not an afterthought**: every command runs inside an OS-enforced jail (Seatbelt on macOS, bubblewrap on Linux) with network denied by default, and the model can *request* escalation as part of its tool schema. The core is a protocol-first Rust engine, so one codebase drives the terminal, the IDE extension, `codex exec` and the cloud. And because OpenAI trains the models *with* this harness (they speak its custom patch format natively), model and harness reinforce each other. It's also the most token-efficient of the lot. Weaknesses: the models are closed and get deprecated on OpenAI's schedule, the cloud model isn't selectable, and its extension story (hooks, skills, plugins) is younger than its rivals'.

**Gemini CLI**: *the most feature-complete, and free.* Google's CLI is the most generous on-ramp in the field: a real free tier, a 1-million-token context window, and a genuinely impressive harness: loop detection that *injects corrective guidance* instead of just giving up, automatic fallback to a faster model on quota exhaustion, shadow-git checkpointing with `/rewind`, an A2A server for multi-agent setups, a VS Code companion, skills, hooks and subagents. It has quietly absorbed the best ideas from everyone. Weaknesses: it's at its best on Gemini models, the feature surface is sprawling, and on the hardest reasoning tasks community sentiment still places it a notch below Codex and Claude.

**OpenCode**: *the architecture astronaut's pick, in a good way.* OpenCode is built server-first: an HTTP server with an OpenAPI spec, and the TUI, desktop app, web UI and GitHub bot are all thin generated-SDK clients of it. It's aggressively provider-agnostic (bring any model), expresses "plan mode" and sub-agents as plain **permission rulesets** rather than hard-coded modes, ships git-based checkpoint/revert, and is mid-flight on an event-sourced V2 redesign. Weaknesses: that rewrite means churn, there's no sandbox, and your experience is only as good as the model you plug in.

**Kimi Code**: *the cleanest layering, and a cost story.* Moonshot's CLI is the best-factored codebase of the group, built as three crisp layers (`kosong` for LLM abstraction, `kaos` for OS abstraction, `agent-core` for the loop) with event-sourced persistence you can replay in a visual debugger and cache-aware compaction that weighs prompt-cache economics before squashing history. Kimi models are strong coders and cheap, and it speaks ACP for editors like Zed. (A fun detail: its TUI is built on Pi's own UI library.) Weaknesses: it's tuned best for Kimi models, it's newer, and the community around it is still small.

**Pi**: *not really a product; a masterclass.* Pi ships zero models and refuses features on principle (no sandbox, no permission popups, no plan mode), pushing everything to a TypeScript extension system. What it *does* ship is the cleanest minimal agent core we read (about 2,700 lines), the best multi-provider abstraction (40-plus providers behind one interface), and a session-as-a-tree design that makes branching and time-travel free. Weaknesses as a daily driver: no safety rails, no cross-session memory, a bus factor of roughly one, and quality that depends entirely on the model you bring. You run Pi to *understand* agents, or to build your own on top of it, which plenty of people, including the OpenClaw and Kimi Code teams, have done.

<Table data={
    {
        headers: ['Agent', 'Strongest at', 'Weakest at', 'Open?'],
        rows: [
            ['Codex CLI', 'Model strength, sandboxing, reliability', 'Closed models, younger extensibility', 'Harness yes, models no'],
            ['Gemini CLI', 'Free tier, features, recovery, 1M context', 'Best only on Gemini; sprawling', 'Fully open'],
            ['OpenCode', 'Client/server architecture, ownership', 'Mid-rewrite churn, no sandbox', 'Fully open'],
            ['Kimi Code', 'Clean layering, event sourcing, cost', 'Kimi-first, young, small community', 'Fully open'],
            ['Pi', 'Minimal core, provider abstraction', 'No safety, no memory, bus factor ~1', 'Fully open'],
        ]
    }
} />

## A note on lineage: who built on whom

It's tempting to read these as five isolated inventions. They aren't. This is a small field whose teams read and reuse each other's code, and the family tree is worth tracing because it tells you where the reusable ideas actually live.

The clearest root is **Pi**. Its "toolkit, not product" philosophy isn't just rhetoric. Other agents are literally built from its parts. **Kimi Code did not write its own terminal UI; it depends on `@earendil-works/pi-tui`, Pi's own TUI library**, and built its component layer on top (you can see it right there in `apps/kimi-code/package.json`). And **OpenClaw**, one of the most popular open personal assistants, is built on Pi's agent packages; its whole agent stack is Pi underneath. So while Pi loses a head-to-head as a *daily driver*, it quietly wins as *infrastructure*: when you use Kimi Code or OpenClaw, you're partly using Pi.

Then there's the softer lineage of shared conventions, where an idea from one agent becomes a standard everyone adopts:

- **`AGENTS.md`**, the project-instructions file, was originated by Codex and is now read by Pi, OpenCode, Gemini CLI (as `GEMINI.md`) and others.
- **The [agentskills.io](https://agentskills.io) `SKILL.md` format** is shared, independently, by Pi, Hermes, OpenCode, Kimi Code and Gemini CLI: five teams converging on one open standard.
- **Shadow-git checkpointing** (snapshot before risky edits, `/rewind` to undo) shows up in both OpenCode and Gemini CLI using the same underlying git trick.
- **The read / edit / grep / glob / bash tool ergonomics** that all five share trace largely back to Claude Code, which is part of why, as we'll see, it casts a long shadow over a comparison it isn't formally in.

The lesson for anyone building an agent: you are not starting from a blank page. The good patterns are already open, already converging, and in Pi's case already packaged as libraries you can import.

## The verdict among the five

Of the agents we actually read the code for, **Codex is the best coding agent**, and it isn't especially close on the axis that matters most for serious work. Hard engineering is where the model-times-harness product is most punishing, and Codex maxes both terms: the strongest coding models in the industry, co-designed with a harness that sandboxes by default and is built to grind through long autonomous runs. If your test is "fix the difficult bug in an unfamiliar codebase without supervision and without `rm -rf`-ing my home directory," Codex is the safe bet.

That doesn't make it the right pick for everyone. If you want zero cost and maximum features, Gemini CLI. If you want to own the whole stack and run your own models, OpenCode or Pi. If you live in Kimi's ecosystem and care about cost, Kimi Code. "Best" is conditional on what you're optimising for, but if you force a single ranking on *raw coding capability*, Codex tops the five.

## The asterisk: the best harness we couldn't read

Here's the honesty the title promised. We left **Claude Code** out of the code study for a simple reason: its harness isn't open source the way the others are, so there was nothing to dissect at the same depth. But leaving it out of the *code* study is not the same as leaving it out of the *conversation*, because by reputation, by community blind-comparison, and by the simple fact that everyone else keeps adopting its conventions, Claude Code arguably has the **best harness in the field.**

Consider the evidence that doesn't require reading its source:

- **It set the conventions the others copied.** The ergonomics of the modern coding-agent tool suite (the read / edit / grep / glob / bash design, the `AGENTS.md`-style project memory, plan mode, sub-agents, the skills format, MCP support) trace largely to Claude Code, which either originated or popularised much of this. When you see five independent harnesses converge on the same patterns, a lot of those patterns trace back to one place.
- **Blind output comparisons keep favouring it.** In the community write-ups that pit agents head-to-head on real tasks, Claude Code's output is preferred more often than not, frequently over Codex, even when Codex uses fewer tokens. The recurring verdict is "Codex is more efficient, Claude Code is more *right*."
- **Its harness sophistication is visible from the outside.** Sub-agent orchestration, a genuinely good permission UX, hooks, output styles, tight IDE integration, careful context management: the surface area you interact with is the most polished in the category, and polish in a harness is exactly what determines how much of the model's capability you actually realise.

So the fair summary is not "Codex is best, full stop." It's: **Codex and Claude Code sit at the top tier, trading blows**, with Codex winning on raw model strength, sandboxing and token efficiency and Claude Code winning on harness craft and output quality, while the five open agents below them serve different masters (cost, ownership, learnability, ecosystem). The reason Codex "wins" our shootout is partly an artefact of the rules: we ranked the agents whose code we could read, and the strongest harness in the field wasn't one of them.

There's a small irony worth admitting: the research behind this entire series (cloning six repos, reading tens of thousands of lines, drafting these posts) was itself done with a coding agent. The tools have gotten good enough to study, and explain, each other.

## What to actually use

- **You want the strongest autonomous coding agent and don't need to own it:** Codex, or Claude Code. Try both on your own hard task and trust your eyes over any benchmark; these two are close enough that fit-to-your-workflow decides it.
- **You want maximum capability at zero cost:** Gemini CLI.
- **You want to own the stack, pick your models, or self-host:** OpenCode (as a product) or Pi (as a foundation).
- **You're cost-sensitive or already in Moonshot's ecosystem:** Kimi Code.
- **You're building your own agent:** read Pi for the core, Codex for sandboxing, OpenCode for the client/server split, Kimi Code for event sourcing, Gemini CLI for recovery engineering, and study how Claude Code behaves, because it's the reference the others are quietly chasing.

The models will keep leapfrogging each other quarter by quarter; the rankings above have a short shelf life. What won't change is the lesson under all four posts in this series: the model sets the ceiling, but the harness decides how close to it you get to live. Choose the harness you'll actually enjoy operating. That, more than any benchmark, is what determines whether the agent helps you.

*That closes our four-part series on AI agent architecture. All six harnesses we studied are open source; if these posts made you curious, the code is the best teacher we found.*

c3eb856d2353baa1c60afada9ea2a11681b3b0ea10aa6fda67c80db70bc35600

A strengths-and-weaknesses comparison of the open coding agents Codex, Pi, OpenCode, Kimi Code and Gemini CLI, plus why Claude Code arguably has the best harness despite being closed.

After reading the source of five open coding agents (OpenAI's Codex, Pi, OpenCode, Kimi Code and Gemini CLI), here is how they actually stack up, strengths and weaknesses, which one wins, and why the best harness of all belongs to a tool we deliberately left out of the code study: Claude Code.

The Coding Agent Shootout and the Best Harness We Could Not Read

{/* banner prompt: An intricate technical blueprint of an 'AI Agent Harness' on off-white paper, showing a central processor labeled 'REASONING CORE' connected by clean, precise lines to distinct modules labeled 'Context Compaction,' 'Tool Contract,' 'Safety Boundary,' and 'Event-Sourced Persistence,' with dimensional annotations and detailed callouts, schematic diagram style. */}

In the [previous post](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/) we compared three very different AI agents and arrived at a claim that deserves its own article: **the model is a commodity; the harness is the product.** This post is the deep dive we promised.

Since then we've gone further and read the source code of **six** agent harnesses end to end: [Pi](https://pi.dev/), [Hermes Agent](https://github.com/NousResearch/hermes-agent), [OpenCode](https://github.com/sst/opencode), Moonshot's [Kimi Code](https://github.com/MoonshotAI/kimi-code), OpenAI's [Codex CLI](https://github.com/openai/codex) (the Rust harness is fully open even though the models are not), and Google's [Gemini CLI](https://github.com/google-gemini/gemini-cli). Six independent teams, four languages, wildly different products. And, as we'll see, they have all converged on the same set of organs. That convergence is the strongest evidence there is for what a harness actually has to do.

## The naive agent is four lines long

Strip everything away and an AI agent is this:

```
while not done:
    response = model(context)
    results  = execute(response.tool_calls)
    context += response + results
```

You can write that in an afternoon, point it at a frontier model, and it will genuinely work. For about ten minutes. Then reality arrives. The context window fills up. A tool dumps 80,000 tokens of build log into the conversation. The model edits the wrong file because two files have similar names. A network blip kills a forty-minute run. The model gets stuck retrying the same failing command forever. The user wants to say "no, not that way" *while* it's working. And someone, eventually, asks the agent to `rm -rf` something it shouldn't.

Every line of the six codebases we read exists because of that second ten minutes. The model supplies the reasoning per turn; the harness decides **what the model sees, what it can touch, what survives, and what happens when things go wrong.** Those four decisions are most of an agent, and none of them are AI.

## Job 1: Deciding what the model sees

The single biggest misconception about agents is that the model "has" the conversation. It doesn't. Every turn, the harness *constructs* what the model sees from scratch, and context is a scarce, expensive, attention-diluting resource. The engineering here has three layers.

**Construction.** The system prompt is assembled from parts: identity, tool descriptions, project instruction files (the `AGENTS.md` convention Codex originated, Gemini's `GEMINI.md` hierarchy), date, working directory. Skills, which are procedural knowledge files, are injected by *name and one-line description only*; the agent reads the full file on demand. This "progressive disclosure" pattern, now an open standard ([agentskills.io](https://agentskills.io)) adopted by Pi, Gemini CLI and others, is how an agent carries a large library of know-how at a standing cost of a few dozen tokens each.

**Compaction.** When the window fills, history must be squashed, and doing this badly is the number one cause of agents "forgetting" what they were doing. The mature harnesses treat it as a real algorithm, not a hand-wave. Pi only cuts at *valid cut points* (never in the middle of a tool call/result pair), summarises into a structured schema (goal, constraints, progress, decisions, next steps) and carries **cumulative lists of every file read and modified** across any number of squashes, so "what have I touched" is never lost. Kimi Code goes further with *cache-aware micro-compaction* that weighs the cost of invalidating the prompt cache before deciding what to squash.

**Cache discipline.** That last point deserves emphasis because it's invisible from the outside: LLM providers cache prompt prefixes, and a harness that rewrites earlier context casually pays full price for every turn. Hermes keeps its system prompt *byte-stable* for the lifetime of a conversation as a hard design invariant; Codex's own contributor guide codifies prompt-cache rules. Context isn't just curated. It's curated *append-only* whenever possible.

## Job 2: The tool contract

Tools are where the model's text meets the real world, and the contract is fussier than it looks.

**Output is budgeted.** Every harness truncates tool output with hard limits. Pi caps file reads at 2,000 lines or 50KB, grep at 100 matches, with the full output spilled to a temp file the model can fetch if it actually needs it. Without this, one chatty command destroys the context budget (see Job 1).

**Edits are engineered.** The deceptively hard tool is "edit a file." The naive approach, letting the model rewrite the whole file, is slow and destructive. The six harnesses showcase three generations of answers: exact string replacement with *fuzzy fallback* (Pi normalises smart quotes, Unicode dashes and trailing whitespace before giving up); a custom patch grammar the model is trained on (Codex's `apply_patch`); and self-correcting replace, where a failed match triggers an LLM side-call to repair the edit (Gemini CLI). Each is a harness answering the same question: how do you make a probabilistic text generator perform a deterministic operation reliably?

**Execution is stateful.** Codex's "unified exec" gives the model a persistent PTY session it can write stdin to, so it can run a REPL, answer an interactive prompt, keep a dev server running. Gemini CLI parses shell commands with tree-sitter to block command substitution and enforce allowlists *before* anything runs. The shell tool in a mature harness is closer to a tiny operating system interface than a `subprocess` call.

**Scheduling matters.** Tool calls within a turn run in parallel by default in Pi, unless any tool in the batch declares itself sequential, because parallel file edits to the same file are a race. Kimi Code has a conflict-aware scheduler for exactly this. These are classic concurrency problems, just with a language model as the caller.

## Job 3: The safety boundary

Here the six harnesses genuinely diverge, and the divergence is instructive because it's *philosophical*, not technical.

**Codex treats sandboxing as architecture.** Every command runs inside an OS-enforced sandbox (Seatbelt on macOS, bubblewrap + seccomp on Linux) with the filesystem scoped and network denied by default. The clever part is the layering: the *sandbox* is the technical boundary, the *approval policy* is the human boundary, and they are configured independently. The model can request escalation, and that request is itself part of the tool schema.

**Pi and Hermes refuse to pretend.** Pi ships no sandbox and no permission popups at all, on the stated grounds that in-process "sandboxing" of an agent that runs your toolchain is security theatre. Real isolation comes from the OS or a container, so use one. Hermes's SECURITY.md says the same thing in writing: OS-level isolation is the only boundary; everything else is best-effort. You may disagree with shipping that default, but the honesty is clarifying, and it's the correct *analysis* even for the harnesses that do more.

**OpenCode and Gemini CLI turn policy into data.** OpenCode's "plan mode" is not a mode at all. It's a permission ruleset (`edit: deny`, plus a carve-out directory for writing plans). Sub-agents are permission rulesets too. Gemini CLI has a TOML policy engine behind its approval modes. The lesson: don't hard-code behavioural modes; express them as declarative policy and the modes fall out for free.

## Job 4: Persistence and recovery

A forty-minute agent run accumulates state that must outlive crashes, restarts, and the model's own mistakes.

**Sessions are logs, not snapshots.** The convergent design is append-only: Pi's session is a JSONL *tree* (every entry has an `id` and `parentId`, so branching and time-travel are free); Kimi Code event-sources everything to a `wire.jsonl` you can literally replay in a visual debugger; OpenCode's v2 redesign is an event-sourced SQLite core. Nobody who has operated an agent at scale stores "current state." They store *what happened* and derive state from it.

**The escape hatch is git.** Both OpenCode and Gemini CLI maintain a *shadow git repository* (using git's `objects/info/alternates` mechanism, so it's cheap even on huge repos) and snapshot before risky operations, giving the user `/rewind` and the agent a guilt-free undo. The agent will sometimes do the wrong thing; the harness's job is to make the wrong thing cheap.

**Failure is classified, not caught.** Mature loops distinguish rate limits from auth failures from transient network errors, with different policies for each: exponential backoff here, model fallback there (Gemini CLI automatically falls back to Flash on quota exhaustion), a hard stop elsewhere. And the most human failure mode, the agent stuck in a loop trying the same fix forever, gets dedicated machinery: Gemini CLI detects loops and *injects corrective guidance* rather than just aborting; OpenCode escalates doom-loops into a permission ask, putting the human back in the loop exactly when the model has stopped making progress.

**Steering without breaking.** Users want to redirect a running agent without killing it. Pi's answer is two queues with different semantics: *steering* messages are injected between turns (the agent course-corrects mid-task), *follow-ups* are delivered only when it would otherwise stop. A small design, but it's the difference between an agent you collaborate with and one you can only fire-and-forget.

## Job 5: The engine/face split

The last organ is structural. Every mature harness has separated the **engine** (the loop, tools, state) from the **face** (whatever the user is looking at) behind an explicit protocol: Codex's core speaks a typed submission-queue/event-queue protocol, and the TUI, the IDE extension, `codex exec`, and the SDK are all just clients of it; OpenCode is an HTTP server with an OpenAPI spec, and the terminal UI, desktop app, web UI and GitHub bot are all generated-SDK clients; Gemini CLI routes tool confirmations through a serialisable message bus, which is what lets the same agent run interactively, headless in CI, or behind its Agent2Agent server.

This is the harness decision with the highest long-term leverage, because every future surface (IDE plugin, chat bot, CI job, another agent) is a client you haven't written yet.

## Convergent evolution, and the two real differences

Six teams, working independently, in Rust, TypeScript, and Python, shipped the same five organs:

1. **Context engineering:** constructed views, progressive disclosure, structured compaction, cache discipline
2. **A tool contract:** schemas, output budgets, engineered edits, concurrency rules
3. **A policy boundary:** permissions and/or sandboxing, increasingly expressed as data
4. **Append-only persistence with cheap undo:** event logs plus git checkpoints
5. **An engine/face protocol:** one core, many clients

When independent evolution produces the same anatomy, that anatomy is the job description. If you are building an agent, this list *is* your roadmap.

What actually differentiates the six is just two things. **Sandboxing depth:** only Codex treats OS-level isolation as load-bearing architecture; everyone else delegates it to you. And **what persists across sessions:** only Hermes ships a learning loop (self-written skills, a curating background process, searchable memory of every past conversation); the rest start each session from zero plus instruction files. Our bet, for what it's worth: the second gap is where the next round of competition happens, because per-session capability is converging fast.

## Why the model doesn't make this go away

The obvious objection: models keep getting better, so won't a sufficiently smart model just not need all this scaffolding?

The evidence so far says no, for a structural reason: **the harness governs the realised fraction of the model's capability, and that fraction is set by information logistics, not intelligence.** A smarter model still cannot reason about a file that was truncated out of its context, still cannot remember a decision that compaction discarded, still cannot un-delete a directory, and still burns money if its cache is invalidated every turn. Model progress raises the ceiling; the harness determines how close to the ceiling you operate. If anything, better models *raise* the stakes on the harness. A model capable of an eight-hour autonomous run is only useful inside a harness whose compaction, checkpointing and recovery can survive eight hours.

The strongest proof is the vendors themselves. OpenAI co-designs the Codex models *with* the Codex harness, training them to use its specific patch grammar and escalation protocol, and the same model measurably performs differently in different harnesses. If the harness were incidental, none of that investment would make sense.

## What this means in practice

If you're *choosing* an agent: stop comparing models and start comparing harnesses. Ask how it compacts, what it does when an edit fails, whether you can rewind, what's sandboxed, and what survives a restart. Those answers predict your daily experience far better than a benchmark score.

If you're *building* one, as we are, the encouraging news is that the harness is the one layer where a small team can genuinely compete. You cannot train a frontier model; you can absolutely write a better loop. Pi's entire agent runtime is about 2,700 lines of TypeScript. The five organs above are a few thousand lines of careful, testable, boring code, and all six of the codebases we studied are open. Read Pi for the cleanest minimal core, Codex for sandboxing and the protocol-first split, OpenCode for the client/server architecture, Kimi Code for event sourcing, Gemini CLI for recovery engineering, and Hermes for memory and self-improvement.

The model reasons. Everything else is the harness, and everything else, it turns out, is most of the work.

*Next in the series: skills and memory. How an agent captures knowledge, why a human still has to write some of it down, and what "self-improving" actually means in code.*

5b5e87209cd297a2fe9811335bfa9eeb4af981408db2591b1aadef1b966ffcda

A deep dive into the AI agent harness, covering context management, tool contracts, safety boundaries, persistence and recovery, based on reading the source code of six open agent harnesses.

We read the source code of six AI agent harnesses: Pi, Hermes, OpenCode, Kimi Code, OpenAI's Codex CLI and Google's Gemini CLI. This is what the harness really does for the model, why almost every agent failure is a harness failure, and why the engineering lives there, not in the model.

What an AI Agent Harness Actually Does

{/* banner prompt: A digital paper collage depicting three different AI agents on a textured craft paper background, flat-lay perspective: one is a single, sharp-edged piece of black cardstock with an embossed circuit pattern; another is a deconstructed set of neatly cut geometric paper shapes; the third is a layered, growing scrapbook of many small, overlapping paper snippets, tangible textures, warm natural lighting --ar 16:9 */}

Put OpenAI's Codex, Pi from pi.dev, and Nous Research's Hermes Agent side by side and the first reaction is: why are these three even in the same article? One is a commercial product from the biggest AI lab in the world. One is a minimalist open-source toolkit largely written by a single developer. One is a sprawling Python personal assistant that lives in your Telegram.

That is exactly why they belong together. We recently spent some time reading the source code of Pi and Hermes (both are open source, so we cloned and analysed them properly, not just the READMEs) and the public documentation of Codex (whose CLI is open, but whose brains are not). The three of them are so different that the differences themselves become a map. If you understand why these three agents are built the way they are, you understand most of the AI agent landscape: what each type of agent *is*, what it *can do*, and which one can actually *help you*.

## First, what is an "agent" anyway?

Strip away the hype and every AI agent is the same machine: a loop that sends context to a language model, reads back text and tool calls, executes those tools (run a command, edit a file, search the web), feeds the results back in, and repeats until the job is done.

The model supplies the reasoning. Everything else (what the model sees, what it is allowed to touch, what survives between sessions, what happens when things go wrong) is supplied by the software wrapped around it. That wrapper is called the **harness**, and the three agents in this article are, more than anything, three very different opinions about what the harness should be.

## Codex: the vertically-integrated product

Codex is what you get when a frontier lab builds the whole stack: proprietary models (the GPT-5.x-Codex family) co-designed with the harness, delivered through every surface you might work in: terminal CLI, IDE extensions, a desktop app, mobile, a cloud service that runs tasks in parallel containers, and a GitHub bot that reviews your pull requests when you tag `@codex`.

Its strengths come from that integration:

- **Raw coding capability.** The Codex models lead or sit near the top of agentic coding benchmarks (SWE-bench Verified, Terminal-Bench), and they are tuned for long autonomous runs, hundreds of tool calls without a human nudge.
- **A real security model.** Codex is the only one of the three that ships OS-enforced sandboxing by default: Seatbelt on macOS, bubblewrap on Linux, network access denied unless you opt in. There is a clean separation between the *sandbox* (a technical boundary) and the *approval policy* (when to ask a human). This is what a production-grade safety story looks like.
- **Token efficiency.** Community comparisons consistently report it doing comparable work with two to three times fewer tokens than its rivals, which matters when you pay by subscription tier.

The trade-off is just as clear: the models are closed, deprecations are forced on OpenAI's schedule, the best models are sometimes gated behind expensive tiers, and the extension ecosystem (hooks, skills, plugins arrived only in 2026) is young. You are renting a very good employee from a company that can change the terms at any time.

**Codex is the answer to:** "I want the strongest coding agent today and I don't need to own it."

## Pi: the harness as a library

Pi (from [pi.dev](https://pi.dev/)) is the opposite philosophy taken seriously. It ships **zero models**. It is a TypeScript monorepo of four packages: a unified LLM API over roughly forty providers, a tiny agent runtime (~2,700 lines), the coding-agent CLI, and a terminal UI framework. Its core design rule is to refuse features. No MCP. No built-in sub-agents. No permission popups. No plan mode. No to-do lists. The author has written publicly about why each of these is excluded; everything on that list can be added back as a TypeScript extension in about a hundred lines.

Reading Pi's source is the best education in agent engineering we have found anywhere:

- **Sessions are an append-only tree.** Every entry in the session file has an `id` and a `parentId`, so branching, forking, and time-travelling through a conversation cost nothing: no database, no file rewrites.
- **Compaction is engineered, not hand-waved.** When the context window fills, older history is summarised into a structured format (goal, progress, decisions, next steps) with cumulative lists of every file read and modified, so the agent never forgets what it has touched, no matter how many times the history is squashed.
- **The provider layer absorbs chaos.** Thinking levels, tool-call ID formats, cache semantics and reasoning quirks of ~30 providers are normalised behind one streaming interface, so you can switch models mid-conversation without breaking anything.

The trade-offs are deliberate: there is no sandbox at all (you are expected to containerise it yourself), and the project carries a bus factor of roughly one. It is a power tool, with everything that implies.

**Pi is the answer to:** "I want to own my agent, choose my models, and understand every moving part, or build my own agent on proven foundations." (Not a hypothetical: OpenClaw, one of the most popular open personal assistants, is built on Pi's packages.)

## Hermes: the agent that lives with you

Hermes Agent, by Nous Research, is not really a coding agent at all, and that is the point. It is a **persistent personal assistant**: a Python daemon that connects to more than twenty messaging platforms (Telegram, Discord, Slack, WhatsApp, Signal, Matrix, WeChat...), executes work on any of six backends (your machine, Docker, SSH, or serverless clouds like Modal where an idle agent hibernates for fractions of a cent), runs scheduled jobs while you sleep, and, its signature feature, **improves itself**.

Hermes closes a loop the other two don't have:

- It writes its own **skills** (procedural knowledge captured as markdown files) from experience.
- A background **curator** periodically reviews those skills, consolidating, patching and archiving, like a librarian maintaining the agent's own operations manual.
- A pluggable **memory** system builds a model of who you are across sessions, and full-text search over every past conversation means it can ask itself "have I solved this before?"

Where Codex and Pi start every session from zero (plus whatever instructions you hand them), Hermes compounds. The costs are equally visible in the code: a huge attack surface (twenty-three platform adapters all parsing untrusted input), a single SQLite file as the global state store, and general-purpose breadth instead of coding depth: no IDE integration, no native git-worktree workflow. Its `SECURITY.md` is refreshingly honest: OS-level isolation is the only real boundary; everything else is best-effort.

**Hermes is the answer to:** "I want an assistant that is always on, reachable from my chat apps, and gets better the longer it works for me."

## The same machine, three species

<Table data={
    {
        headers: ['', 'Codex', 'Pi', 'Hermes'],
        rows: [
            ['What it is', 'Commercial coding agent', 'Open agent toolkit', 'Persistent personal assistant'],
            ['Lives in', 'Terminal, IDE, cloud, GitHub', 'Your terminal', 'Your chat apps, your servers'],
            ['Models', "OpenAI's, closed", 'Any of ~40 providers', 'Any OpenAI-compatible provider'],
            ['Memory between sessions', 'Preview feature', 'None (by design)', 'Its defining feature'],
            ['Safety', 'OS sandbox, default on', 'None (bring your own container)', 'OS isolation via exec backends'],
            ['Extensibility', 'Config, skills, MCP, plugins', 'Full TypeScript runtime', 'Skills, plugins, MCP server'],
            ['Biggest risk', 'Vendor lock-in', 'Bus factor of one', 'Security surface'],
            ['Best at', 'Hard coding tasks, today', 'Being understood and owned', 'Compounding over months'],
        ]
    }
} />

Three lessons fall out of this comparison, and they generalise well beyond these three projects.

**1. The model is a commodity; the harness is the product.** All three can call the same class of frontier models. What you are actually choosing between is harness decisions: how context is managed, what tools look like, what persists, what is sandboxed. Nearly every agent failure you will ever observe (the agent "forgetting" the goal, going off the rails late in a session, missing the error that contained the answer) is a harness failure, not a model failure.

**2. "Agent" is not one product category.** A coding agent optimises for depth on a workstation: tight tool loops, repository awareness, review workflows. An assistant agent optimises for breadth and time: reachability, scheduling, memory, identity. The harness decisions that make one excellent make the other worse. Asking "which agent is best?" without specifying the species is like asking whether a truck is better than a motorbike.

**3. Memory and skills are where the next differentiation happens.** Codex represents peak *per-session* capability; Hermes represents the bet that *across-session* learning matters more. A skill, a piece of captured procedural knowledge the agent can load on demand, lets a mid-tier model with the right manual outperform a frontier model improvising. The agent that remembers your environment, your conventions, and its own past mistakes eventually beats the smarter agent with amnesia.

## So which one helps *you*?

- **You write code for a living and want results now:** Codex (or its direct competitors). Bundled with a ChatGPT subscription, sandboxed by default, strongest models.
- **You are a tinkerer, you care about model choice and data ownership, or you are building your own agent:** Pi. Read its source even if you never run it; the session-tree and compaction designs alone are worth the afternoon.
- **You want a personal assistant rather than a pair programmer, something that triages, schedules, researches, and remembers:** Hermes, or its TypeScript cousin OpenClaw (which, fittingly, is built on Pi).

And if you are building agents rather than just using them, the real takeaway is to read Pi and Hermes side by side: one shows you how to build the *engine*, the other shows you how to build the *life* around it. The whole field is currently busy discovering that both halves matter.

*This post is the first in a series from our research into AI agent architecture. Next up: a deep dive into what an agent harness actually does, and why it, not the model, is where the engineering lives.*

5223343311040bff68bdf49cffaabb36c1afbea3432e0f319d53a014dce4ace7

A code-level comparison of OpenAI Codex, Pi (pi.dev) and Nous Research's Hermes Agent, using their differences to explain the different types of AI agents.

Comparing OpenAI Codex, Pi (pi.dev) and Hermes Agent by Nous Research: an odd trio at first glance, but their differences map out the whole landscape of AI agents: what each type is, what it can do, and which one can help you.

Tag: Coding Agents