{/* banner prompt: A playful and polished 3D diorama of a cozy workspace, a cute Linux penguin mascot carefully organizes files within a single, solid terracotta-colored folder on a desk, while a vast, abstract library of semi-transparent files gently swirls in the background, rendered with soft lighting and matte materials, distinctive color palette of sage green, terracotta, and cream, 16:9. */}

There is a particular kind of cloud-storage problem that only appears after you have accumulated years of useful files: you rarely need the whole drive, but you still want the part you are working on to feel local.

For me, that might be a tax folder for one financial year. I want to open it in ordinary Linux tools, add receipts, edit spreadsheets, run scripts, and eventually send the finished folder back to Google Drive. I do not want a background mirror trying to make two entire trees identical while I am still deciding what belongs in the project.

That is why we built [TYO CloudDrive](https://github.com/tyolab/tyo-clouddrive), an open-source Google Drive client for Linux. The command is `tyocd`, and its design is deliberately conservative: browse Drive lazily, check out only the folder you need, and make uploads an explicit, reviewable operation.

## Three ways to work with Drive

TYO CloudDrive has three separate modes because browsing, editing, and one-off transfers have different safety requirements.

<Table data={
    {
        headers: ['Mode', 'Best for', 'What it does'],
        rows: [
            ['Lazy virtual drive', 'Browsing a large Drive', 'Shows folders and fetches file content when opened'],
            ['Editable workspace', 'Working on one project or year', 'Checks out one Drive folder into a local directory'],
            ['Additive transfer', 'Sending a separate local tree', 'Creates new Drive objects from an immutable plan'],
        ]
    }
} />

The important word is **separate**. TYO CloudDrive is not a background mirror-sync engine. It does not watch your local filesystem and silently propagate every rename or deletion. That makes it a little less magical, but much easier to reason about when the files matter.

## Browse without downloading the whole drive

The read-only FUSE mount gives Linux a familiar directory view of Google Drive:

```sh
mkdir -p ~/TYOCloudDrive
tyocd mount ~/TYOCloudDrive
```

Folder metadata is loaded when a directory is listed. A binary file is downloaded only when something opens it, then stored in a private cache and checked against Drive's size and MD5 checksum when available. If Drive's version has not changed, the cached bytes can be reused.

The mount is read-only at several layers. It does not write through changes, rename Drive objects, or delete anything. Google Docs, Sheets, and Slides appear as small link files such as `.gdoc`, `.gsheet`, and `.gslides`, which is more useful than pretending an exported copy is the original document.

There is one honest limitation: opening a binary file currently downloads that file in full. Sparse range caching is not implemented yet. The lazy part is selecting which files to fetch, not fetching only the first few kilobytes of a large file.

## Check out one folder as a local workspace

When I am doing real work, the editable workspace is the more interesting feature. Give it a Drive folder ID and a local destination:

```sh
tyocd workspace checkout DRIVE_FOLDER_ID /home/me/Documents/Tax
```

The checkout copies only the selected subtree. TYO CloudDrive keeps an exact-ID manifest in private application state rather than adding metadata files to the project directory. That manifest records the Drive IDs, local checksums, remote versions, and folder relationships needed to tell a genuine change from a confusing filename coincidence.

The normal loop is:

```sh
tyocd workspace status /home/me/Documents/Tax
tyocd workspace plan /home/me/Documents/Tax --path 2026 --update
tyocd --yes workspace apply PLAN_ID
```

The plan is a saved JSON document identified by a SHA-256 digest. Before the first write, apply checks the local files, the exact Drive objects, their versions, and the relevant parent folders. If the cloud copy changed while you were working, the plan becomes a conflict instead of guessing which copy should win.

That is useful for a year-by-year folder. You can prepare `2027` locally, add only the receipts and working files that belong there, inspect the plan, and then upload that scoped subtree. A new local folder can be created in Drive without pulling unrelated changes from `2025` or the rest of the account.

## Safety rules that shape the product

The safety model is not a warning printed above an unsafe operation. It is part of the command structure.

- There is no permanent-delete command and no mirror-delete algorithm.
- A remote-only item becomes `keep_remote`; it is never treated as permission to delete a local copy or vice versa.
- A changed bound file is a conflict unless `--update` is explicitly supplied.
- Every upload is snapshotted and checksum-verified before mutation.
- Creates use pre-generated Drive IDs, so an interrupted apply can be replayed without making duplicate objects.
- File replacements retain the current binary revision before the conditional update.
- Updates use an HTTP `If-Match` check so a last-second remote change stops the write.
- A local deletion does not delete the Drive object.

The last rule is especially intentional. A local workspace is a place to work, not a mirror with authority to erase the source of truth.

## Open source, and still early

TYO CloudDrive is MIT licensed and available at [github.com/tyolab/tyo-clouddrive](https://github.com/tyolab/tyo-clouddrive). It is written in Go and targets Linux. The ordinary CLI and workspace commands do not require FUSE; the optional mount does.

To build it:

```sh
git clone https://github.com/tyolab/tyo-clouddrive.git
cd tyo-clouddrive
make build
./bin/tyocd --version
```

You bring your own Google OAuth desktop client, authorize the scope you need, and then choose whether to browse read-only or use the write-capable workspace flow. The full [usage guide](https://github.com/tyolab/tyo-clouddrive/blob/main/docs/USAGE.md) explains the commands and the recovery behavior.

This is early alpha software. The core safety model is implemented and tested, but it is still wise to start with a disposable Drive folder and keep an independent backup. Retained Drive revisions and the trash are recovery layers, not a backup strategy.

The goal is modest: make Google Drive feel useful on Linux without making local work feel like a high-stakes synchronization experiment. If you want a whole-account mirror, this is probably not the tool. If you want a small, deliberate part of Drive to behave like a project, that is exactly what we are building.


eb78adb9267b40fdda634acc0fc9773e9b186877b0c71ad3622d8d9c0eb4da0d

TYO CloudDrive is an open-source Linux Google Drive client with a read-only lazy mount and safety-first selective folder checkouts.

Introducing TYO CloudDrive, our open-source Linux CLI for safer Google Drive browsing, lazy virtual mounts, and selective editable folder workspaces.

TYO CloudDrive: An Open Source Google Drive Alternative for Linux

{/* banner prompt: A warm matte clay-render still life in a tactile miniature-diorama style, four rounded friendly robot characters each a distinct colour — soft indigo blue, mint green, gentle violet, warm terracotta — gathered in a circle around a small softly glowing central hub shaped like a little rounded server, passing tiny luminous rounded speech-bubble tokens and small envelopes between them along delicate glowing light threads. Sculpted matte modelling-clay materials with subtle fingerprints and soft edges, cream and sage-green studio background, soft directional studio lighting casting gentle long shadows, shallow depth of field, cozy toy-like handcrafted feel, 16:9 aspect ratio. */}

A few weeks ago I wrote about [the group chat we built for our AI agents](/blog/2026/08/21-cross-machine-ai-agent-messaging-built-on-tyo-mq/): Claude Code sessions on four different machines that could open a room, message each other, and wake an idle peer to respond. It worked, we use it every day, and the response to that post was the same question over and over: *can I run this?*

Back then the honest answer was no. It was welded to our stack. It assumed Postgres was there, it assumed [tyo-mq](https://github.com/tyolab/tyo-mq) was the broker, and it was tangled up with a pile of our own internal machinery — job queues, tweet review, a Telegram bot. It was a system, not a product.

So we pulled it apart and rebuilt the useful core as a standalone, open-source project. It is called **Parley**, and you can install it right now:

```sh
pip install parley-agents
# or
docker pull tyolab/parley
```

Source is at [github.com/tyolab/parley](https://github.com/tyolab/parley), MIT licensed.

## What Parley actually is

Parley is a messaging layer for AI agents. It gives you durable, ordered **rooms** — group chats where agents (or humans) post messages and each reader sees every message exactly once, in order, surviving restarts — and it gives you optional **push**, so an idle agent stirs within seconds of a message landing instead of waiting on its next poll.

That is the whole idea, and it is deliberately small. Parley is not an orchestration framework. It does not plan tasks, assign work, or decide who does what. It is the pipe the agents talk through. We think that boundary is exactly right: the intelligence lives in your agents, and the messaging layer should be boring, durable plumbing that gets out of the way.

If you read the earlier post, you will recognise the shape. A message has to be **remembered** and it has to be **noticed**, and those are two different jobs. Remembered is a database's job. Noticed is a message bus's job. Parley keeps them strictly separate — and, crucially, lets you choose what backs each one.

## Broker-agnostic, and zero-broker by default

The biggest change from our internal version is that Parley assumes nothing about your infrastructure.

The system of record — the rooms and their messages — can live in **SQLite** or **PostgreSQL**. Out of the box it is SQLite, in a file under your home directory. No server to stand up, no connection string, nothing to configure. You run one command and you have working rooms.

The nerve — the push signal that wakes an idle agent — is a pluggable transport. If you have no broker at all, Parley degrades to **polling**, which needs nothing extra and always works. If you want real push, point it at **tyo-mq**, **Redis**, or **NATS**. The transport only ever carries a tiny nudge (`there is something new in room X`); it never carries message bodies. The store stays the single source of truth, so a dropped or duplicated nudge can never lose or corrupt a message. Worst case, you notice a beat later.

<Table data={
    {
        headers: ['Layer', 'Job', 'Options'],
        rows: [
            ['Store (system of record)', 'What was said, and did I read it?', 'SQLite (default), PostgreSQL'],
            ['Transport (the nerve)', 'Did something just happen?', 'Polling (default), tyo-mq, Redis, NATS'],
        ]
    }
} />

This is the part I care most about. You should be able to try Parley on a laptop with zero moving parts, and later swap in Postgres and a real broker for a fleet — without changing a line of your agent code. Same rooms, same identities, same API. Just a different backend.

## Four ways to talk to it

An agent messaging layer is only useful if your agents can actually reach it, and agents come in different shapes. So Parley exposes the same rooms four ways:

- **An MCP server.** Any MCP-capable agent — Claude Code, or anything else that speaks the Model Context Protocol — gets a set of room tools (`join_conv`, `say`, `poll_convs`, and friends) with no glue code. This is how our own agents use it.
- **A REST API and a Python SDK.** For agents and services that are not MCP-based, there is a plain HTTP API and an async Python client.
- **A CLI.** `parley serve` runs the gateway; `parley join` / `parley say` / `parley watch` let a human sit in the same rooms as the agents from a terminal. Being able to watch your agents talk, live, is worth more than it sounds.
- **A Claude Code Stop hook.** This is the push delivery pattern in practice: a hook that runs at the end of each turn, checks for messages from peers, and surfaces them into the session. It is what makes an agent actually *respond* to another agent, not just receive.

## Identity you can trust

The moment agents can message each other across machines, identity stops being cosmetic. If any process can claim to be any agent, a room is just a place to be impersonated.

Parley resolves identity **server-side**. Each box authenticates with a bearer token, and the server derives who you are from that token — it does not trust a client-supplied header for the important part. An agent can present a per-session handle (so you can tell two sessions on the same machine apart), but that handle has to belong to the box the token authenticates. A forged claim to be another box is simply ignored. We verified this the fun way, by trying to spoof it, and watching it refuse.

There is a subtle lesson baked into that design, and it came from a real bug. Early on, a session with no explicit handle collapsed to a bare machine identity — and a bare machine turned out to be *deaf* to the individual sessions running on it. The fix, and now the rule, is that every interactive session gets its own distinct handle. Parley's default behaviour follows from that: two distinct identities always hear each other; the only thing you are ever filtered from is your own echo.

## Joining should take one command

The original system was hand-wired onto every machine, which is fine for six boxes you own and miserable for anyone else. So Parley has self-serve enrollment. An operator turns on a join code; then any new agent runs:

```sh
parley enroll --gw https://your-gateway --join-code CODE --box my-agent
```

and that single command mints a token, writes the MCP entry into the agent's config, registers the Stop hook, and drops a locked-down env file into place. The agent is in the room and reachable, one command, no manual editing. A leaked join code cannot mint a second token for a box that already has one, so the anti-spoof guarantee holds even with self-serve on.

## We run our whole workshop on it

This is the part that matters most to me, because it is easy to open-source something you do not actually depend on.

Parley now carries the real agent-to-agent traffic across our workshop — the desktop, the laptops, the always-on NUC in the corner — with Postgres as the store and tyo-mq for push. The internal system it replaced is still there as a fallback, but the agents are talking over Parley. We did that on purpose before writing this post. I did not want to tell you it was good; I wanted to be running on it myself first.

And because it turns out four AI agents in a shared room is also just *fun*, we wired four different models (DeepSeek, Codex, Kimi, and GLM) into one Parley room and had them run a live arcade fight: two playing the characters, two calling the match like sports commentators, all of it coordinated over Parley. The voices are text-to-speech; every move, taunt, and line of commentary really did travel over Parley.

<YouTube id="1jOfq5CftEs" title="Four AI Models Play Street Fighter Together, Over Parley" />

## The honest caveats

Parley is young. The SQLite-plus-polling path is the most battle-tested; the Redis and NATS transports are newer and marked beta. It is a messaging layer, not an orchestration framework — if you want automatic task planning and assignment, that lives above Parley, in your agents. And the two-layer design means push is best-effort by nature: the store guarantees correctness, the transport only improves latency. That is a deliberate trade, and I would make it again, but it is worth knowing going in.

If you have agents that need to talk to each other — across sessions, across machines, or just across the two terminals open on your desk — give it a try:

```sh
pip install parley-agents
parley serve
```

Then, in another terminal, `parley watch` and say hello to your agents. The docs on [GitHub](https://github.com/tyolab/parley) cover Postgres, Docker, brokers, and wiring it into Claude Code.

*This is the sequel to [Cross-Machine AI Agent Messaging, Built on tyo-mq](/blog/2026/08/21-cross-machine-ai-agent-messaging-built-on-tyo-mq/) — that post is the design; this one is the product.*


619fa17f4649ef0a7c8559a63c3b85066a99314a3648b6c0f90e72e16406bd3d

Parley is an open-source, broker-agnostic messaging layer for AI agents: durable rooms in SQLite or Postgres, push delivery over tyo-mq, Redis or NATS, server-side anti-spoof identity, and MCP, REST, SDK and CLI surfaces.

We took the cross-machine agent group chat we built on tyo-mq, made it broker-agnostic, and open-sourced it. Parley is a durable rooms-and-push messaging layer for AI agents that you can pip install or docker pull today.

Introducing Parley: Broker-Agnostic Messaging for AI Agents


{/* banner prompt: Clean editorial illustration of a magnifying glass moving over a cloud-shaped invoice, small dials and toggles floating around it (min-instances, CPU, region, disk), a few of them glowing to show waste being switched off, warm white background with teal and orange accents, calm and practical, 16:9. */}

Our monthly Google Cloud bill came in at just over 600 dollars. Nothing was on fire, no traffic had spiked, and no new service had launched. The bill had simply crept up, one reasonable-looking decision at a time.

So we sat down and audited it, line by line. By the end we had a plan that cut the bill by more than half, without leaving the cloud, without moving regions, and without giving up anything we actually used. Almost all of the savings came from understanding *settings* — the dials that cloud platforms leave on their most convenient default rather than their cheapest one.

Here is what we learned.

## Read the bill like a map, not a total

The single most useful thing we did was stop looking at the total and start grouping every line item by what it actually was. Google's cost table breaks charges into dozens of tiny SKUs — "CPU (request-based billing)", "SSD backed PD capacity", "network inter-region data transfer" — and on their own they mean very little. Grouped by *cause*, a shape appears fast.

When we did that, one pattern jumped out immediately:

<Table data={
    {
        headers: ['Where the money went', 'Roughly', 'What it actually was'],
        rows: [
            ['One background workload', 'About 60%', 'A fleet of small agents writing to the database on a timer'],
            ['Web serving', 'A chunk', 'Cloud Run handling real requests'],
            ['Always-on worker', 'Around 50 dollars', 'A container that could never scale to zero'],
            ['One database server', 'Around 90 dollars', 'A virtual machine sitting almost completely idle'],
            ['Small stuff', 'The rest', 'Logs, disks, snapshots, an orphaned IP address']
        ]
    }
} />

The lesson of the audit was almost boring: **one workload was 60% of the bill.** Everything else combined — every website, every API, every bit of storage — was the minority. If you only have time to fix one thing, find your elephant first.

## Cloud Run is cheap, until a setting says otherwise

We love Cloud Run. It scales to zero, you only pay when it runs, and you never patch a server. But "you only pay when it runs" hides two settings that decide whether that promise holds.

### Minimum instances

By default a Cloud Run service scales to zero when no one is using it. Set a minimum number of instances above zero and you are asking Google to keep that many copies warm at all times — which removes cold starts, and bills you around the clock whether or not a single request arrives.

<Table data={
    {
        headers: ['Setting', 'What it does', 'What it costs'],
        rows: [
            ['Min instances = 0 (default)', 'Scales to zero when idle', 'Nothing when idle; a cold start on the first request'],
            ['Min instances = 1 or more', 'Keeps copies always warm', 'You pay for those copies 24/7, including while you sleep'],
            ['CPU only during requests (default)', 'CPU is throttled between requests', 'Cheapest for normal request and response work'],
            ['CPU always allocated', 'CPU keeps running between requests', 'Bills like an always-on VM, not per request']
        ]
    }
} />

We had a couple of services quietly holding a minimum instance warm for a workload that saw almost no traffic. Setting them back to zero cost us nothing but a cold start once in a while, and removed the charge entirely.

### The always-on trap

Here is the sharper lesson. One of our services was not a website at all. It was a background worker that holds a long-lived connection to a message queue and reacts to events. It has no incoming web requests to speak of — just a socket it needs to keep open.

That is exactly the workload Cloud Run is *worst* at. To keep the connection alive, it needs a minimum of one instance and CPU that is always allocated. In other words, it has to behave like a server that never turns off — while being billed as premium serverless compute. It cost around 50 dollars a month to do something a tiny always-on virtual machine, or a container on a machine we already ran, could do for close to nothing.

**The rule we took away:** serverless is brilliant for spiky, request-driven work that can genuinely go to zero. For anything that must stay awake all the time — a queue consumer, a poller, a persistent connection — check whether a small VM you already operate is cheaper. Often it is, by a wide margin.

## When to self-host a database instead of using a managed one

This was the big one, and it is the least obvious.

Managed databases — Cloud SQL, Firestore, DynamoDB and friends — are wonderful. Backups, failover and scaling are handled for you. But many of them bill **per operation**: so much per read, so much per write. That pricing is fantastic when your traffic is small or spiky. It turns brutal when your workload is *chatty*.

Our background fleet was as chatty as it gets. Each agent wrote a small status update to the database on a fixed timer — every 30 to 60 seconds — whether anything had changed or not. Multiply a handful of writes per minute by hundreds of agents across a month and you land at **hundreds of millions of tiny operations**. On per-operation pricing, those cheap little writes added up to the largest single line on the whole bill.

The same writes, sent to a plain MySQL server running on a virtual machine we already operate, cost effectively nothing extra. The disk was already spinning; a few hundred million small writes a month is comfortably within what one modest server handles without noticing.

So when should you reach for each?

<Table data={
    {
        headers: ['A managed database fits when', 'Self-hosting starts to win when'],
        rows: [
            ['Traffic is small, spiky or unpredictable', 'Traffic is steady and high in volume'],
            ['You would rather not run a server at all', 'You already run a VM with spare capacity'],
            ['Per-operation cost is a rounding error', 'Your workload is chatty: many tiny reads and writes'],
            ['You want managed backups and failover', 'You are happy to run your own backups'],
            ['You are early and want to move fast', 'You are steady and want predictable, flat cost']
        ]
    }
} />

There is a second reason we moved this workload onto our own database: **portability.** A plain MySQL server over a standard connection runs anywhere — another cloud, another region, your own hardware. Per-operation managed services are convenient precisely because they lock the shape of your data and your bill to one vendor.

A word of caution, because it cuts both ways: self-hosting means *you* own the backups, the patching and the 3am restart. We did not move a customer-facing production database off its managed home for a few dollars. We moved a high-volume, non-critical, internal workload where the maths was overwhelming and the risk was low. Pick your battles.

Two cheap habits also soften per-operation pricing without moving anything:

- **Write only on change.** A status update that is identical to the last one does not need to be written at all. Debouncing our timer writes alone would have removed most of the volume.
- **Do not read to decide whether to write.** It is easy to "save a write" by reading first — and end up paying for the read instead. On a chatty workload, reads can cost as much as the writes you were trying to avoid.

## Right-size relentlessly

One database server on the bill cost around 90 dollars a month. We logged in to look at it and found the load average sitting at 0.00. Two dedicated CPU cores, 99% idle, around the clock.

We had sized it once, long ago, for a load that never arrived — and then paid for that guess every month since. Swapping it for a smaller, shared-core "burstable" machine — the kind that idles cheaply and borrows extra CPU only when it briefly needs it — cut the compute cost roughly in half. It even came with more memory than before.

The habit worth building: **size machines to what they actually do, measured, not to what you once imagined they might need.** A minute with a load monitor is worth more than an hour of guessing.

## The quiet extras: region, disk, IPs and logs

Once the big items were handled, a set of small ones were each worth a few dollars, and together a meaningful slice.

**Region.** The exact same machine costs noticeably more in some regions than others — a premium of 30 to 40% is common for the more expensive locations. That premium is often worth paying: keeping a database close to your users cuts latency, and moving a production system across the world is risky. But pick your region *on purpose*, knowing what the proximity costs, rather than out of habit.

**Disk tier.** Fast SSD disks cost roughly double a "balanced" disk of the same size. A small database that is mostly idle does not need premium disk performance. Matching the disk tier to the actual workload is an easy, invisible saving.

**Idle IP addresses.** Reserved static IP addresses are billed even when they are attached to nothing. We found one reserved address doing absolutely nothing and released it. Worth a scan.

**Logs.** Log ingestion is free up to a generous monthly allowance, then billed per gigabyte. One noisy service — including a bug that threw the same error thousands of times a day — was pushing us over the line. We added a filter to drop the low-value noise (while keeping real errors) and flagged the bug to fix at the source. Verbose logging is a cost, not just a habit.

## About those "commit and save" discounts

Cloud providers offer real discounts if you commit to a resource for one or three years. They can be worth it for a steady, permanent workload. But read the fine print: the cheap shared-core machines are often *not* eligible, and — importantly — right-sizing first can beat committing. In our case, downsizing an idle machine saved more than a one-year commitment on the oversized one would have. Do the cheap, reversible optimisation before you sign a year-long contract.

## The takeaway

None of this was clever. We did not rewrite anything or adopt a new platform. We read the bill like a map, found the one workload that was most of it, and turned the right dials:

<Table data={
    {
        headers: ['Move', 'Why it worked'],
        rows: [
            ['Group the bill by cause', 'One workload was 60% of it; the rest was noise'],
            ['Mind Cloud Run min-instances and CPU', 'Warm instances and always-on CPU bill 24/7'],
            ['Self-host the chatty database', 'Per-operation pricing punishes high-volume writes'],
            ['Right-size idle machines', 'A 0.00 load average is money burning'],
            ['Sweep the small stuff', 'Region, disk tier, idle IPs and noisy logs add up']
        ]
    }
} />

Cloud platforms are a wall of dials, and the defaults are tuned for convenience and for the provider's revenue — not for your bill. The good news is that the same wall of dials is exactly what lets you fix it. Set a reminder to audit the bill every quarter. The cloud rewards attention, and quietly taxes the lack of it.

*If you run anything in the cloud and have never grouped your bill by cause, that is the single most valuable hour you can spend this month.*


df14e3d9271520b989cd074edc113525f817c67a06a7dcc5b44d3319e182b71f

Practical lessons from auditing a Google Cloud bill: Cloud Run min-instances and CPU billing, when to self-host a database instead of a managed one, and region and machine right-sizing.

A real 600-dollar-a-month cloud bill, audited line by line, and the Cloud Run settings, database choices and machine sizes that quietly drove the cost.

How We Cut Our Google Cloud Bill in Half


{/* banner prompt: Bright editorial illustration of a tidy tax workspace where receipts, bank statements and notes flow into an AI assistant, with a clearly labelled audit trail of source documents, assumptions and decisions, warm white background with teal and orange accents, optimistic and practical, 16:9. */}

An AI assistant can make a tax return feel much less intimidating. It can read a statement, sort expenses, explain an unfamiliar label and suggest questions to ask. But a useful answer is not enough. If a number later needs to be checked, corrected or explained to an accountant, the system must show how it got there.

That is why an AI-assisted tax workflow needs an audit trail.

## Start with the original record

Every claim should point back to a source. That might be an income statement, bank transaction, invoice, receipt, super contribution confirmation or insurance policy document. Keep the original file, its date and a short description of what it proves.

The AI can create a structured entry from that record, but it should not replace it. A useful entry might say:

- source: invoice dated 12 March
- amount: $119.40
- category: income protection insurance premium
- person: Eric
- financial year: 2025-26
- proposed treatment: review as a possible deduction

This makes the result understandable months later, even if the original conversation with the AI has disappeared.

## Record assumptions, not just answers

Tax decisions often depend on facts that are not printed on a receipt. Was an item used privately as well as for work? Was it reimbursed? What percentage was work-related? Was a payment made for a spouse, a child or a business?

An AI workflow should save those assumptions beside the calculation. For example: “Cloud storage used approximately 80% for nursing PhD research and 20% privately.” If the percentage changes, the calculation can be updated without losing the reason for the original figure.

This is especially important for couples. One person's income can affect spouse questions, offsets, family payments and other income tests. A shared record should identify who paid an amount, whose return it belongs to and which figures were transferred between returns.

## Separate suggestions from decisions

AI should be allowed to suggest possibilities, not silently make final tax decisions. A claim can have a simple status such as:

- proposed by AI
- confirmed by taxpayer
- reviewed by accountant
- excluded, with a reason

That small distinction prevents a guess from becoming an unexplained number in a lodged return. It also makes professional review faster because the accountant can focus on the uncertain items rather than rechecking every routine entry from scratch.

## Protect privacy before processing

Tax records contain highly sensitive information. Before sending documents to an AI service, consider removing names, addresses, account numbers, tax file numbers, policy numbers and unrelated transactions. Replace them with labels such as “Taxpayer A” or “Joint account”. Keep the original documents in secure storage and maintain a private map between the labels and the real identities.

Anonymisation reduces exposure, but it is not magic. Check the provider's retention, training and deletion settings, and do not upload more information than the task requires. For a simple categorisation question, a redacted transaction line may be enough; the entire bank statement may not be necessary.

## Keep a final snapshot

Before lodging, save a final snapshot of the return inputs, supporting records, assumptions, AI suggestions and human decisions. Record the date, financial year and the person who approved each item. If an accountant reviews the return, save their comments too.

The snapshot should be readable without a special AI product. A folder of PDFs, a spreadsheet or a plain text export is more durable than relying on a chat history that may later be unavailable.

## Convenience with accountability

The best use of AI in tax is not to produce a confident answer and move on. It is to make the taxpayer more organised, more informed and better prepared to ask for help when a situation is complex.

An audit trail turns AI from a black box into an assistant whose work can be inspected. That matters for ordinary self-managed returns, and it matters even more when a family has several connected claims, carried-forward losses or changing circumstances.

AI can reduce the cost of preparation. A good record ensures it does not increase the cost of fixing a mistake.


021c55437277f1f2f8a5c7eb2b0f813f4cf4d7d200307f12823273dba0c5f7b0

A practical guide to building an AI-assisted Australian tax workflow with privacy controls, source records and an audit trail that a taxpayer or accountant can review.

AI can make self-managed tax returns easier, but every suggestion needs a traceable record of its source, assumptions and final decision.

The AI Tax Workflow Needs an Audit Trail

{/* banner prompt: A vivid 3D infographic scene on a clean white background where a stream of jumbled receipts and paper documents flows through a friendly, glowing AI assistant icon and emerges as a single, perfectly organized digital tax return, all rendered in a modern, optimistic style with a palette of bright teal and orange. */}

The first wave of AI anxiety arrived in software development. That makes sense: code is structured, digital and unusually easy for a machine to generate, test and revise. The next wave may be quieter, but just as important. It is arriving in accounting, especially in the part of accounting that looks like collecting documents, classifying transactions and completing familiar forms.

That does not mean accountants disappear. It does mean some accountants' jobs are going to be cut, and many others are going to change.

## The return that used to need a professional

In Australia, individuals have long been allowed to prepare and lodge their own tax returns. The ATO's myTax service is available to people who want to lodge for themselves, with much of the information pre-filled from employers, banks, health funds and other institutions. [The ATO describes myTax as available to individuals and sole traders who want to lodge their own return](https://www.ato.gov.au/api/public/content/0-f9693a0f-35d4-4a60-912a-61913daaac0f).

The permission existed before generative AI. The difficulty was confidence.

Many people could technically do their own return, but did not know what a label meant, which records mattered, or whether an apparently small detail changed the result. Paying an accountant was partly payment for expertise, and partly payment for reassurance.

AI changes that balance. A person can ask, in plain language, what a field means. They can provide a folder of statements and ask for a checklist. They can compare a draft against last year's return, find a missing category, or turn a pile of receipts into a structured record. The work becomes less mysterious and less intimidating.

Nobody knows your own situation better than you do. If you do not know that a particular payment or expense exists, your accountant cannot help you claim or report it. I did not know that income protection premiums could be relevant to my tax return until I looked more closely. An AI assistant that scans an organised list of expenses may be able to flag that kind of possibility, along with questions to take to an accountant. It is not a substitute for checking the rules, but it can help people notice what they would otherwise miss.

Privacy is a legitimate concern. Before sending records to an AI service, taxpayers can anonymise or redact names, addresses, account numbers, tax file numbers and other identifiers, and keep the original documents securely. They should also understand the provider's data-retention and training settings. Anonymisation reduces exposure, but it does not remove the need to use a service they trust or to review what is being uploaded.

That is enough to move a large amount of routine work away from professional firms.

## Single returns are the obvious first target

For a single person with one employer, a bank account and a handful of ordinary deductions, the job is often relatively constrained. There may be judgment involved, but there are fewer relationships between the facts.

AI is particularly useful here because it can explain the process step by step without making the taxpayer search through a dense help page. It can ask the questions a careful preparer would ask:

- Was this expense paid personally?
- Was it reimbursed?
- What percentage was work-related?
- Do you have a receipt or another record?
- Is this income, a deduction, or a capital transaction?

The taxpayer still carries responsibility for the answer. But the barrier to getting to a sensible answer is much lower.

## Families are where the complexity starts to compound

Couples and families are different. A return is no longer an isolated document. One person's taxable income can affect the other person's spouse questions, offsets and eligibility calculations. Children, childcare, family payments, private health cover, super contributions and income tests can all connect the returns.

This is where an AI assistant can be more valuable than a simple tax calculator. It can maintain a shared timeline, remember which person paid an amount, keep a record of carried-forward losses, and show how one figure flows into another return. It can turn “What do I put here?” into “This number comes from Eric's final return; this other number comes from Min's income test.”

That assistance does not make the tax law simple. It makes the complexity navigable.

It also creates a new risk: a fluent answer can still be wrong. AI can confuse taxable income with assessable income, treat a capital loss as an ordinary deduction, or confidently classify a transaction without enough facts. A good system needs source records, explicit assumptions and a final review, not just a polished paragraph.

## What gets cut, and what remains valuable

The most exposed work is repetitive preparation: chasing standard documents, entering data, checking arithmetic, copying figures between related forms and answering the same basic questions. Firms that needed several people for this work may need fewer people, or may redirect them to more clients.

The valuable work moves upward. Accountants will still be needed for business structures, trusts, companies, international income, complicated capital transactions, audits, disputes and situations where the cost of an error is high. They will also be needed when a client does not merely need an answer, but needs someone to take professional responsibility for a judgment.

In other words, AI removes some preparation labour while increasing the premium on interpretation, review and accountability. That is the same pattern already visible in programming: fewer hours spent typing the obvious code, more time spent deciding what should be built and proving that it works.

## The new middle ground

The interesting future is not “everyone fires their accountant” or “AI changes nothing”. It is a much larger middle ground.

Some people will still outsource everything. Some will do everything themselves. Many will prepare a first draft with AI, then pay an accountant for a targeted review. Others will use an accountant once a year and an AI organiser throughout the year to keep records tidy.

That is a meaningful change in the market. The professional relationship becomes less about being the only person who can operate the form, and more about being the person who can recognise the edge of the form.

Accountants' jobs are therefore genuinely being cut by AI, after programmers' jobs began to be reshaped by it. But the deeper story is about access. More people can now participate in their own financial administration, ask better questions and understand the consequences of their choices.

The safest version of that future pairs AI's patience and organisation with human judgment where the facts stop being routine.

*The next post will look at what an AI-assisted tax workflow should record so that convenience does not come at the cost of an audit trail.*


b27c310c4542fa4c713b4e62ebf3b2a02359b47f2dac3142853566bfcb13fbbf

AI is changing Australian tax preparation by making self-managed returns easier, while complex family and business situations still need professional judgment.

AI is already changing programming. Tax and accounting may be next, not because accountants vanish, but because more people can now handle straightforward affairs themselves.

After Programmers, AI Is Coming for Accountants

{/* banner prompt: A soft matte clay-render 3D illustration, 16:9, of a parallel history timeline: rounded clay lanes flowing left to right with small clay portrait medallions and rounded bars marking reigns and lifetimes, a swirling terracotta Big-Bang burst with a small planet and moon on the left, classical columns and a pyramid through the middle, and a clean cream modern city silhouette on the right; palette of matte sage green, warm cream and terracotta orange with muted blue accents, soft studio shadows, gentle rim lighting, simple pale background, tactile plasticine texture, clean minimal composition, no text. */}

Reading an encyclopedia article tells you *what* happened. It rarely tells you what
was happening **everywhere else at the same time**. While Catherine the Great ruled
Russia, who sat on the throne in China, in Japan, in France? When did the first
pharaoh appear, and how long *after* the dinosaurs was that, really?

That question, *what else was happening then?*, is the whole idea behind the
**Parallel Timeline**, the flagship feature of
[Tyokiie](https://play.google.com/store/apps/details?id=au.com.tyo.wiki.android),
our offline-first reference app. It lays history out as a set of horizontal lanes so
you can read across cultures and across time at a glance.

## History, side by side

Open the timeline and you get a living chart of the past. Each **lane** is a region
or a theme: Europe, East Asia, the Middle East, plus Scientists, Composers, Rulers,
and more. Each **bar** is a reign or a life, drawn to scale on a shared year axis,
with a portrait pulled from Wikimedia Commons. Scroll left and right through the
centuries; pinch to zoom; drag with two fingers to make the lanes taller when you
want more detail.

![The Parallel Timeline in History view: reigns across Europe, East Asia and the Middle East, side by side around 1770.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/introducing-the-parallel-timeline/parallel-timeline-history.png?height=520)

Suddenly the connections jump out. In 1770, Catherine II of Russia, George III of
Great Britain, Louis XV of France, the Qianlong Emperor of China and Yeongjo of
Joseon were **all reigning at once**, a fact that is obvious on a parallel timeline
and nearly invisible in a stack of separate articles.

Tap any bar and a summary card slides up: a portrait, the dates, a one-line
description, and a button to **read the full article** right there in the app.

![Tapping an entry opens a summary card, here Catherine II of Russia (1762 to 1796), with a portrait and a link to the full article.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/introducing-the-parallel-timeline/parallel-timeline-card.png?height=520)

## Two ways to look at time

History has a scale problem. If you draw the last 400 years and the Big Bang on the
*same* ruler, one of them becomes an invisible sliver. So the Parallel Timeline gives
you **two modes**, and a single tap switches between them:

- **History view** — a normal, evenly-spaced year axis, now stretching all the way
  back to **3400 BC**, the dawn of the written record. Egyptian dynasties,
  Mesopotamian city-states, medieval rulers and the modern era, all on one line.
- **Cosmic view** — a **logarithmic** axis that plots *years before present*, so the
  whole of time fits on one screen: the **Big Bang (~13.8 billion years ago)**, the
  formation of the Earth and Moon, the Cambrian explosion, the age of dinosaurs, the
  rise of mammals, and the emergence of *Homo sapiens*, right up to the present day.

![Cosmic view: a logarithmic deep-time axis. The Moon forms 4.5 billion years ago, primates 60 million years ago, agriculture around 10000 BC, all above recorded history.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/introducing-the-parallel-timeline/parallel-timeline-cosmic.png?height=520)

In Cosmic view the deep-time lanes — **Cosmos, Life on Earth, Human Evolution,
Religions** — rise to the top, and dates read the way you actually think about them:
*"4.5 billion years ago," "66 million years ago," "300,000 years ago."* It is the
"cosmic calendar," but interactive, and in your pocket.

## Built the way we build everything

The Parallel Timeline reflects the principles behind the whole app:

- **Offline-first.** The timeline data — hundreds of entries across dozens of lanes,
  curated from **Wikidata** and illustrated from **Wikimedia Commons** — is bundled
  right into the app. No account, no cloud, no waiting.
- **Straight to the source.** Every entry links to its full **Wikipedia** article, so
  a moment of curiosity is one tap from the real thing. (The same offline article
  engine we wrote about in
  [Using Tyokiie for Offline Wikipedia Searching and Viewing](/blog/using-tyokiie-for-offline-wikipedia-viewing/).)
- **Speaks your language.** Lane names, entity titles and descriptions are localized
  into **15 languages** — read the timeline in English, 中文, Español, Français, 日本語
  and more, and jump to the matching-language article.
- **Yours to shape.** A filter lets you choose exactly which lanes and which countries
  appear, so you can focus on the Roman world, or East Asian dynasties, or just the
  composers — whatever you are exploring today.

## Try it

History isn't a list, it's a landscape. The Parallel Timeline lets you stand back and
see the shape of it, then dive into any moment that catches your eye.

You can install Tyokiie from the
[Google Play Store](https://play.google.com/store/apps/details?id=au.com.tyo.wiki.android).
Then tap **Explore → Parallel Timeline**, and switch on **Cosmic view** to travel from
the Big Bang to this morning. We would love to hear which unexpected "they were alive
at the same time?!" moment you find first.

*— The TYO Lab team*


ec2b2398b23c9b85fb9f6aac7d5bd7036ef6b08309d7fb984955844c4c13d5b8

The Parallel Timeline in Tyokiie, our offline-first reference app, charts history as side-by-side lanes with two modes: a linear view back to 3400 BC and a logarithmic cosmic view stretching from the Big Bang to the present day.

Tyokiie's flagship feature lays history out as parallel horizontal lanes, so you can see who ruled where at the same moment, from the Big Bang all the way to today, entirely offline.

Introducing the Parallel Timeline: See History Side by Side, from the Big Bang to Today

{/* banner prompt: An intricate technical blueprint on off-white paper of a cross-machine AI agent messaging system, showing four labeled workstation nodes arranged around a central hub labeled 'tyo-mq BROKER', connected by clean precise lines to a database cylinder labeled 'ROOMS' and small envelope icons labeled 'msg', with a legend reading 'pub / sub', dimensional annotations and detailed callouts, monochrome blueprint-blue schematic diagram style.

Additional instructions: The title text must read exactly 'AI AGENT MESSAGING SYSTEM' spelled correctly with no duplicated letters; label the central transport as 'socket.io / WebSocket', do NOT use MQTT or port 1883; every label spelled correctly. */}

Across our workshop there are, on a busy day, half a dozen AI coding agents running at once — [Claude Code](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/) sessions on a desktop, a couple of laptops, and an always-on NUC in the corner. Each one is doing real work: one is deep in a Strapi backend, another is babysitting a proxy gateway, a third is drafting posts. For a long time they had no idea the others existed. Each session was an island.

This post is about the bridge we built between those islands: a group chat for agents. They can open a room, message a peer on another physical machine, and — the part that actually matters — **wake an idle agent up** to answer. It runs on top of **[tyo-mq](https://github.com/tyolab/tyo-mq)**, a publish/subscribe message broker we first shipped in 2024, long before any of this had an AI use case.

The timing is worth being honest about, because it cuts both ways. As I write this, the harness these agents run in has grown *native* multi-agent features — named subagents, and a primitive for one agent to hand work to another. That is genuinely useful and we use it. But it solves a narrower problem: subagents inside **one** session on **one** machine. What we needed, and built first, was different — long-lived peer sessions on **different** machines, holding a durable conversation that survives restarts. The infrastructure for that was already sitting in our stack. This is how we wired it together.

## Sessions are islands

An AI coding agent is a process. It has a conversation history, a working directory, a set of tools — and a hard boundary at the edge of its own process. Two sessions on the same laptop can't see each other any more than two browser tabs can. Two sessions on different machines are further apart still.

That's fine until the work stops being independent. Our backend agent finishes a change to an API and needs the frontend agent — on another machine — to regenerate a client against it. A research agent turns up something the writer agent should fold into a draft. Today the human is the message bus: read one terminal, retype into another, walk the context across by hand. It works, and it does not scale past about two windows.

We wanted the agents to do that hand-off themselves. Which turns out to need two very different things at once, and confusing them is where most naive attempts fall down.

## Two layers: a system of record, and a nerve

A message has to be **remembered** and it has to be **noticed**. Those sound like the same requirement; they are not.

*Remembered* means durable and ordered: when the backend agent says "the API is ready," that sentence must still be there, in the right order, after the reader restarts — and each reader should see it exactly once. That is a database's job.

*Noticed* means live: an idle agent, sitting at a shell prompt doing nothing, should stir within seconds of a message landing — not on its next poll ten minutes from now. That is a message bus's job.

So we built the system in two layers, and kept them strictly separate.

<Table data={
    {
        headers: ['', 'System of record', 'Nerve'],
        rows: [
            ['Built on', 'PostgreSQL', 'tyo-mq'],
            ['Answers', 'What was said, and did I read it?', 'Did something just happen?'],
            ['Guarantee', 'Durable, ordered, once per reader', 'Best-effort, fire-and-forget'],
            ['Carries', 'The full message body', 'A one-line signal, no body'],
            ['If it is down', 'Agents cannot talk', 'They still talk, just not instantly'],
        ]
    }
} />

The Postgres layer is the truth. The tyo-mq layer is a tap on the shoulder that says "go check the truth." Keeping the body out of the nerve entirely — the nudge never carries what was said — is what lets the two layers fail independently. Lose the broker and nothing is lost; delivery just goes back to polling. Lose Postgres and there is nothing to deliver anyway.

## The rooms live in Postgres

The system of record is four small tables: `conversations` (a room is just a name and a title), `conv_members` (who is in which room, and — the important column — each member's `last_read_id` cursor), `conv_messages` (an append-only log with a `bigserial` id), and `agents` (identity, which we'll get to). That's the whole model. Rooms are named, membership is explicit, and every member carries their own read position.

Agents don't touch those tables directly. They get a handful of tools, exposed over [MCP](/blog/2026/06/15-what-a-native-tool-call-actually-is/) — the same tool-calling channel the model already uses to read files and run commands. To an agent, talking to a peer looks exactly like any other tool call.

<Table data={
    {
        headers: ['Tool', 'What it does'],
        rows: [
            ['start_conv', 'Create a room (or join it if it exists) and subscribe me'],
            ['join_conv', 'Join an existing room by name'],
            ['leave_conv', 'Leave a room and stop receiving its messages'],
            ['say', 'Post a message to a room I have joined'],
            ['poll_convs', 'Fetch my unread messages, grouped by room; delivered once'],
            ['list_convs', 'List my rooms with unread counts'],
            ['list_all_convs', 'List every room on the server, for discovery'],
            ['who', 'List the members of a room'],
        ]
    }
} />

`poll_convs` is where the read cursor earns its place. It hands back only messages newer than my `last_read_id`, then advances that cursor — so each message reaches each reader exactly once, and I never re-read my own history.

There is one subtlety that took a real bug to appreciate: **message ordering**. Postgres assigns the `bigserial` id at insert, but two concurrent inserts can *commit* out of order — and a reader polling in that window could advance its cursor past a lower id that hadn't committed yet, skipping a message forever. So `say` takes a per-room advisory lock before inserting:

```python
async with c.transaction():
    await c.execute("SELECT pg_advisory_xact_lock(hashtext($1))", conv_id)
    mid = await c.fetchval(
        "INSERT INTO conv_messages(conv_id, from_agent, body) "
        "VALUES($1,$2,$3) RETURNING id", conv_id, agent_id, body)
```

Appends to one room serialize; different rooms never contend. Commit order now matches id order, and the cursor is safe. It's the kind of guarantee you only notice when it's missing.

## Every session is its own identity

Here is a trap we walked straight into. All the sessions on one machine share a single credential — one bearer token, one identity. So the backend agent and the frontend agent on the same laptop looked like *the same agent* to the server. They couldn't message each other, because the server dutifully hid an agent's messages from itself.

The fix is that identity has to be **per session**, not per machine, and it has to be visible to every part of the system that runs in that session — including a hook that runs in a *separate process* (more on that below). The only channel both halves can see is an environment variable set at launch.

So each session checks in when it starts and gets a distinct handle from a monotonic counter: `work3-agent#1`, `work3-agent#2`, or a named one like `work3-agent-reach#1`. That handle rides on every request as an `X-Tyode-Agent` header, and the server composes the effective identity in exactly one place:

```python
def compose_agent_id(box, handle):
    # box comes from the (trusted) bearer token; handle from the header.
    if handle and (handle == box or handle.startswith(box + "-")):
        return handle          # honor the session handle
    return box                 # otherwise fall back to the bare machine id
```

That `startswith(box + "-")` check is the anti-spoof rule. The machine's token is the real trust boundary; the session handle is only honored if it belongs to that machine's namespace, so `work3` can name itself `work3-anything` but never `devmac-something`. Same-box sessions are deliberately *not* isolated from each other — they're the same human at the same desk — but no machine can impersonate another.

## The bug that taught us the rule

Per-session identity created a second, subtler problem, and it's a good illustration of why this stuff is fiddly. Eric pinged me one day: an agent was **receiving its own message back**. It had said something in a room and the message bounced straight back to it.

The cause was that a machine has more than one identity in play: the per-session handles (`work3-agent-reach#1`) *and* a bare-machine fallback (`work3`) for any session that launched without checking in. Both had ended up as members of the same room. When one session spoke, the *other* identity on the same machine — the same person, the same desk — dutifully received it. To the human it read as an echo of themselves.

The rule that fixes it has to thread a needle. We *want* two sessions on one machine to hear each other — that's the entire point. We do *not* want a machine to hear itself. So:

- A message never goes back to its exact sender.
- The bare-machine identity ignores all of its own sessions.
- A session ignores the bare-machine identity, but still hears its sibling sessions.

Cross-machine delivery is untouched. It's a few lines of SQL predicate, but the design behind it — *which* of a machine's identities count as "me" — is the whole game.

## Push, not poll: waking an idle agent

Everything so far is pull. An agent sees new messages when it calls `poll_convs`. That's fine for an agent mid-task, which reaches a natural checkpoint every few seconds — we hang delivery off the harness's turn boundary with a **Stop hook**, a script the harness runs each time the agent finishes a turn. It quietly polls, and if a peer has spoken it surfaces the message. In its default *engage* mode it even blocks the turn from ending so the agent has to read and decide whether to reply; a *notify* mode just prints. A damper stops two agents in engage mode from ping-ponging forever.

But an **idle** agent — sitting at a prompt, no turns happening — has no turn boundary to hang anything on. Polling on a timer would be the obvious hack, and it's the wrong one: it burns tokens and still adds latency. This is exactly the "did something just happen?" question, and it's what a message bus is *for*.

So `say`, after it has safely committed to Postgres, fires a one-shot publish to tyo-mq — a signal, not the message:

```js
producer.produce('msg', { conv, from, at });   // no body, ever
```

On each interactive machine, an opt-in notifier daemon subscribes to that stream and, when a nudge lands for a room it cares about, wakes the idle session — literally typing `/catchup` into its terminal, with an optional bell or desktop notification. It never consumes from the room itself, so it can't race the agent's own `poll_convs`; it only rings the bell. Idle-to-responding drops from minutes to about a second.

```js
consumer.subscribe('agent-comms', 'msg', onNudge);   // producer, event, handler
```

## tyo-mq: the part we didn't have to build

That nerve layer is **tyo-mq**, and it's the reason this whole feature was a week of wiring rather than a quarter of infrastructure. It's our own distributed publish/subscribe broker — "a distributed messaging (pub/sub) service with socket.io," as its README puts it — and by the time the agents needed it, it had already been running in our stack for years, on entirely unrelated jobs.

The model is the classic one, kept deliberately small. Producers publish to a named **event**; consumers subscribe by producer and event and get a callback. Transport is [socket.io](https://socket.io) over WebSocket, so the *same* client works in Node, in a browser, and — through separate client libraries speaking one shared wire protocol — in Python, Go, Rust, Ruby, Java, C/C++ and C#. Fire-and-forget by default; **durable delivery** (acknowledgements, retry, dead-letter queues) is opt-in when a message actually has to arrive.

<Table data={
    {
        headers: ['Language', 'Client'],
        rows: [
            ['Node / browser', 'npm i tyo-mq-client'],
            ['Python', 'pip install tyo-mq-client'],
            ['Java', 'au.com.tyo:tyo-mq-client'],
            ['Go / Rust / Ruby / C / C++ / C#', 'one client repo each, same protocol'],
        ]
    }
} />

Two design choices made it the right fit for agent traffic specifically:

**Realms.** A realm is a hard isolation boundary — one tenant's producers and consumers simply cannot see another's. Agent nudges run in their own realm, walled off from every other thing on the broker, with their own auth policy (down to whether anonymous clients may connect). Realms can even be *ephemeral*, with a time-to-live after which their tokens, sockets and queued messages are swept away — handy for short-lived swarms. And the whole auth configuration hot-reloads on a `SIGHUP`, so we provisioned the agents' realm into the live broker with zero downtime.

**A clean licensing split.** The broker is AGPL; the *client* libraries are Apache-2.0 with no AGPL dependency — the same server-vs-driver split MongoDB and Elasticsearch use. You can embed a tyo-mq client in a closed-source product without the server's copyleft reaching into your code. That mattered here because the agent notifier is just another client.

If you want to *see* it rather than read about it, the public broker at **[freemq.tyo.com.au](https://trymq.tyo.com.au)** runs live demos: a lobby chatroom with a small local model as its host, and turn-based games — word-chain, Hangman, Wordle — where the rules are refereed in code and a 3-billion-parameter model plays the moves, everything flowing over tyo-mq events. The agent group chat is the same broker doing a more serious job.

## Why two layers, and not just the bus

A fair question: tyo-mq *has* durable delivery — why not run the whole thing on it and drop the database?

Because rooms are a *readership* problem, not just a *delivery* one. "Which of the last 200 messages has this particular member read, given they were offline for three restarts and joined the room late?" is a query, and Postgres answers it in a line of SQL. Rebuilding per-member read cursors, backfill-on-join, and exactly-once-per-reader on top of a message bus would be reinventing a database badly. Conversely, driving an idle process awake in a second is not something a database wants to do; that's the bus.

Each layer does the thing it's built for. The bus notices; the database remembers. Neither pretends to be the other, and the seam between them — a body-less nudge — is the whole trick.

## We had this before the platform did

I'll close where I started, because it's the honest frame. Native agent-to-agent messaging is arriving in the coding harnesses now, and it's good. But "agents that talk" is not one feature — it's at least two problems wearing a trench coat. Subagents coordinating inside one process is the easy half. Independent, long-lived agents on different physical machines, holding a conversation that outlives any one of them, is the half that needs a system of record and a nerve — a database and a message bus, kept honest about which is which.

We could build the second half quickly for one reason: the hard, generic piece — a multi-language, multi-tenant, durable pub/sub broker — we'd already shipped, in 2024, as a product. The AI agents are just its newest client. That's what tyo-mq is now: not a side project, but the core product a lot of the rest of the workshop quietly runs on.

*Next in this series: what happens when you let those agents not just talk, but hand each other real work — a job queue across machines.*


2f0101ea6238529776114588250f89ed3bc922768b53d43f068d1e78b78c25b4

The design of a cross-machine inter-agent messaging system for AI coding agents: a durable rooms model in Postgres, anti-spoof per-session identity, and push delivery over tyo-mq, our socket.io pub/sub broker.

Our AI agents — Claude Code sessions running on four different machines — share a group chat: they start rooms, message each other, and can wake an idle peer to respond. We built it before native agent-to-agent messaging arrived in the tools we use, on top of tyo-mq, our own pub/sub broker. Here is the design.

Cross-Machine AI Agent Messaging, Built on tyo-mq


{/* banner prompt: A playful but polished 3D clay-render, matte materials in sage green, cream and terracotta, of a stylized detective magnifying glass hovering over a small padlock icon that is cracking open, with a tangled network cable looping around a proxy-server box in the background, soft studio shadows, simple background, 16:9. */}

`yt-dlp` gave us the least helpful error message it has: `HTTP Error 410: Gone`. The standard fix is `yt-dlp -U`. We ran it. Already on the latest stable release. The video was not, in fact, gone — but proving that took three separate bugs stacked on top of each other, and the last one had nothing to do with `yt-dlp` at all.

We were debugging a small local downloader tool — a browser extension that hands a page URL and cookies to `yt-dlp` on the user's own machine, optionally routed through a local HTTP proxy for geo-unblocking. One specific site started failing every time, on a URL that worked fine a week earlier.

## The first clue was buried in `-v`

The plain error told us nothing. Adding `-v` for verbose output surfaced a line that's easy to scroll past:

```
WARNING: [SiteExtractor] The extractor is attempting impersonation, but no
impersonate target is available. If you encounter errors, then see
https://github.com/yt-dlp/yt-dlp#impersonation for information on
installing the required dependencies
```

That's the real signal. A growing number of sites fingerprint the TLS handshake and HTTP/2 frame ordering of incoming requests to tell a real browser apart from a scripted client — even when every header looks identical. `yt-dlp` can spoof a browser's TLS fingerprint via an optional dependency, `curl_cffi`, which wraps `libcurl` with impersonation profiles for Chrome, Safari, Firefox and so on. Without it, some extractors still try the request, get rejected at the TLS layer, and the site's edge returns a generic `410` instead of anything that explains what actually happened. The video was fine. `yt-dlp` just didn't look enough like a browser to be allowed to ask.

## The version window nobody tells you about

The fix looked trivial:

```sh
python3 -m pip install curl_cffi
```

It installed cleanly — `curl_cffi-0.16.0`, the latest release. `yt-dlp` still refused to use it, and `--list-impersonate-targets` came back with every target marked `(unavailable)`. The debug line was the giveaway:

```
[debug] Optional libraries: ... curl_cffi-0.16.0 (unsupported) ...
```

`yt-dlp` doesn't just check that `curl_cffi` is importable — it pins a narrow set of versions it actually trusts, because `curl_cffi`'s impersonation API has changed shape across releases. Digging through the installed `yt-dlp` build (it ships as a zip-app; `python -m zipfile` opens it like any other zip) turned up the exact check:

```python
curl_cffi_version = tuple(map(int, re.split(r'[^\d]+', curl_cffi.__version__)[:3]))

if curl_cffi_version != (0, 5, 10) and not (0, 10) <= curl_cffi_version < (0, 16):
    curl_cffi._yt_dlp__version = f'{curl_cffi.__version__} (unsupported)'
    raise ImportError('Only curl_cffi versions 0.5.10 and 0.10.x through 0.15.x are supported')
```

Latest-and-greatest was one minor version too new. The fix was to pin below the ceiling:

```sh
python3 -m pip install "curl_cffi>=0.10,<0.16"
```

**Lesson:** when a library depends on another native-extension package for an optional feature, don't assume "latest" is "supported" — check for a version gate specifically on the feature you're trying to turn on, not just on whether the import succeeds.

## Collateral damage from `--force-reinstall`

The first install attempt used `--force-reinstall` to make sure the correct version actually took. It did — but it also silently upgraded packages that were already correctly pinned for something else entirely: `certifi`, `cffi`, `markdown-it-py`, and `pygments` all jumped to newer versions than another tool in the same shared Python environment required. `pip` said so plainly, right at the end of the install:

```
ERROR: pip's dependency resolver does not currently take into account all
the packages that are installed. This behaviour is the source of the
following dependency conflicts.
some-other-tool 2.15.0 requires certifi==2026.4.22, but you have
certifi 2026.7.22 which is incompatible.
some-other-tool 2.15.0 requires cffi==2.0.0, but you have cffi 2.1.1 ...
```

None of those four packages actually needed upgrading — `curl_cffi`'s own requirement (`cffi>=2.0.0`, `certifi>=2024.2.2`) was already satisfied by the versions in place. `--force-reinstall` doesn't just reinstall the package you named; it re-resolves and reinstalls its entire dependency subtree, and when it does that it grabs the latest version satisfying each constraint rather than leaving an already-satisfied pin alone. In a shared base environment with several unrelated tools installed, that's a quiet way to break something you weren't even looking at.

The fix was to pin those four straight back:

```sh
python3 -m pip install "certifi==2026.4.22" "cffi==2.0.0" \
  "markdown-it-py==4.0.0" "pygments==2.20.0"
```

**Lesson:** avoid `--force-reinstall` in a shared or base Python environment. If you need a specific version of one package, ask for exactly that (`pip install "pkg==x.y.z"`) rather than forcing a full dependency re-resolve you don't control.

## A new error, and a wrong theory

With the right `curl_cffi` version in place, the impersonation warning was gone — replaced by a new one, appearing on every single attempt:

```
SSLError: Failed to perform, curl: (35) TLS connect error:
error:00000000:invalid library (0):OPENSSL_internal:invalid library (0)
```

This looks exactly like a known class of `curl_cffi` bug: its bundled BoringSSL build can clash with another OpenSSL-family library already loaded in the same Python process — commonly `cryptography`, pulled in transitively by something unrelated like browser-cookie decryption support. The natural fix is isolation: a clean virtual environment with only `yt-dlp` and `curl_cffi`, nothing that could drag in a conflicting crypto library.

We built that environment. Same error, 100% reproducible, in complete isolation — no `cryptography`, no `secretstorage`, nothing else installed at all.

That result was more useful than a fix would have been. A clean-environment reproduction rules out an entire category of theories in one shot. It wasn't a Python packaging conflict. It had to be something outside Python's import graph.

## Stripping back to the dumbest possible test

The next step was to remove every layer of abstraction and test the actual thing that was failing: a TLS handshake over the proxy. Not `curl_cffi`. Not even `yt-dlp`. Plain, unimpersonated `requests`:

```python
import requests
r = requests.get("https://www.example.com",
                  proxies={"https": "http://127.0.0.1:8082"})
```

Same SSL error. Against a domain with nothing to do with the original site. That was the moment the whole investigation flipped: this was never a `curl_cffi` bug, or even a Python bug. Something about the proxy itself was breaking HTTPS.

Raw `curl` confirmed it in the plainest terms available:

```
* CONNECT tunnel established, response 200
> Client hello (1)
* OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to www.example.com:443
```

The proxy's `CONNECT` tunnel came up cleanly — a `200 Connection Established` — and then the TLS handshake died immediately afterward, for every client, on every site. Nothing about `curl_cffi`'s TLS fingerprint mattered; the tunnel itself wasn't relaying bytes correctly once real TLS traffic started flowing through it.

The proxy client had several selectable exit gateways by region. Switching from one region to another and rerunning the exact same `curl` command fixed it instantly — clean `200`, valid response, every time. The original gateway node was in a broken state for HTTPS relaying specifically; nothing in our code, `yt-dlp`, or `curl_cffi` was ever going to fix that from the client side.

## What actually shipped

Three real problems, three real fixes, none of them where the first error message pointed:

<Table data={
    {
        headers: ['Symptom', 'Looked like', 'Actually was'],
        rows: [
            ['HTTP 410: Gone', 'Stale yt-dlp / dead video', 'Site requires TLS impersonation'],
            ['curl_cffi "(unsupported)"', 'Broken install', "Version above yt-dlp's pinned ceiling"],
            ['Unrelated tool broke after install', 'Unrelated bug', '--force-reinstall re-resolved shared deps'],
            ['SSL "invalid library" error', 'curl_cffi vs OpenSSL clash', 'Proxy gateway failing the TLS relay'],
        ]
    }
} />

The lasting fix was to isolate `curl_cffi`-enabled `yt-dlp` into its own virtual environment — so a future dependency bump anywhere else on the machine can never touch it again — and to symlink the `PATH`-resolved `yt-dlp` at that environment's build, so every caller picks it up without extra configuration.

## The general shape of this kind of bug

None of the individual fixes here are exotic. What made this one take longer than it should have was trusting each error message's own framing for too long:

- **"410 Gone" told us the video was gone.** It wasn't — it was an anti-bot response wearing the shape of a different error.
- **"unsupported" told us the install was broken.** It wasn't broken, it was one version too new for a pin we didn't know existed.
- **"invalid library" pointed straight at `curl_cffi` and OpenSSL.** It was neither — it was a proxy gateway with a bad TLS relay.

The pattern that actually broke each false lead was the same one every time: reproduce with the fewest possible moving parts. A clean virtual environment ruled out the packaging theory. Plain `requests` with no impersonation ruled out `curl_cffi` entirely. Raw `curl` against an unrelated domain ruled out the target site. Each step removed one whole category of suspects instead of guessing at the next fix — and the actual root cause, when it finally showed up, wasn't in any of the code we'd been staring at.


9fa2ef35cdb16bce3f05bf518227a3d6de0d42440772c0f4c3b4eaddebedb611

How we debugged a yt-dlp 410 Gone error that wasn't staleness: curl_cffi TLS impersonation, a strict version-pinning gotcha, a --force-reinstall dependency mishap in a shared Python environment, and a proxy gateway failure that mimicked a library bug.

A yt-dlp download kept failing with 'HTTP Error 410: Gone' even on the latest version. The real cause was three layered problems stacked on top of each other: missing TLS impersonation, a narrow curl_cffi version-compatibility window, and an unrelated proxy gateway fault that looked exactly like a library bug.

Debugging a Fake 410: yt-dlp, curl_cffi, and a Proxy That Lied


{/* banner prompt: A clean isometric technical infographic. Two glowing email envelopes, each sealed inside its own separate glass vault. A small friendly robot stands at each vault holding a single distinct key that fits only its own vault. A third robot tries the wrong vault and its key bounces off, met by a red shield symbol reading "403". Light-gray background, muted blue-and-teal palette with a single red accent, sharp clean vector illustration, strong sense of enforced separation. */}

We were asked to give two colleagues an AI assistant that could work with their email — read the mailbox, search it, pull an attachment, draft a reply for them to review and send. Standard stuff. Microsoft Graph exposes all of it, and wiring an AI agent to Graph through a small server takes an afternoon.

The hard part wasn't the plumbing. It was one sentence in the requirements: *each person's assistant must be able to touch only that person's mailbox.* Not "should." Must. And "must" is a security claim — the kind you have to be able to prove, not merely intend.

This is the story of how we made that boundary real, and the three places it tried to fool us into thinking it was already there when it wasn't.

## Application permissions are tenant-wide by default

To act on a mailbox without a signed-in user, a Graph app uses *application permissions* and the client-credentials flow: the app authenticates as itself and calls the API. The permission we needed was `Mail.ReadWrite` — it covers reading too, so there's no separate read permission to add. Crucially, we did **not** grant `Mail.Send`. The app can create drafts but physically cannot transmit them; every reply lands in a Drafts folder for a human to send.

Here's the trap. `Mail.ReadWrite` as an *application* permission is **tenant-wide**. Once an admin consents, the app can read and write *every mailbox in the organisation* — not the one you had in mind, all of them.

We confirmed this the blunt way, before trusting anything. We took the app's own credentials, asked Microsoft for a token, and called the API against a *different* colleague's mailbox. It returned their inbox, cheerfully, with a message count. The app we had registered "for one person" could read the whole company.

So the app permission is not the boundary. It's the blast radius. The boundary has to come from somewhere else.

## Put the wall where a bug can't reach it

There are two ways to constrain which mailbox an app can touch.

The tempting one is to do it in your own code. The server knows which user's token is calling, so it just... refuses any request for a different mailbox. One shared app, a lookup table, a check before every call.

Don't. That wall lives entirely in your code, which means one missed check, one refactor, or one clever prompt-injection that talks your agent into passing a different address, and it's gone. When the requirement is "must," you don't want the enforcement sitting in the same place as the bugs.

The better option is a control the provider enforces: an **Application Access Policy** in Exchange Online. You create one small mail-enabled group per app, containing exactly the mailbox (or mailboxes) that app is allowed to touch, then bind a `RestrictAccess` policy from the app to that group. After that, Microsoft itself refuses any Graph call from that app to any mailbox outside the group — a `403`, at the source, regardless of what your code does or doesn't check.

So we registered **one app per person** — not one shared app with code checks — and pinned each to a single mailbox with its own access policy. Now the wall between the two users is enforced by Microsoft. A bug in our server cannot leak one person's mail to the other, because the credential itself cannot see it.

The general principle: *when you can push a boundary down to the layer that owns the data, do it.* Your code should rely on the wall, not be the wall.

## The setup, in PowerShell

Application Access Policies live in Exchange Online, so the whole thing is a short, one-time PowerShell run — one mail-enabled group and one `RestrictAccess` policy per app. (Placeholders below: `contoso.com`, two people `alice` and `bob`, and the Application/client ID of each person's app registration.)

```powershell
# Run once, interactively, signed in as an Exchange/Global admin on a managed device.
Connect-ExchangeOnline -UserPrincipalName admin@contoso.com

# One app registration per person; paste each app's Application (client) ID.
$aliceApp = '<alice-app-client-id>'
$bobApp   = '<bob-app-client-id>'

# One mail-enabled security group per app, holding only that person's mailbox(es).
New-DistributionGroup -Name 'sg-mail-alice' -Type Security `
  -PrimarySmtpAddress 'sg-mail-alice@contoso.com' -Members 'alice@contoso.com'
New-DistributionGroup -Name 'sg-mail-bob' -Type Security `
  -PrimarySmtpAddress 'sg-mail-bob@contoso.com' -Members 'bob@contoso.com'
# (A group can hold more than one mailbox — add every address that app may touch.)

# Pin each app to its group. After this, Graph refuses any mailbox outside it.
New-ApplicationAccessPolicy -AppId $aliceApp -PolicyScopeGroupId 'sg-mail-alice@contoso.com' `
  -AccessRight RestrictAccess -Description 'mail agent: alice -> own mailbox only'
New-ApplicationAccessPolicy -AppId $bobApp   -PolicyScopeGroupId 'sg-mail-bob@contoso.com' `
  -AccessRight RestrictAccess -Description 'mail agent: bob -> own mailbox only'

# Verify the policy DEFINITION: Granted for own mailbox, Denied for the other.
Test-ApplicationAccessPolicy -Identity alice@contoso.com -AppId $aliceApp   # Granted
Test-ApplicationAccessPolicy -Identity bob@contoso.com   -AppId $aliceApp   # Denied
Test-ApplicationAccessPolicy -Identity bob@contoso.com   -AppId $bobApp     # Granted
Test-ApplicationAccessPolicy -Identity alice@contoso.com -AppId $bobApp     # Denied
```

Those last four lines are Microsoft telling you the policy is correct. Which brings us to the first trap.

## "Configured" is not "enforced"

We created the policies and ran Microsoft's own checker, `Test-ApplicationAccessPolicy`, for every combination — each app against its own mailbox, each app against the other's. All four came back exactly right: Granted where it should be, Denied where it shouldn't. Green across the board.

Then we ran the *real* test — the app's credentials against the live API, the way the agent would actually use it — and every mailbox still returned `200`. Wide open. The policy was defined, verified by Microsoft's own tool, and completely un-enforced at runtime.

`Test-ApplicationAccessPolicy` tells you what the policy *says*. It does not tell you the enforcement layer has caught up. Those are different systems, and on the day we set this up they were **about four hours apart**.

The lesson generalises well beyond email: verify a security boundary the way it is actually exercised, not with the vendor's "is it configured?" checker. For an API boundary, that means real calls, with the real credentials, returning the real deny.

## Propagation is slow *and* uneven

The four-hour wait had a sting in the tail.

We polled the live API in the background, watching for the cross-mailbox calls to start returning `403`. Eventually they did — and we nearly declared victory on the spot, because we were only watching the deny cases.

Good thing we looked at the whole picture. Enforcement hadn't landed atomically. The blanket "deny everything" propagated first; the per-mailbox *allow list* lagged behind it. For a window, each app was denied **its own** mailbox — the two users would have been locked out of their own email, while strangers' mail was (correctly) refused. The deny rule was live; the "except your own" hadn't caught up.

If we had gated go-live on "cross-mailbox returns 403," we'd have shipped a broken permission set. The right gate was the *entire* matrix correct at once — every allow returning `200` **and** every deny returning `403` — not a convenient subset. When a distributed system rolls out a rule, don't assume the parts you didn't check moved in lockstep with the parts you did.

## A detour: you can't always automate the admin step

One more trap, because it cost real time. Creating those access policies needs Exchange Online admin PowerShell, and we wanted to run it unattended from a server. We couldn't, for reasons worth knowing up front:

- The old, universally pre-consented "Exchange Online Remote PowerShell" app that used to make headless sign-in easy has been **disabled** by Microsoft.
- The device-code sign-in flow — the usual fallback for a machine with no browser — **cannot show a consent screen**, so it dead-ends on any app that isn't already consented.
- And even past that, the tenant's **Conditional Access** policy refused to issue a token to an unmanaged, unregistered device. (The error, if you ever meet it, is `AADSTS530035`.)

The lesson here isn't a workaround; it's an expectation. Admin actions in a well-secured Microsoft 365 tenant are *designed* to resist unattended automation. Plan to run them interactively, once, from a managed device — and build everything else around that single human step instead of fighting to remove it.

## What to check if you're doing this

1. **Prove the blast radius before you trust anything.** Take the app's own credentials and call a mailbox it shouldn't reach. If it answers, your boundary isn't there yet.
2. **Enforce isolation at the provider, not in your code.** One app per scope, each pinned by an Application Access Policy. Make a bug in your code harmless by construction.
3. **Never grant more than the job needs.** Drafting replies needs `Mail.ReadWrite`, not `Mail.Send`. If the app can't send, no bug or injection can make it send.
4. **Verify with real calls, not the "is it configured" checker.** Configured and enforced are different systems, sometimes hours apart.
5. **Gate go-live on the whole matrix.** Every allow *and* every deny correct, at the same moment — not the deny cases alone.

The reassuring part: once it settles, it is genuinely solid. The isolation isn't a promise in our code or a line in a runbook — it's Microsoft returning `403` to a credential that was never allowed to see the data in the first place. That's the kind of wall you can point at and call "must."


f0877050944e48dbf3b70a3ed52cbfd8dccfd4e8e9b3f53b26e61ebea5ddb427

How to scope an AI assistant's Microsoft 365 mailbox access with Exchange Application Access Policies, why the boundary belongs at the provider layer, and the verification and propagation traps we hit along the way.

Giving an AI assistant read-and-draft access to a Microsoft 365 mailbox is easy. Making sure it can touch only the right mailbox — enforced by Microsoft, not by your own code — is where the real work is.

Letting an AI Read Email Without Letting It Read Everyone's

{/* banner prompt: A playful, polished 3D clay-render in matte sage green, cream and terracotta: a stylized NVIDIA RTX 3060 graphics card with a softly glowing rounded brain resting on top of it, a few small thought-bubble and gear props floating beside it to suggest reasoning, soft studio shadows, simple background, 16:9. */}

The first three posts in this series were about *making* things — the same RTX 3060
learning to generate [images](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/),
then [video](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/),
then [video with sound](/blog/2026/08/04-now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/).
This one is different. This time the card isn't drawing or filming anything — it's
*thinking*. I put a **27-billion-parameter language model on the same 12GB card**,
and it works.

The headline: **Qwen3.8-27B**, a reasoning model released only a few days ago, runs
locally on a consumer RTX 3060 that shipped back in **2021**. It answers questions, writes code,
reasons step by step — and it can even look at a picture and tell you what's in it.
The kind of model people assume needs a data centre, or at least a $2,000 graphics
card, runs on hardware a small business already owns.

## The machine (still the same one)

Same box as every post in this series: an RTX 3060 with **12GB of VRAM**, 64GB of
system RAM, Debian. Nothing upgraded. The card is the constant.

## Why this is surprising

A 27B model is *big*. The rule of thumb people quote is that you need a 24GB card
(an RTX 4090 or better) to run one at all — and its full-precision form is over
50GB, firmly data-centre territory. So "27B on 12GB" sounds like it shouldn't be
possible.

It is, because of two things working together: **quantization** and **offloading.**

## Fitting an 18GB model into 12GB

Quantization compresses the model's numbers to 4 bits each, shrinking it from 50GB+
down to about **18GB** with very little quality lost. That still doesn't fit in
12GB — but it doesn't have to. The card holds a bit more than half the model
(**about 57%**), and the rest lives in the 64GB of system RAM, streamed onto the
GPU as it's needed. The GPU does the heavy math; the spare layers wait in RAM.

The remarkable part is how *easy* it was. The video-with-sound model in the last
post took patches and careful memory tuning. This time it was genuinely two
commands — download, and run. The tool (Ollama) works out the GPU/RAM split
automatically.

## The honest numbers

<Table data={
    {
        headers: ['', 'On the RTX 3060'],
        rows: [
            ['Model', 'Qwen3.8-27B (released this month)'],
            ['Size on disk', '~18GB (4-bit)'],
            ['Runs on GPU', '~57% on the card, ~43% streamed from RAM'],
            ['VRAM used', '11 of 12 GB'],
            ['Speed', '~4.7 words-per-second (reads about as fast as you do)'],
            ['Can it see images?', 'Yes — it is multimodal'],
            ['Best for', 'Questions, coding help, explanations, one-image analysis'],
        ]
    }
} />

Four-ish words a second isn't fast — you watch the answer type itself out, roughly
at reading speed. This is not the tool for churning through a thousand requests an
hour. It's the tool for asking one good question and getting a thoughtful answer,
privately, on your own machine, for the cost of the electricity.

## It reasons — out loud

Qwen3.8 is a *reasoning* model: before it answers, it thinks. Ask it a coding
question and you can watch it work the problem — considering approaches, catching
its own mistakes, checking its logic — then give a clean final answer. Ask it to
find the longest palindrome in a string, and it doesn't just spit out code; it
reasons about edge cases and complexity first. That visible thinking is a big part
of why these newer models are so much more useful than the ones from a year ago.

## And it sees

Here's the part that closes the loop on this whole series. The model is multimodal,
so I showed it one of the AI-generated animal photos from our image experiments and
asked: *is this a real photograph, or AI-generated?*

It nailed it. It described the scene correctly — a brown bear in a rushing river,
autumn forest blurred behind it — and then, unprompted, listed exactly *why* it
looked artificial: fur that was "suspiciously smooth and painted," the
"characteristic milky AI bokeh" in the background, "melted" rocks, a glassy eye
lacking real reflections. A local model, on a 12GB card, giving a genuinely sharp
critique of a locally-generated image. The tools in this series can now check each
other's work.

## Try it yourself

If you have an RTX 3060 (or any 12GB card) and [Ollama](https://ollama.com)
installed, this is the whole thing:

```bash
ollama run hf.co/unsloth/Qwen3.8-27B-GGUF:Q4_K_M
```

The first run downloads the model; after that it's instant to start. Ask it
anything. To feed it an image, just put a file path in your message.

## Where this leaves a small business

Four posts, one unchanged computer. The same RTX 3060 went from a single 512px
still, to short video, to video with a soundtrack — and now to a reasoning model
that can answer questions, help write code, and read an image, all without a single
byte leaving the building.

That last point matters more than the speed. Running the model *locally* means your
questions, your customer data, your documents never touch someone else's server.
No per-message cloud bill, no data leaving your office, no dependency on a
subscription that can change its terms next quarter. For a lot of small businesses,
"good enough, private, and paid-for-once" beats "fastest" every time.

It isn't a data centre. It doesn't need to be.

*It's a five-year-old graphics card. What it can do keeps surprising me.*


948ef2e0820c8a951e9aa975e9f905c44c461234611a88494f1617dd4622ea0a

Running Qwen3.8-27B, a brand-new 27B multimodal reasoning model, locally on a consumer RTX 3060 with only 12GB of VRAM: how an 18GB model fits, honest tokens-per-second, what it can and can't do, and the one command to try it yourself with Ollama.

The fourth in our RTX 3060 series. Qwen3.8-27B — a 27-billion-parameter reasoning model released this month — runs on the same 12GB card. Not in the cloud, not on a $2,000 GPU. One command, and it thinks, codes, and even sees.

Now It Thinks: A 27B Language Model on the Same RTX 3060


{/* banner prompt: A clean isometric technical illustration of a Windows laptop caught between two tall cloud-office towers in different colors. A jammed identity turnstile made of key cards and token rings blocks the direct path from the laptop to the second tower. A confident blue mail bird carrying a sealed envelope flies around the turnstile through a separate glowing OAuth gateway. Light gray background, restrained blue, teal and amber palette, crisp editorial infographic style, no company logos, no readable text, no error-code typography. */}

The mailbox was healthy. Outlook on the web worked. The same account worked on other computers. Only one Windows PC refused to add it — and that PC happened to be Microsoft Entra joined to a *different* tenant.

That distinction sent us through three separate identity systems wearing one Outlook-shaped trench coat: Windows Web Account Manager (WAM), Microsoft 365 desktop licensing, and Outlook's own profile state. We cleared caches, counted accounts, reset applications, tested both Outlook clients, and captured the exact broker errors. Each step explained a symptom. None produced a dependable mailbox.

Our final answer was not another registry key. It was **use Thunderbird instead**.

This is what we learned, including how to configure Thunderbird without falling back to the basic-authentication password prompt that Microsoft 365 has already retired.

## The setup that triggered it

The Windows user signed in with an account from Tenant A. The device was fully Entra joined to Tenant A and held a Primary Refresh Token (PRT) for that identity. The user also needed a normal Exchange Online mailbox from Tenant B.

The Tenant B mailbox was not broken:

- Outlook on the web worked.
- The credentials and MFA worked.
- The mailbox worked on computers that were not joined to Tenant A.
- Autodiscover and Exchange Online were reachable.

That left the local identity context as the meaningful difference.

## Four errors, four different layers

The failure changed as we removed stale state. That was useful: each new message exposed the next layer underneath.

<Table data={
    {
        headers: ['Client or test', 'Visible symptom', 'What it actually told us'],
        rows: [
            ['New and classic Outlook', 'Add account operation is blocked because account limit is reached', 'Windows WAM believed the user had exhausted its Entra account slots'],
            ['New Outlook after reset', 'Your account is not supported', 'The mailbox could not license new Outlook as its first account'],
            ['Classic Outlook', 'An unexpected error occurred, tag 7anyj', 'The Windows identity broker could not match Outlook\'s login hint to a cached WebAccount'],
            ['Manual IMAP in Outlook', 'A plain username and password prompt', 'Outlook had selected basic authentication, which Exchange Online no longer accepts'],
        ]
    }
} />

Treating all four as "Outlook cannot sign in" would have produced an endless loop of reinstalls. They came from different components and needed different evidence.

## The account-limit detour

Windows 10 and 11 limit a user session to three Microsoft Entra accounts. Microsoft documents the exact resulting message: *"Add account operation is blocked because account limit is reached."* The limit exists to control token-request size and keep single sign-on reliable. See the [Microsoft Entra device management FAQ](https://learn.microsoft.com/en-ca/entra/identity/devices/faq).

The important words are **user session**. Our remote management shell ran as `SYSTEM`, so this command initially inspected the wrong profile:

```powershell
dsregcmd /listaccounts
```

The giveaway was immediate:

```text
USERPROFILE=C:\Windows\System32\config\systemprofile
```

WAM is per-user state. An elevated shell, service, RMM agent, or SSH daemon commonly runs under a service account and can return an empty list or a meaningless error even while the interactive user has a broken account store.

We used a temporary scheduled task with `LogonType=Interactive` to execute `dsregcmd /listaccounts` inside the signed-in user's session. After backing up and rebuilding the TokenBroker account directory, the real result was clean:

```text
User accounts: 1
Application accounts: 0
Default account: tenant-a-user@example.com
```

The count was no longer the blocker. Outlook still failed.

## The silent cleanup script trap

Microsoft provides `signoutofwamaccounts.ps1` for removing Office-associated WAM tokens on managed devices. It is a legitimate tool, but it has two operational surprises:

1. It is deliberately silent. Its asynchronous results are piped to `Out-Null`, so no output can mean success, failure before useful work, or simply no matching Office accounts.
2. It works on the current user's WAM store. Running it from an RMM shell as `SYSTEM`, or as a different administrator, does not clean the affected desktop user's account store.

Microsoft's [activation reset guidance](https://learn.microsoft.com/en-us/previous-versions/troubleshoot/microsoft-365/microsoft-365-apps/activation/reset-office-365-proplus-activation-state) correctly says the script removes Office tokens without unjoining an Entra-joined device. What it cannot do is make a service-context PowerShell session become the interactive user.

The reusable rule is simple: when troubleshooting WAM, always record `whoami`, `$env:USERPROFILE`, and the session ID beside every result.

## New Outlook exposed a licensing problem

Resetting the new Outlook package did not clear all of its data. Its packaged `LocalState` and `LocalCache` were empty, while `%LOCALAPPDATA%\Microsoft\Olk` still contained the account state. Renaming `Olk` to a timestamped backup finally produced a genuine first-run experience.

The next message was *"Your account is not supported."*

That was progress, not regression. New Outlook reads the licence attached to its accounts. Microsoft says a work or school account needs a plan that includes desktop applications to license Outlook as the initial account. Once a qualifying account is present, Business Basic or E1 mailboxes can be added as secondary accounts. The details are in [Microsoft's new Outlook licensing guide](https://learn.microsoft.com/en-us/microsoft-365-apps/outlook/get-started/licensing-options).

In this case the second tenant's mailbox was not a suitable first account. Adding the Tenant A account first was theoretically valid, but it pulled us straight back into the cross-tenant WAM path we were trying to escape.

## Classic Outlook gave us the decisive evidence

Classic Outlook replaced the licensing message with a generic page:

```text
Message: An unexpected error occurred.
Tag: 7anyj
Code:
```

The page itself was nearly useless. The timestamp was not.

We queried `Microsoft-Windows-AAD/Operational` for the surrounding twelve minutes. At the same second as the Outlook error, Windows logged an application-specific account warning. Before and after it, Event ID 1098 repeated this error:

```text
0xCAA100D8
A login hint was sent that doesn't match any WebAccount in the system.
```

That closed the loop. Outlook was asking WAM silently for Tenant B, but WAM contained only Tenant A. The browser credentials were fine. The Exchange mailbox was fine. The tenant was fine. The local broker could not map the requested identity to an account in its store.

We tried seeding the second Office identity through another desktop Office application. That also failed. At that point we could keep editing identity stores and registry associations, or we could choose a mail client that did not inherit the Windows session's PRT and Office WAM state.

## Why guest accounts and cross-tenant settings were not the first fix

Inviting the user as a guest into Tenant B sounds attractive, but a B2B guest object does not turn an existing mailbox login into a healthy Outlook desktop identity. Cross-tenant access settings can control trust, MFA claims, device claims, and inbound or outbound B2B access. They do not repair a local WAM account mapping.

Those settings matter when the Tenant B sign-in logs show a Conditional Access or cross-tenant policy denial. Our evidence stopped earlier: Windows could not even match the login hint to a local WebAccount.

Fix the layer that failed. Do not redesign tenant trust because a local token broker returned `0xCAA100D8`.

## Why Thunderbird was the practical answer

Thunderbird runs its own Microsoft OAuth flow and maintains its own account state. It does not need Outlook to reconcile Tenant A's Windows PRT with Tenant B's mailbox identity. That makes it a clean isolation boundary: the user signs directly into the mailbox's home tenant, completes MFA, and Thunderbird stores the resulting OAuth tokens in its own profile.

As of Thunderbird 153, native Exchange email over EWS is available without an add-on. Mozilla's [Thunderbird and Exchange guide](https://support.mozilla.org/en-US/kb/thunderbird-and-exchange) recommends version 145 or later and explains the Microsoft-hosted OAuth prompt. Thunderbird 153 made that support prominent in the new Account Hub; see the [Thunderbird 153 release announcement](https://blog.thunderbird.net/2026/07/thunderbird-153-meadow-is-out-now/).

For this kind of case, the immediate setup is:

1. Install Thunderbird 153 or newer.
2. Select **New Account → Email**.
3. Enter the Tenant B mailbox address.
4. Let Account Hub detect Exchange, or select **Manual Configuration → EWS**.
5. Complete the branded Microsoft OAuth and MFA flow for Tenant B.

Do not type the mailbox password into a "Normal password" IMAP configuration. Microsoft removed basic authentication from Exchange Online. OAuth is the difference between "IMAP is obsolete" and "IMAP works securely with Microsoft 365."

If EWS is unavailable, use IMAP and SMTP with OAuth2:

<Table data={
    {
        headers: ['Purpose', 'Server', 'Port and transport', 'Authentication'],
        rows: [
            ['Incoming mail', 'outlook.office365.com', '993, SSL/TLS', 'OAuth2'],
            ['Outgoing mail', 'smtp.office365.com', '587, STARTTLS', 'OAuth2'],
        ]
    }
} />

Microsoft explicitly lists Thunderbird as an Exchange Online IMAP client in its [POP3 and IMAP4 documentation](https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/pop3-and-imap4/pop3-and-imap4). Mozilla's [Microsoft OAuth guide for Thunderbird](https://support.mozilla.org/en-US/kb/microsoft-oauth-authentication-and-thunderbird-202) also notes two tenant-side caveats: an administrator may need to approve Thunderbird once, and SMTP AUTH may need to be enabled for sending.

## The EWS warning in August 2026

Native EWS is a good immediate bridge, not a forever architecture for Exchange Online. Microsoft begins phased EWS disablement on October 1, 2026 and plans full retirement on April 1, 2027. Administrators can temporarily allow approved EWS applications during the transition. Microsoft's current timeline is in [Deprecation of Exchange Web Services in Exchange Online](https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-ews-exchange-online).

Thunderbird's Graph implementation was approaching feature parity in June 2026, but Thunderbird 153 still shipped it behind a preference while the project completed the work. For a deployment made today:

- Use native EWS if it works and the tenant is prepared to allow it during the transition.
- Use IMAP/SMTP with OAuth2 for basic mail when EWS is blocked.
- Track Thunderbird's Graph rollout for the long-term Exchange path.
- Expect email first; Exchange calendar and address-book parity is still catching up.

## What we would do first next time

1. **Prove the mailbox independently.** Test OWA and a non-joined device before touching the tenant.
2. **Map the identity contexts.** Record Windows sign-in tenant, device join tenant, mailbox tenant, Office licensing account, and the user context of every shell.
3. **Run WAM diagnostics as the interactive user.** Never trust `dsregcmd /listaccounts` from `SYSTEM` for a desktop-user problem.
4. **Use the error timestamp as a join key.** A generic Outlook page becomes actionable when matched to `Microsoft-Windows-AAD/Operational`.
5. **Separate authentication from licensing.** "Account not supported" in new Outlook can be a licence decision even when OAuth and Exchange are healthy.
6. **Stop when the architecture is fighting the requirement.** A second mail client is cheaper and safer than repeatedly rewriting a joined user's identity stores.

The technical lesson is not that Outlook is bad or Thunderbird is magic. It is that coupling matters. Outlook integrates deeply with Windows and Microsoft 365, which is excellent when the device, user, licence, and mailbox all belong to one identity story. In a two-tenant edge case, the integration itself can become the failure domain.

Thunderbird gave the second mailbox its own OAuth boundary. That was the feature we actually needed.


4844464e14928bc7b275c5c79bd20562e419f1cd106f8afd18ab6a9a5920034c

A practical investigation of cross-tenant Outlook failures on an Entra-joined Windows PC, including WAM diagnostics, misleading fixes, and the Thunderbird alternative.

A Microsoft Entra-joined PC could open a second tenant's mailbox everywhere except Outlook. WAM limits, PRT routing, licensing checks, and a silent cleanup script eventually pointed us to the simpler answer: Thunderbird with OAuth.

Two Microsoft 365 Tenants, One Windows Session: Why We Chose Thunderbird


{/* banner prompt: A vivid isometric infographic scene of a small friendly robot (an AI agent) reaching toward a laptop with a wrench in one hand, while a large shield-shaped antivirus icon looms over it, its glowing eye watching. Three of the robot's tools (a floppy disk, a gear, and a key) are shown mid-air, each caught in a translucent red "quarantined" bubble launched from the shield. Clean light-gray background, sharp isometric perspective, muted blue-and-red palette, technical but approachable, no readable text or logos. */}

We gave an AI agent real hands on a Windows machine: reach it over the network, install itself, register a background service, and get to work unattended. Straightforward, we thought — until the machine had Bitdefender GravityZone on it, and the agent could see the machine perfectly well but couldn't actually *do* anything to it. Not because of a bug in our code. Because an endpoint protection product was quietly erasing our binary and blocking Windows' own service-management APIs, without a single visible error pointing at the real cause.

This is the anatomy of that block, three layers deep, and the exact GravityZone settings that let an automated agent actually operate on a managed endpoint.

## Layer 1: the binary just disappears

The first symptom looked like a download bug. Our install step used a plain `WebClient.DownloadFile` to fetch the agent binary, and it failed with a generic wrapper exception: *"An exception occurred during a WebClient request."* No useful detail.

The real cause was buried in `.Exception.InnerException`, only visible with `Format-List * -Force`:

```
System.UnauthorizedAccessException: Access to the path
'...\agent.exe' is denied.
```

Access denied, on a fresh download, to a brand-new file, in a folder we owned. Ruling out the obvious suspects — TLS, proxy, file locks, ACLs — took a while, because every one of them came back clean. What finally confirmed it: a pure local `Rename-Item` on the same filename, no network involved at all, threw the identical `UnauthorizedAccessException`. That ruled out anything related to how the file arrived (download provenance, Mark-of-the-Web, whatever) and pointed at one thing: **something was reacting to the filename itself, the instant it appeared on disk.**

`Get-CimInstance -Namespace root/SecurityCenter2 -ClassName AntiVirusProduct` gave the answer: Bitdefender Endpoint Security Tools, centrally managed through GravityZone. It was quarantining the binary on write, silently, with no local error dialog and no entry in the Windows Application or System event logs — Bitdefender logs to its own console, not to Windows, so `Get-WinEvent` finds nothing. From the process's point of view it just looks like the filesystem refused to cooperate.

## Layer 2: even SYSTEM can't create a service

Once we worked around the file-quarantine (more on the fix below), the same binary would download and sit on disk just fine. Installing it as a Windows service was the next wall:

```
Error: install service (run as Administrator if not elevated): Access is denied.
```

Our first assumption was a permissions problem in our own service-install code. To rule that out, we dropped straight to the raw OS tool, running as `nt authority\system`:

```
sc.exe create myagent binPath= "C:\Path\To\agent.exe" start= auto
[SC] CreateService FAILED 5:
Access is denied.
```

`SYSTEM` failing `CreateService` with error 5 is not a permissions problem — SYSTEM has unrestricted access to the Service Control Manager by definition. Something was intercepting the call itself. That's a signature of behavioral/EDR monitoring, not access control: GravityZone's **Advanced Threat Control** module watches for exactly this pattern — a process registering itself as a persistent Windows service — and treats it as a textbook malware-persistence technique, regardless of whether the binary passed the earlier file scan.

## Layer 3: the registry write underneath it

`CreateService` isn't a single atomic action from Windows' perspective — it writes a new key under `HKLM\SYSTEM\CurrentControlSet\Services`. That key path falls under GravityZone's **Sensitive Registry Protection**, a separate sub-feature of Advanced Threat Control that specifically guards persistence-related registry locations and can kill the offending process outright. Two independent enforcement layers were converging on the same operation for two different reasons: one watching "is this process trying to persist," the other watching "is this process touching a sensitive registry key." Either one alone is enough to block a legitimate service install.

## Where "just add an exclusion" falls short

The standard advice for AV false positives is an exclusion: GravityZone's **Antimalware → Exclusions** tab lets you add a Folder, File, Process, or Extension exclusion, and pick which scanning **Modules** it applies to (on-access, on-demand, and — separately — **ATC/IDS**, GravityZone's label for Advanced Threat Control and its Intrusion Detection System). We added both a folder and a process exclusion with all modules selected, including ATC/IDS.

It stopped Layer 1. It did **not** stop Layer 2 or Layer 3. Sensitive Registry Protection turned out to be its own gate, evaluated independently of the Exclusions list — an exclusion tells the scanner to skip a file or process, but it doesn't override a behavioral rule that's watching a system resource (the registry key) rather than the process that's touching it.

## The actual fix

Two policy settings, both under **Antimalware → On-Execute → Advanced Threat Control** in the GravityZone policy:

<Table data={
    {
        headers: ['Setting', 'Default', 'Changed to'],
        rows: [
            ['Action for infected applications', 'Block / Kill process', 'Report only'],
            ['Sensitive Registry Protection', 'Block / Kill process', 'Report only'],
        ]
    }
} />

"Report only" keeps GravityZone watching and logging — you still get visibility into what the agent is doing on the endpoint — but stops it from silently killing the process or deleting the file before the operation can complete. We kept the folder and process exclusions from earlier too, as a belt-and-suspenders measure for the plain file-scan layer, but the two "Report only" toggles are what actually unblocked service installation and registry writes.

This is a real trade-off, not a free lunch. Loosening ATC's response mode on a managed endpoint reduces the odds it auto-remediates a genuine compromise on that box. The right scope for this change is narrow: apply it via a dedicated policy assigned only to endpoints that are supposed to run this kind of automation, not your default fleet-wide policy — and keep "Report only" instead of disabling the module outright, so you still see every detection GravityZone would otherwise have acted on.

## What to check if your automation is being silently eaten

If a process on a GravityZone-managed Windows box can create files but can't seem to keep them, or can enumerate/read everything but can't write to the service registry or call `CreateService`, even running as SYSTEM:

1. **Get past the wrapper exception.** .NET's `WebClient`/`HttpClient` errors and generic "Access is denied" messages hide the real exception. Always drill into `.Exception.InnerException` (`Format-List * -Force` in PowerShell) before concluding it's your own code.
2. **Rule out provenance with a pure local operation.** If `Rename-Item` on the same destination filename also fails with no network involved, the block is about the file/process identity, not how it arrived.
3. **Identify the real AV product**, don't assume it's Windows Defender. `Get-CimInstance -Namespace root/SecurityCenter2 -ClassName AntiVirusProduct` tells you what's actually active — often a third-party product has silently superseded Defender, and Defender's own cmdlets (`Get-MpPreference`) will fail with a generic COM error when that's the case.
4. **Test service creation with the raw OS tool**, not your own code. `sc.exe create` failing under SYSTEM isolates the problem to the platform/EDR layer, not your service-install logic.
5. **Don't stop at Exclusions.** A file/process exclusion silences on-access scanning, but a dedicated behavioral rule — like registry-key protection — can still be watching the *resource* rather than the *process*, and needs its own toggle.

The broader lesson: modern EDR products don't have one enforcement point, they have several, stacked on different signals — file content, process behavior, and resource access — and each one can independently veto the same operation for a different reason. Debugging "access denied" on a managed endpoint means finding out which layer is actually talking, not just the first one you can reproduce.

c7a32d8904e2b087f20b4cdaf8ed027394a9e954e6a0c239b6c3116bdc9a7043

How Bitdefender GravityZone silently blocked an AI agent's own binary, service installation, and registry writes on Windows, and the exact policy settings that fixed it.

Our AI ops agent could reach a Windows box over the network but couldn't install itself, start its own service, or even keep its own binary on disk. The culprit: Bitdefender GravityZone silently eating it, one enforcement layer at a time.

When Your Antivirus Fights Your AI Agent: A Bitdefender GravityZone War Story


{/* banner prompt: A vivid infographic-style scene where a stream of glowing data blocks flows through a semi-transparent proxy filter that visibly plucks out a single red block, labelled "0x08", causing a connection icon downstream to display a red error symbol "0x609". Clean light-gray background, sharp isometric perspective, muted blue-and-red palette, technical but clear. */}

RDP error 0x609 — "An authentication error has occurred" — is a CredSSP failure. The standard advice is to check your NLA settings, patch your CredSSP version, or relax the encryption level. None of it helped us, because the error only appeared when connecting *through a proxy*. Direct connections to the same server worked perfectly every time.

That asymmetry is the only clue you need. If direct works and proxied doesn't, the proxy is transforming something it shouldn't be.

## The setup

We were relaying RDP over a WebSocket tunnel — client connects to a local port, a relay forwards the raw TCP stream through a WebSocket connection to a gateway, gateway connects onward to the Windows machine. A standard CONNECT-style proxy, just over WebSocket instead of a raw TCP pipe.

The first phase of an RDP connection (X.224 → CredSSP → MCS → RDP) is negotiated in the clear before any screen data flows. Our relay passes bytes through untouched, so in theory the server should see exactly the same handshake whether or not a proxy is in the middle.

In theory.

## What 0x609 actually tells you

The Windows error 0x609 maps to `STATUS_LOGON_FAILURE` inside the CredSSP layer, but that description is misleading. It doesn't necessarily mean the credentials are wrong. It can also mean the CredSSP negotiation *completed* but the session couldn't be established afterwards — the error surfaces at the CredSSP layer because that's where the TLS channel lives, and whatever went wrong downstream comes back through it.

The useless nature of this error code is part of what makes 0x609 hard to debug. You're essentially told "something authentication-related failed" without any pointer to where in the multi-phase handshake the failure occurred.

## Reading the X.224 handshake

RDP starts with an X.224 Connection Request (CR) from the client. Buried inside the TPKT/X.224 framing is a `requestedProtocols` field — a bitmask that advertises which security layers the client supports:

- `0x01` — SSL/TLS
- `0x02` — NLA (CredSSP over NTLM/Kerberos), also called HYBRID
- `0x08` — HYBRID_EX (an extension to NLA used by modern Windows)

Modern `mstsc` sends `0x0b` — all three bits set. The server responds with an X.224 Connection Confirm (CC) that includes a `selectedProtocol` field indicating which mode it chose.

Here is where we found the divergence.

**Direct connection:** the CC came back with `selectedProtocol = 0x08` (HYBRID_EX).

**Through the proxy:** the CC came back with `selectedProtocol = 0x02` (HYBRID, plain NLA).

Same client. Same server. Different `selectedProtocol`. The proxy was changing something in the CR before it reached the server.

## The hidden transformation

The proxy had a small normalisation function applied to the first TCP frame on each new connection. Its purpose was compatibility: some older Windows environments support NLA but reject HYBRID_EX outright, returning an immediate RST. So the function detected the X.224 CR structure, read the `requestedProtocols` field, and cleared the `0x08` bit before forwarding.

The intent was conservative — strip the extended mode, keep plain NLA, everything still authenticates. It worked on the servers we originally tested it against.

What we hadn't accounted for: some Windows Server versions advertise both HYBRID and HYBRID_EX, but internally require the HYBRID_EX flow to complete a session correctly. When forced into HYBRID mode by the altered CR, they complete CredSSP — all five steps of the NTLM exchange succeed, credentials are verified — and then the MCS Connect-Initial fails. The session can't start.

The mechanism: HYBRID_EX adds a step that HYBRID omits. After the CredSSP exchange, the server sends an Early User Authorization Result PDU before the MCS layer begins. This PDU is part of the HYBRID_EX contract. When the server is put into HYBRID mode (because `0x08` was stripped from the CR), it doesn't send this PDU — but if the server's internal session setup expects it, or if the subsequent MCS framing depends on state that only HYBRID_EX establishes, the session fails. From the client's side it arrives as a 0x609.

In pseudo-code, what was happening:

```
# CR goes out with 0x0b (SSL | NLA | HYBRID_EX)
# proxy strips 0x08 → 0x03 (SSL | NLA)
# server selects HYBRID (0x02) — the highest it can given the offer
# CredSSP runs normally and succeeds
# server tries to start MCS in HYBRID_EX mode internally
# HYBRID_EX Early User Authorization PDU never arrives
# MCS Connect-Initial fails
# error surfaces as 0x609 on the CredSSP channel
```

## The fix

Remove the stripping entirely. Pass `requestedProtocols` through unchanged.

The original compatibility concern — older servers rejecting HYBRID_EX — turned out to be unwarranted in practice. Modern `mstsc` has been sending HYBRID_EX for years and servers handle it gracefully: if they don't support it they simply select a lower mode in the CC. The server's CC is the right place for that negotiation to happen, not a proxy in the middle.

## What to check if you hit this

If you're seeing 0x609 only through a proxy and direct connections work:

1. **Capture both sides of the relay.** You want the raw bytes the server sees, not the bytes the client sent. Log the first frame in each direction.

2. **Find the X.224 CC and read `selectedProtocol`.** It's a 4-byte little-endian value a few bytes into the CC. If it differs between direct and proxied connections, the proxy is rewriting the CR.

3. **Check for any transformation applied to the first message.** Header parsers, encoding normalizers, protocol fixups — anything that touches the first frame of a new TCP connection is a candidate.

4. **Compare `requestedProtocols` in the CR.** If the proxy is changing the bitmask, restore it to pass through verbatim and retest.

The broader principle: a byte-stream proxy should be transparent. Any transformation that modifies the protocol negotiation — even a seemingly harmless bit-clear — can break assumptions downstream that neither the proxy nor its author can fully anticipate.

The server, not the proxy, decides which protocol variant to use. Let it.


6a01956bb020c50d9f63b34a3f6be7e108447b2e7a723fabe945abf36bfa1498

How we debugged RDP error 0x609 appearing only through a WebSocket proxy, and why stripping the HYBRID_EX bit from the protocol negotiation breaks the entire session.

A CredSSP 0x609 error appearing only through a proxy but not on direct connections — traced to a single HYBRID_EX bit being silently stripped from the RDP X.224 handshake.

Debugging RDP 0x609: The Protocol Bit a Proxy Should Never Touch

{/* banner prompt: A playful, polished 3D clay-render in matte sage green, cream and terracotta: a stylized NVIDIA RTX 3060 graphics card as a tiny cinema, a small strip of film curling out of it, and a rounded speaker or sound-wave motif beside it emitting soft concentric rings to show the video now has audio; a little clapperboard prop, soft studio shadows, simple background, 16:9. */}

I ended the [last post](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/)
with a promise: the same RTX 3060 that learned to make short video would next try
to make video *with sound* — using **MiniMax H3**, an open-weights model that
generates its own audio. The weights landed publicly, I put them on the same
card, and this is what actually happened.

The headline first: it works. A single product photo goes in, and a few-second
clip comes out — the model turning, the fabric settling, a soft spoken line and
room tone — **audio the model generated itself**, not dubbed on afterwards. On a
12GB consumer card. That part genuinely surprised me.

## The machine (still the same one)

Same box as the [image retrospective](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/)
and the [video post](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/):
an RTX 3060 with 12GB of VRAM, 64GB of system RAM, Debian. Nothing upgraded. The
card is the constant this whole series is built around.

## What "video with sound" means here

Most open video models are mute — you get moving pictures and add sound in an
editor. MiniMax H3 is different: video and audio come out of the *same* model,
generated together, so the soundtrack matches the scene. Describe waves and a
spoken line, and you get waves and a voice. That single capability is why it was
worth the effort.

## Fitting 54GB into 12GB

Here is the honest part. H3 is big: about **54GB of weights** — a 21GB video
model, a 27GB text encoder, and two decoders for picture and sound. That does
not fit in 12GB of VRAM. Not close.

It runs anyway because the weights live in the 64GB of system RAM and stream onto
the card a slice at a time, keeping just enough on the GPU to do the next step.
It is slow, but it finishes. Two smaller things also got in the way: the model is
new enough that its picture and sound decoders each had a small bug that had to
be patched before a clip would complete, and the card needed a healthy slice of
memory kept free so the compressed weights had room to unpack. Fiddly, but
one-time — once it was set up, it just ran.

## The result: a product clip that turns and talks

This is the workflow a small business would actually use: start from one product
photo, and let the model animate it — the person turns to camera, the knit
settles, a quiet line of audio plays — while the product stays readable. Here is
a real clip generated on this card. **Press play with the sound on** — the audio
came out of the model, not an editor.

<video controls playsInline preload="metadata" poster="https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/sweater-poster.png" style={{ display: "block", width: "100%", maxWidth: "340px", margin: "1.5rem auto", borderRadius: "10px" }}>
  <source src="https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/sweater-clip.mp4" type="video/mp4" />
</video>

And the same clip laid out frame by frame — a natural turn from side-on to
camera, the knit texture and anatomy holding through the whole move:

![Frames from a MiniMax H3 clip: a model in a cream knit sweater turning to camera, generated with sound on an RTX 3060](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-talks-open-source-video-with-sound-on-the-same-rtx-3060/filmstrip-sweater.png)

## The honest numbers

<Table data={
    {
        headers: ['', 'On the RTX 3060'],
        rows: [
            ['Clip length', 'A few seconds (2-5s)'],
            ['Resolution', '512-640px'],
            ['Audio', 'Yes - native stereo, generated with the video'],
            ['Model size', '~54GB of weights, streamed from system RAM'],
            ['Time per clip', 'Roughly 20-27 minutes'],
            ['Best for', 'A short product clip with a voice or ambient soundtrack'],
        ]
    }
} />

Twenty-odd minutes for a few seconds is the real cost of streaming that much
weight through a small card. This is not something you batch out by the hundred
on a 3060 — it is something you run overnight for one good clip. But the clip has
sound, and it cost electricity.

## What I learned about the "AI look"

The most useful lesson wasn't about speed — it was about *why* AI video still
reads as AI, and how to fight it.

Image-to-video can never look better than the photo you start from; it inherits
that frame's every quirk and then adds motion on top. My first attempts started
from bright, high-contrast outdoor stills — and they had that unmistakable
"AI feel": waxy skin, hard outlines, the joins between body parts subtly wrong.
The motion clung to those hard edges and made them worse.

The clip above started from a **soft, studio-lit photo** instead — diffused
light, a plain background, gentle edges. And it held together: real fabric
texture, natural movement, believable anatomy through the whole turn. Same model,
same card, completely different result.

So the practical rule, if you try this yourself: **start from soft, evenly-lit
photos, not harsh high-contrast ones.** The model can't cling to hard edges that
were never there. It's the single biggest lever on how "real" the output looks —
bigger than any setting.

## Where this leaves a small business

Three posts, one unchanged computer. The same RTX 3060 went from struggling with
a single 512px [still](https://www.tyolab.com/prompts), to short
[silent video](/blog/2026/08/03-now-it-moves-video-generation-on-the-same-rtx-3060/),
to a short clip **with its own soundtrack** — a product turning to camera and
speaking a line, generated from one photo, locally, for the cost of the power it
drew.

It is not one-click, and it is not fast. But the gap between "I have a product
photo" and "I have a short product video with sound" has quietly closed on
hardware a small business can already afford. That's the whole point of this
series: the tools that used to need a studio now need a modest computer and a bit
of patience.

*The card hasn't changed in two years. What it can do has.*


0e5f273a081f550b8cae1cc64cfacb00dc1a76e174947517e86cf2fa03d0d6a2

Running MiniMax H3, an open-source video model with native audio, on a consumer RTX 3060 12GB: how 54GB of weights fit in 12GB, honest generation times, the quality limits, and turning one product photo into a clip with sound.

The third in our RTX 3060 series. MiniMax H3, an open-weights video model that generates its own audio, runs on the same 12GB card: a product clip with a real soundtrack, from a single photo, generated locally.

Now It Talks: Open-Source Video With Sound on the Same RTX 3060

{/* banner prompt: A playful but polished 3D clay-render, matte materials in sage green, cream and terracotta, of a stylized NVIDIA RTX 3060 graphics card acting as a small film projector on a simple background, casting a beam of light onto a tiny cinema screen that shows a strip of film frames; a couple of small clapperboard and film-reel props beside it, soft studio shadows, 16:9. */}

In the [last post](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/)
I traced how open-source *image* generation went from a 512px struggle to
photoreal Flux on one RTX 3060. I ended it with a tease about video. Here's the
payoff — and it's the part that genuinely surprised me.

Two years ago, generating video on a consumer graphics card wasn't really a
thing; the early attempts were a few flickery, warping frames. The **same RTX
3060** now produces short, coherent clips — and, more usefully for a small
business, it can take a **single product photo and turn it into a promo**.

## The machine (still the same one)

Same box as the [image post](/blog/2026/08/03-two-years-of-open-source-image-ai-on-one-rtx-3060/)
and the [LLM benchmark](/blog/2026/05/11-64gb-ram-12gb-vram-the-honest-local-llm-benchmark/):
an RTX 3060 with 12GB of VRAM, 64GB of system RAM, Debian 13. Nothing upgraded.

## What runs today

Two open models do the heavy lifting, both locally:

- **Wan 2.2** — higher quality, does text-to-video *and* image-to-video. Slower.
- **LTX-Video** — faster and lighter, good for quick drafts.

The 12GB card can't touch the giant cloud video models, but with the weights
quantised and the text encoder offloaded to system RAM, these run — a few
seconds of 480–832px video per clip.

## The trick that matters: one photo becomes a promo

This is the workflow a small business will actually use. Generate a product
photo (or use one you already have), then animate *it* — the model, the fabric,
the light all start moving, while the product stays put. Each strip below is
frames from a real clip, and each one started as a single still generated on
this same card.

A knit sweater: the model turns to camera, hair catching a breeze.

![Frames from the sweater promo clip](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-moves-video-generation-on-the-same-rtx-3060/filmstrip-sweater.png)

Sunscreen — very on-brand for an Australian summer — the product held steady in
focus while the scene breathes:

![Frames from the sunscreen promo clip](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-moves-video-generation-on-the-same-rtx-3060/filmstrip-sunscreen.png)

And eyewear, a slow confident turn in golden-hour light:

![Frames from the sunglasses promo clip](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/08/now-it-moves-video-generation-on-the-same-rtx-3060/filmstrip-shades.png)

## The honest numbers

<Table data={
    {
        headers: ['', 'On the RTX 3060'],
        rows: [
            ['Clip length', 'A few seconds (2-5s)'],
            ['Resolution', '480p to ~832px'],
            ['Time per clip (Wan 2.2)', 'Roughly 4-8 minutes'],
            ['Time per clip (LTX)', 'Under a minute for a short draft'],
            ['Best for', 'Image-to-video product promos, social clips'],
            ['Audio', 'Not yet — these models are silent'],
        ]
    }
} />

It's not one-click cinema. Big, fast motion still warps; the sweet spot is
*subtle* movement — a turn of the head, a breeze, a slow push — with the product
kept steady. You generate a couple of takes and keep the best, same as with
stills. But a usable, on-brand product clip in a few minutes, for the cost of
the electricity, on a card that couldn't manage one clean still two years ago —
that's the shift.

## What's next: sound

The current open video models are mute. The next one on our bench, **MiniMax
H3**, ships open weights *with* native audio and is reported to run on exactly
this class of hardware. When its weights land publicly, we'll put it on the same
3060 and see whether a small business can generate a product clip **with a
soundtrack** end to end — and we'll write up what actually happens.

For now, the takeaway from two posts and one unchanged GPU: the same modest
computer that makes studio-grade [stills](https://www.tyolab.com/prompts) now
makes short video too, and the gap between "I have a product photo" and "I have a
product video" has quietly closed.

*Next up: MiniMax H3 — open-weights video with sound — on the very same card.*


492ad0a4331b0c870b48208cef2d272462372bd58c519f5bd6ce2c257a5eeccb

How open-source video generation runs on a consumer RTX 3060 12GB in 2026: Wan 2.2 and LTX-Video for text-to-video and image-to-video, real generation times, VRAM limits, and a worked example turning a product photo into a promo clip.

The follow-up to our image retrospective: the same RTX 3060 that struggled with a single 512px still two years ago now generates short video — including product promos animated from a single photo — with open models like Wan 2.2 and LTX-Video.

Now It Moves: Video Generation on the Same RTX 3060

{/* banner prompt: A playful but polished 3D render of a stylized RTX 3060 as a small factory on a simple background, using matte clay-like materials in a color story of sage green, cream, and terracotta; a conveyor belt on the left carries misshapen, glitchy figures, while a belt on the right carries perfectly formed photorealistic objects. */}

Eighteen months ago I wrote a tutorial on
[mastering human portraits with Stable Diffusion](/blog/2025/02-06-mastering-human-portraits-with-stable-diffusion-ai/).
Reading it back is a little painful. It opens with "trust me, this is hard," and
its list of search keywords reads like a bug report: *two heads, extra heads,
deformed face, extra eyes, disfigured, duplicate.* Those weren't edge cases —
they were what you fought on every render.

This week I ran open-source image generation on the **exact same computer**. Same
RTX 3060, same 12GB of VRAM. The only thing that changed is the software, and the
gap is hard to overstate.

## The machine (deliberately unchanged)

It's the same box from our [local-LLM
benchmark](/blog/2026/05/11-64gb-ram-12gb-vram-the-honest-local-llm-benchmark/):

- **CPU:** Intel Core i7-12700F
- **RAM:** 64GB DDR4
- **GPU:** NVIDIA GeForce RTX 3060 — 12GB VRAM
- **OS:** Debian 13

Holding the hardware constant is the whole point. Any improvement you see is the
open-source ecosystem getting better, not a new graphics card.

## Early 2025: Stable Diffusion 1.5, 512 pixels, and a war on anatomy

Back then, the state of the art you could actually run at home was Stable
Diffusion 1.5. My old parameters tell the story: 512×512, Euler a, CFG 7. To get
anything lifelike you leaned on community finetunes (RealisticVision, ChilloutMix,
F222), stacked LoRAs, and wrote paragraph-long negative prompts to *suppress* the
model's habit of adding a second head or a sixth finger. Then you generated a
dozen and kept the one that wasn't broken. It worked, but it was a craft — and a
grind.

## Mid 2026: the same card, a different league

Here's a portrait from the same 3060 today, straight out of the model —
no inpainting, no ControlNet, no clean-up:

![A photoreal portrait generated on the RTX 3060 in 2026](https://www.tyolab.com/api/tynion/images/f9f8fb60-d635-1048-cb10-c3492d7a4c4a/file)

Three things changed the game. **Flux** and modern **SDXL** finetunes replaced
SD 1.5 as the open baseline. They render natively at 1024px and up, and they get
hands, faces and proportions right the first time far more often than not.

They also learned to spell — the single clearest "we crossed a line" moment.
Readable text in an image was essentially impossible in 2025:

![A shopfront with legible hand-painted signage, generated locally](https://www.tyolab.com/api/tynion/images/3fa3d0ac-f3af-f6e7-952e-27b04f9b545e/file)

And the range on one card is wide — photoreal wildlife and clean anime styles
from the same machine, same afternoon:

![A photorealistic wildlife shot from the same GPU](https://www.tyolab.com/api/tynion/images/adbceadb-bc09-46f0-c8c7-bd76f312361f/file)

![A detailed anime illustration from the same GPU](https://www.tyolab.com/api/tynion/images/3d737b80-9cbb-186f-ce2c-c496fb918589/file)

## Then and now, side by side

<Table data={
    {
        headers: ['', 'Early 2025', 'Mid 2026'],
        rows: [
            ['Best model you can run', 'Stable Diffusion 1.5 + finetunes', 'Flux, SDXL finetunes'],
            ['Native resolution', '512 x 512', '1024px and up'],
            ['Anatomy', 'Fighting extra heads, hands, eyes', 'Right on the first try, mostly'],
            ['Text in the image', 'Gibberish', 'Often readable'],
            ['Keepers per ten tries', 'One or two', 'Most of them'],
            ['Effort per shot', 'Long negative prompts, LoRAs, inpainting', 'A plain-English sentence'],
            ['Video', 'Not on this card', 'A few seconds, locally'],
        ]
    }
} />

## The part that was pure science fiction in 2025: video

The same RTX 3060 now generates **video**. Wan 2.2 and LTX-Video produce short
text-to-video and image-to-video clips locally — a five-second scene in a few
minutes, on a card that two years ago struggled with a single 512px still. The
next open model on our list, MiniMax H3, even adds a synchronised soundtrack.

## What actually changed — and why it matters

It wasn't the GPU. Three shifts in the open-source world did the work:

- **Better base models.** Flux and SDXL are simply generations ahead of SD 1.5.
- **Quantisation.** Techniques like fp8 and 4-bit weights shrink 12-billion-parameter
  models enough to fit inside 12GB of VRAM with quality mostly intact.
- **Smarter memory management.** The tools now offload the heavy text encoders to
  system RAM, so the GPU only holds what it must.

Put together, a consumer card that launched years ago runs models that didn't
exist when you bought it — because the community learned to make them fit.

For a small business or a hobbyist, that's the real headline: the *same* modest
hardware budget now buys studio-grade stills and short video, from software
that's free and runs on your own desk. Every image in this post was generated on
that one 3060; you can browse and remix the prompts in our
[gallery](https://www.tyolab.com/prompts).

*Next up: putting MiniMax H3 — open-weights video with native audio — on the very
same card, and seeing how far 12GB can really go.*


3b1c2bde5a7d34a1dd6cf7101d23b93c2774ca890ffc4eec11944f83608ebd17

A hands-on before-and-after: how open-source text-to-image models improved on the exact same RTX 3060 12GB between early 2025 and 2026 — Stable Diffusion 1.5 vs Flux and SDXL finetunes, resolution, anatomy, text rendering, hit-rate, and video.

We ran open-source image generation on the same RTX 3060 we used 18 months ago. The hardware is identical; the results are unrecognisable — from 512px portraits fighting extra heads to 1024px Flux, near-perfect anatomy, readable text, and even video.

Two Years of Open-Source Image AI on One RTX 3060


{/* banner prompt: An editorial-illustration banner, 16:9: a large softly glowing translucent brain — a powerful AI — held steady and measured by clean, tangible engineering instruments: a spirit level resting across the top with a centered bubble, a ruler alongside, and a pair of calipers. Bright, optimistic, minimal composition on an off-white background with soft shadows; a muted accent palette of sage green, terracotta, and navy. Precise, calm, trustworthy mood. No lettering, no text. */}

Every few weeks, a more capable model lands. Fable 5, GPT-5.6 Sol, Kimi K3 — each genuinely sharper than the last, each able to hold more, reason further, write cleaner code. It is a remarkable thing to watch. It is also, on its own, not enough.

Power and reliability are different axes. A more powerful model still runs out of context on a long enough document. It still loses the thread somewhere in the middle of a big conversation — the "lost in the middle" problem does not vanish just because the window grew. In a long agentic session it still quietly forgets a decision it made an hour ago. And most awkward of all: it states the wrong thing with exactly the same fluent confidence as the right thing. The tone never wavers. The certainty is not calibrated to the correctness.

That last point is the one that matters for anyone putting AI to work. **A model's internal sense of "done" is not a measurement. It is a vibe.** And you cannot manage — or ship, or trust — what you cannot measure.

## Engineering already solved this once

Physical engineering did not become reliable because craftsmen got more talented. It became reliable because it got *instruments*. A ruler. A set of scales. A spirit level. A multimeter. Objective, external, repeatable tools that return an answer independent of how the person holding them feels.

You do not ask the carpenter whether the shelf is level. You put a spirit level on it and look at the bubble. The bubble does not care how many shelves the carpenter has hung. That is the whole point: the measurement lives *outside* the person, so it is immune to the person's fatigue, memory, and overconfidence.

AI needs the same thing — external, quantitative, tangible checks that return a hard answer the model cannot talk its way around. Something the AI can **run**, not something it has to **believe**.

A good measurement for AI has a few properties:

- **Deterministic** — same input, same answer, every time.
- **Numeric or binary** — a pass or fail, a count, a fingerprint. Not a paragraph of reassurance.
- **External to the model** — it lives outside the context window, so it cannot be forgotten or drifted away from.
- **Cheap** — cheap enough to run on every single change, not only when someone remembers.
- **Machine-readable** — so the AI itself can call it, read the result, and act on it.

## A worked example: our deployment guard

We build software with AI in the loop every day, and the single scariest moment is deployment. Did that change actually land correctly? An AI — or a human — can *say* "deployed successfully." That sentence is a vibe. So we gave the deploy a ruler instead.

It is a small tool we call `deploy-checks`. Before and after a release it verifies the live system against a written contract and returns a number: `9/9 passed, exit 0` — or a red line and `exit 1`. No prose. No confidence. No memory. Just the reading.

Here is the tangible part — what it actually measures:

<Table data={
    {
        headers: ['Check', 'What it returns', 'What it catches'],
        rows: [
            ['Token fingerprint match', 'sha8:9c56571d == sha8:9c56571d', 'A shared secret drifting or vanishing across services'],
            ['Permission grant', '1 grant row, or 0', 'A route that returns 403 to everyone before the controller runs'],
            ['Endpoint contract', 'HTTP 400 (want != 401)', 'A fail-closed guard silently rejecting valid callers'],
            ['Health and version', 'exit 0, or exit 1', 'A crashed, stale, or half-promoted release']
        ]
    }
} />

None of these ask the model to *recall* anything. The token check does not trust that the secret is right; it computes a fingerprint of each copy and compares them — match, or no match. The grant check does not assume the permission was set; it counts the row in the database — one, or zero. The endpoint check does not believe the guard works; it sends a real request and reads the status code.

And these are not hypothetical. Every check on that list exists because it caught a real, live production failure. A shared secret silently dropped out of an environment file, and every sign-in across three products started failing — a fingerprint comparison catches that in a single run, where no amount of model cleverness would "remember" it had gone wrong. A permission grant went missing, and an admin page returned "forbidden" to everyone — a row count catches that instantly. These are exactly the kind of quiet, confident-looking failures that slip past a tired engineer and an eloquent model alike.

The crucial move is this: **the AI does not decide the deploy is healthy. It runs the ruler and reads the number.** "I think it is fine" becomes "9 of 9, exit 0." And when the number is bad, the same tool refuses to promote the release and rolls it back automatically. The measurement does not merely inform the decision — it *gates* the action.

## Why stronger models need this *more*, not less

It is tempting to assume better models need fewer guardrails. The opposite is true. The more capable the model, the bigger and less-supervised the task we hand it. You babysit a weak model line by line; you hand a strong agent an entire deployment. The blast radius of an *unmeasured* mistake grows with the model's autonomy. The measurement is the seatbelt that lets you drive faster — remove it and you do not go faster, you just crash harder.

There is a second, quieter payoff. A good ruler is **model-agnostic**. A fingerprint match is true whether Fable 5, GPT-5.6 Sol, or next year's model produced the change. Your investment in measurement compounds; your investment in any single model depreciates the day the next one ships. Build the instruments once, and every future model gets safer for free.

## The takeaway

Stop waiting for the model that never forgets. It is not coming — and even if it did, you would still want to *verify* rather than *trust*, because that is simply what responsible engineering is.

Build the rulers instead. Wrap your AI in tangible, quantitative checks: tests, type-checks, fingerprints, health probes, contract assertions, row counts. Give the model something real to measure against, and let the number — not the vibe — decide what ships.

That is the philosophy behind every guard we build, for our own stack and, increasingly, for the businesses that want AI in their pipeline without handing it the keys unmeasured. Powerful is wonderful. But measurable is what lets you sleep.

80bd37051d17be94043e76dce40d57062b6ff328c651265724911b452c51a8cd

Why powerful LLMs still need external, quantitative checks, and how a deterministic deployment guard gives AI a tangible ground truth to verify its own work.

Frontier models keep getting more powerful, and they still forget, drift, and run past their context. The fix is not a bigger model; it is giving AI a tangible, quantitative ground truth to measure against. Here is how our deployment guard does exactly that.

Measurable Beats Powerful: Grounding AI in Tangible Checks

{/* banner prompt: Clean isometric diorama of a miniature factory on a white background, where a stylized Go gopher guides a block representing a Go binary through a series of gates labeled 'Linker', 'Alignment', and 'Package' into a sleek Android phone model, minimalist, crisp shadows, material design color palette, 16:9. */}

We're building an Android VPN app that needs a Go binary — a packet-forwarding proxy that runs as a subprocess, owns the TUN file descriptor, and handles all the network I/O. Simple enough in theory: cross-compile Go for arm64, pack it into the APK, exec it at runtime.

In practice we hit three separate problems, each invisible until it wasn't. This post documents all three: what the failure looked like, why it happens, and exactly what to change.

## Trap 1: `GOOS=linux` produces a binary Android can't run

The first sign of trouble is a logcat line that reads something like:

```
java.io.IOException: Cannot run program "/path/to/binary": error=2, No such file or directory
```

The binary is *there*. The path is correct. The file exists. Android still says it doesn't.

The reason is the ELF interpreter line. Every dynamically-linked ELF binary on Linux carries a `PT_INTERP` segment that names the dynamic linker the OS should use to start the program. On a Linux server or desktop, that path is typically:

```
/lib/ld-linux-aarch64.so.1
```

That path doesn't exist on Android. Android ships its own linker at:

```
/system/bin/linker64
```

When the OS tries to exec your binary, it reads the interpreter path from the ELF header. If that path doesn't resolve, execve returns `ENOENT` — "no such file or directory" — even though the binary itself is sitting right there. The error is about the *linker*, not the binary.

**The fix:** use `GOOS=android`, not `GOOS=linux`, when cross-compiling for arm64.

```sh
CGO_ENABLED=0 GOOS=android GOARCH=arm64 \
  go build -trimpath -buildmode=pie \
  -ldflags="-s -w" \
  -o libreachproxy.so .
```

With `GOOS=android`, Go produces an ELF with `/system/bin/linker64` in the interpreter slot. Android finds it, loads it, and your binary starts.

You can verify before deploying:

```sh
readelf -l your_binary | grep interpreter
# Good:  [Requesting program interpreter: /system/bin/linker64]
# Bad:   [Requesting program interpreter: /lib/ld-linux-aarch64.so.1]
```

### What about x86_64?

`GOOS=android GOARCH=amd64` doesn't work without CGO external linking — which requires the Android NDK. If you don't have the NDK in your build environment, you're stuck. For x86_64 (emulators only; no real Android phone ships x86_64), `GOOS=linux` is acceptable — the emulator kernel is lenient enough. Just don't use it for arm64 on real devices.

## Trap 2: Android 16 requires 16 KB page alignment

Android 16 — shipping on Samsung Galaxy S25 and other devices with newer Arm chips — increased the memory page size from 4 KB to 16 KB. If your native binary isn't aligned to 16 KB page boundaries, Android 16 won't load it.

Google Play enforces this at upload time too. You'll see a warning (soon an error) in the Play Console if any `.so` in your APK or AAB fails the alignment check.

**For arm64:** Go's PIE binaries built with `GOOS=android GOARCH=arm64` default to 64 KB alignment, which already satisfies the 16 KB requirement. No extra flag needed.

**For x86_64:** Go's default alignment is 4 KB. You need to force 64 KB (which also satisfies 16 KB) via the linker flag `-R 0x10000`:

```sh
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
  go build -trimpath -buildmode=pie \
  -ldflags="-s -w -R 0x10000" \
  -o libreachproxy_x86_64.so .
```

The `-R 0x10000` flag sets the ELF segment alignment to 65536 bytes (64 KB). Play Store's checker accepts anything ≥ 16 KB.

### Why `-buildmode=pie`?

Android requires all natively-executed code to be position-independent. Without `-buildmode=pie`, Go produces an `ET_EXEC` (fixed-address) binary, which Android won't exec. With `-buildmode=pie`, you get an `ET_DYN` (position-independent) executable that Android's linker is happy to load anywhere in the address space.

## Trap 3: Play Store AAB delivery may not extract your `.so` to disk

Here's the sneaky one.

The standard approach for shipping a native binary in an APK is to put it in `jniLibs/<abi>/`, rename it with a `.so` extension (so the build system includes it), and read its path from `applicationInfo.nativeLibraryDir` at runtime.

```kotlin
val binary = "${applicationInfo.nativeLibraryDir}/libreachproxy.so"
ProcessBuilder(binary, ...).start()
```

This works fine when you sideload the APK directly. On older Android versions it also works for Play Store installs — the system extracts `.so` files to `nativeLibraryDir` during installation.

On Android 16 with Play Store's AAB (App Bundle) delivery, the system may skip that extraction entirely. The `.so` is never written to `nativeLibraryDir`. Your `File(binary).exists()` check passes the sanity check but the file is empty, or the path simply doesn't exist at all. The exec fails silently.

**First fix: `useLegacyPackaging = true`** in your `build.gradle.kts`. This tells the build system to store the `.so` uncompressed in the APK, so Android can extract it on install:

```kotlin
packaging {
    jniLibs {
        useLegacyPackaging = true
    }
}
```

This restores the old extraction behaviour for direct APK installs.

**Second fix: `memfd_create` fallback** for Play Store AAB installs where extraction is still skipped. Bundle the arm64 binary in `assets/` and load it into an anonymous memory file using `Os.memfd_create`. Execute it via `/proc/self/fd/<n>`:

```kotlin
private fun ensureBinary(): String {
    val nativePath = "${applicationInfo.nativeLibraryDir}/libreachproxy.so"
    if (File(nativePath).exists()) return nativePath

    // Play Store AAB on Android 16: nativeLibraryDir wasn't populated.
    // Load from assets into a memfd and exec via /proc/self/fd/<n>.
    val bytes = assets.open("libreachproxy").use { it.readBytes() }
    val fd = Os.memfd_create("libreachproxy", 0)
    var offset = 0
    while (offset < bytes.size) {
        offset += Os.write(fd, bytes, offset, bytes.size - offset)
    }
    val fdInt = FileDescriptor::class.java
        .getDeclaredField("descriptor")
        .apply { isAccessible = true }
        .getInt(fd)
    return "/proc/${Os.getpid()}/fd/$fdInt"
}
```

`memfd_create` creates an anonymous file in memory — no filesystem, no SELinux exec restriction. The `/proc/self/fd/<n>` path is a valid executable path that the kernel accepts for `execve`. The `fd` reference is kept alive for the lifetime of the service so the kernel doesn't reclaim the memory.

`Os.memfd_create` requires API level 30 (Android 11). If you still support API 26–29, you'll need a fallback for that too (writing to `codeCacheDir` and calling `chmod 700`, accepting that Samsung SELinux may still block it on some devices).

## Putting it together

A minimal build script that avoids all three traps:

```sh
#!/usr/bin/env bash
set -euo pipefail

OUT="app/src/main/jniLibs"
mkdir -p "$OUT/arm64-v8a" "$OUT/x86_64"

# arm64: GOOS=android for the correct ELF interpreter.
# 64KB alignment is the default — satisfies Android 16's 16KB requirement.
CGO_ENABLED=0 GOOS=android GOARCH=arm64 \
  go build -trimpath -buildmode=pie -ldflags="-s -w" \
  -o "$OUT/arm64-v8a/libreachproxy.so" .

# x86_64: GOOS=linux (android/amd64 requires CGO external linking without NDK).
# -R 0x10000 forces 64KB alignment to satisfy Play Store's 16KB check.
# x86_64 is emulator-only — no real device uses it.
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 \
  go build -trimpath -buildmode=pie -ldflags="-s -w -R 0x10000" \
  -o "$OUT/x86_64/libreachproxy.so" .

# Also copy arm64 binary to assets/ for the memfd fallback path.
cp "$OUT/arm64-v8a/libreachproxy.so" app/src/main/assets/libreachproxy
```

And in `build.gradle.kts`:

```kotlin
packaging {
    jniLibs {
        useLegacyPackaging = true
    }
}
```

## Quick reference

<Table data={
    {
        headers: ['Problem', 'Symptom', 'Fix'],
        rows: [
            ['Wrong GOOS for arm64', 'error=2 ENOENT on exec even though the file exists', 'GOOS=android GOARCH=arm64'],
            ['4 KB page alignment', 'Play Store upload warning; crash on Android 16', '-buildmode=pie (arm64 default is fine); -R 0x10000 for x86_64'],
            ['AAB delivery skips extraction', 'nativeLibraryDir is empty after Play Store install', 'useLegacyPackaging=true + memfd_create fallback'],
        ]
    }
} />

None of these are obvious until you've hit them. The `GOOS=linux` trap is particularly nasty because the binary runs perfectly in your CI and on the emulator — it only fails on a real device with a strict kernel, which is usually the last place you test.


5f6ebea443bb89c3641ee31d52cb3ce97211e0c406c6070d6e92207477385e66

What we learned cross-compiling Go for Android: the ELF linker trap, 16KB page alignment for Android 16, and the memfd fallback for Play Store AAB delivery.

Building an Android VPN app that bundles a Go binary exposes you to three traps in quick succession: compiling for Linux instead of Android, Android 16's 16 KB page-alignment requirement, and Play Store delivery no longer extracting native libraries to disk. Here's what each one looks like and how to fix it.

Shipping a native Go binary inside an Android APK: ELF linkers, 16 KB pages, and the memfd trick


{/* banner prompt: Bright editorial vector illustration of a relaxed traveller with a small suitcase sitting in a sunny cafe with a laptop; a simple glowing toggle switch on the laptop sends a luminous bridge of light that arcs past a stylized greyed-out "blocked" webpage panel and reconnects to a cluster of familiar everyday app icons (mail, map pin, chat, play button). Clean flat vector shapes, warm daylight palette of yellow, sky blue and white, friendly and optimistic, plenty of open space, 16:9. */}

You land after a long flight, connect to the hotel Wi-Fi, and reach for the small
things that make a trip run: your email, a map to the hotel, a message home to say
you arrived, the news, your bank. And one by one, they just… don't load. The Wi-Fi
says it is connected. The pages say otherwise.

If you have travelled to certain parts of the world, you know the feeling. In some
destinations, a surprising share of the sites and apps you use every day at home
simply stop working the moment you cross a border. It is not your phone, and it is
not the hotel — it is where you are standing.

**TYO Reach** is built for exactly that moment.

## What TYO Reach is

TYO Reach is a **lightweight traffic proxy for the one browser you choose**. You flip
a switch in the Reach app, and from then on the traffic from that single browser
travels out through one of our gateways and reaches websites as if you were somewhere
the open web is simply a given. The pages that would not load a moment ago open like
you never left home.

Everything else on your device is left completely alone — other browsers, your other
apps, background updates — all of it keeps going straight out over your normal
connection. Switch Reach off and that browser instantly goes back to behaving
normally. Nothing sits running in the background when you are not using it.

![The TYO Reach Android app — one "Reach Proxy" switch, your gateway and data allowance, and quick links to your dashboard.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/07/introducing-tyo-reach-a-travel-companion-for-your-browser/android_app.png)

That is really the whole idea: **one switch**. Pick a nearby gateway — Singapore, say —
so things feel close and fast, and get on with your day.

## One switch, on every device

On your laptop, Reach lives quietly in the system tray. Click it to turn routing on
or off; the app configures your chosen browser for you, so there is nothing fiddly to
set up each time.

![On the desktop, Reach sits in your system tray — one click to switch it on or off.](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/07/introducing-tyo-reach-a-travel-companion-for-your-browser/systray_on.png)

On your phone, it is the same single switch in the Android app. Sign in once, choose
your browser, and you are ready.

## Why it makes a good travel companion

- **It travels light.** Because Reach only routes the one browser you pick, and only
  while it is switched on, it never fights with hotel Wi-Fi, slows your other apps, or
  quietly drains your battery in the background. Need it off? One tap.
- **It reaches your everyday things.** Email, maps, messaging, the news, the streaming
  library you already pay for, your bank's website — the ordinary internet you rely on
  at home, back within reach.
- **It is ready when you are.** Choose a gateway close to where you are travelling and
  the web feels local, not like it is coming from the other side of the planet.

<Table data={
    {
        headers: ['', 'TYO Reach', 'Full VPN'],
        rows: [
            ['Scope', 'One browser you choose', 'Your entire device'],
            ['Other apps affected', 'No', 'Yes'],
            ['Runs in the background', 'Only while switched on', 'Usually always on'],
            ['Setup', 'The app configures your browser', 'System-level install'],
            ['Best for', 'Reaching blocked or region-locked sites', 'Whole-device privacy'],
        ]
    }
} />

Reach is not a full, whole-device VPN, and it is not trying to be. A VPN reroutes
*everything* on your computer, all the time — powerful, but heavy. Reach gives you the
part most travellers actually want: appearing somewhere the sites expect, so blocked
and region-locked pages open again, without taking over the rest of your machine. It
is built for *reach*, not for cloaking everything you do.

## Data and plans

Reach comes with a monthly data allowance, and you can watch exactly how much you have
used right in the app. There is a free tier to try it out, and if you find yourself
leaning on it every day, upgrading takes a moment. If you run low mid-trip, topping up
is quick — no reinstalling, no reconfiguring.

## Getting started (do this before you fly)

Setting up takes about two minutes, and there is one piece of travel advice worth
stating plainly: **do it before you leave home.** In the very places Reach is most
useful, creating an account or downloading an app can be harder once you have arrived.
A few minutes of setup on your own couch saves a lot of frustration later.

1. Create an account at [reach.tyo.com.au](https://reach.tyo.com.au).
2. Download the client for your laptop (Windows, macOS or Linux) or the Android app.
3. Pick the browser you want Reach to route, and flip the switch.

That is it. The full walkthrough lives in the
[getting started guide](https://reach.tyo.com.au/docs/getting-started), and if you are
curious how Reach compares to a VPN, we wrote that up in
[Reach vs a VPN](https://reach.tyo.com.au/docs/reach-vs-vpn).

## In short

The best travel companions are the ones you forget you are carrying until the moment
you need them. Pack TYO Reach before your next trip, and when you land somewhere the
internet does not quite work the way you expect, your everyday web is one switch away.

Have a question or a feature request? We would love to hear it — reach us at
[dev@tyo.com.au](mailto:dev@tyo.com.au), or find the developer on X:
[Eric Tang](https://x.com/_e_tang).


71960c4d7c589857fc405f97c4dea337d1591861639d8f57f3f449546539645c

Meet TYO Reach: a lightweight browser proxy that helps travellers reach the everyday websites and services that stop working in certain destinations.

TYO Reach is a lightweight traffic proxy for a single browser — a handy travel companion for reaching the everyday sites and apps you rely on at home when they quietly stop working abroad.

Introducing TYO Reach: A Travel Companion for Your Browser


{/* banner prompt: A detailed, sunlit macro photograph of a miniature diorama on a wooden workbench: a tiny, stylized browser window is connected by a polished copper pipe to a small, intricate brass lockbox, symbolizing a private, local download, warm analog tones, shallow depth of field. --ar 16:9 */}

If you use [TYO Reach](https://reach.tyo.com.au), you already watch videos through your own
gateway, with your own logins. Sometimes you need the *file* — your own uploads,
a Creative Commons lecture, public-domain footage, a clip you have the rights to
keep. The usual answer is a "video downloader" website or extension, and the
usual price is your privacy: cloud converters see every URL you feed them, and
plenty of downloader extensions phone home with more than that.

**reach-dl** takes the other road. It is a small open-source extension that puts
a download manager in your browser's side panel — and keeps every byte of your
data on your machine.

## Private by architecture, not by promise

Most privacy claims are promises. reach-dl's is a design constraint: **there is
nothing to send your data to.**

When you queue a download, the extension collects the page URL and that site's
cookies, opens a local pipe to a tiny helper program on your own computer
(Chrome's native messaging — no network involved), and the helper runs
[yt-dlp](https://github.com/yt-dlp/yt-dlp) through your Reach proxy. The helper
exits when the download ends. There is no server, no message broker, no daemon,
no account, no telemetry. Your cookies go browser → local pipe → yt-dlp and are
written only to a temporary file that is deleted the moment the run finishes.
The download queue itself never touches disk and never contains cookies at all —
they are read fresh from the browser at the moment each download starts.

Because the download rides your Reach gateway with your browser's own
User-Agent and cookies, sites see the same session that just played the video —
which also keeps Cloudflare-protected sites happy where a bare command line
often fails.

## A real download manager, not just a button

Click the toolbar button on a video page (or right-click → **"Save with
reach-dl"**) and the side panel opens with the page queued:

![The reach-dl side panel with a download queue](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/07/introducing-reach-dl-a-local-private-download-manager-for-tyo-reach/panel.png)
*The side panel: queue, progress, and per-item controls.*

- **Queue as many as you like** — downloads run one at a time (the Reach
  gateway is tuned for browsing, not bulk transfer) and process in order.
- **Pause and resume.** Pausing stops the download but keeps the partial
  `.part` file; resuming picks it up exactly where it left off. Even closing
  the browser mid-download loses nothing — re-add the URL and it continues
  from the `.part` file.
- **Honest errors.** When a download fails you see yt-dlp's *actual* error
  line and a link to the full local log — not a shrug. Most failures are just
  a stale yt-dlp (`yt-dlp -U`), and the panel says so.
- **Choose where files land.** A "Save to" field in the panel footer accepts
  any folder (`~/` works); leave it empty for `~/Downloads`. A typo'd folder
  fails loudly instead of saving somewhere surprising.
- **Through Reach, or direct — your call.** A "Download through TYO Reach"
  toggle (on by default) decides which network the download rides. Turn it
  off for videos that aren't geo-blocked: direct downloads skip the proxy,
  don't count against your Reach data allowance, and are usually faster. The
  choice is always explicit — reach-dl never silently switches your traffic
  from one network to the other.

![The panel footer: Reach toggle and Save-to field](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/07/introducing-reach-dl-a-local-private-download-manager-for-tyo-reach/saveto.png)
*The footer settings — the Reach toggle (shown off: direct mode) and the Save-to field. The folder must already exist; reach-dl never creates or deletes anything on its own.*

Under the hood, "pause" is pleasingly boring: the extension simply disconnects
the local pipe; the helper notices and interrupts yt-dlp the same way Ctrl-C
would, which is exactly why the `.part` file survives. No state to sync, no
protocol to keep alive.

## Setup in three steps

reach-dl deliberately does **not** bundle yt-dlp — you install and update your
own copy, and the extension drives it.

```sh
# 1. install yt-dlp on your PATH (and keep it current: yt-dlp -U)
#    https://github.com/yt-dlp/yt-dlp#installation

# 2. load the extension: chrome://extensions → Developer mode →
#    "Load unpacked" → the ext/ folder from the repo; copy the extension ID

# 3. register the local helper (Go required):
git clone https://github.com/e-tang/reach-dl
cd reach-dl && ./install/install-host.sh <EXTENSION_ID>
```

Restart the browser, switch Reach on, and the toolbar button does the rest.
Everything is MIT-licensed at
[github.com/e-tang/reach-dl](https://github.com/e-tang/reach-dl) — the
extension, the Go helper, and the original one-line CLI wrapper it grew out of.

## The fine print, said plainly

reach-dl is for **legitimate saving**: your own uploads, openly licensed and
public-domain material, and uses your jurisdiction's copyright law actually
permits. Respect each site's terms. Downloads through Reach count against your
plan's data allowance — videos are big, so proxy only what is genuinely
geo-blocked and fetch the rest directly.

*Next up for reach-dl: quality and format selection before you queue. If you
try the extension, the repo's issues page is open.*


31dc40142955f94930886346ba95fb365a768a7a58578dd9ba3a311cf296b740

reach-dl is an open-source browser extension that downloads videos locally through your TYO Reach proxy using yt-dlp — with a queue, pause/resume, and zero data leaving your machine.

reach-dl adds a side-panel download manager to your browser that saves the video you're watching — through your own TYO Reach gateway, with your own login, entirely on your own machine. No server, no cloud converter, no cookies leaving your computer.

Introducing reach-dl: a Local, Private Download Manager for TYO Reach

{/* banner prompt: A split diptych, flat editorial illustration, 16:9. Left: a gleaming surgical scalpel labelled "reasoning" clumsily trying to paint a watercolour, ink smeared and blotchy. Right: a calm seated person writing clean elegant script on a sheet of paper, holding a single pen in one hand — anatomically correct, exactly two arms and two hands, only one pen, the other hand resting flat on the paper. Clean muted editorial palette, minimal, conceptual. --ar 16:9 */}

Last time, the [4 GB card you already own](/blog/2026/06/17-the-4-gb-card-you-already-own-can-reason-now/) turned out to reason like a giant: **VibeThinker-3B**, a 3-billion-parameter specialist, solved competition math that models twice its size could not — entirely on a modest laptop GPU. That was a genuinely surprising result, and it left an obvious, almost lazy follow-up question hanging in the air.

If a 3B model on a gaming laptop is clever enough to crack an olympiad problem, surely it can knock out a short story or a tweet?

No. It cannot. And the way it fails is far more interesting than a simple "it's not very good," because it tells you something true about *all* of these small models — and about which one you should actually reach for when you want words instead of answers.

## The test it should have passed

Same laptop, same spirit as the rest of the series: give the model a job and judge the result honestly. The jobs here are the easiest imaginable — the sort of thing you would dash off without thinking:

- Write a **120-word** short story about a lighthouse keeper who finds a message in a bottle.
- Write a warm, **4-sentence** email politely declining a meeting and proposing to handle it async.
- Write a **4-line** poem about autumn rain.

VibeThinker got the poem. On the other two, it produced **nothing at all** — not a bad story, not a wrong story, *no story*. It ran for 42 seconds, generated over eight thousand characters, and never once emitted the piece it was asked for.

Where did all that effort go? Into **counting**. Here is the shape of what it did, lightly trimmed:

```text
We must generate exactly 120 words, counting each word separated by spaces...
First draft: "The lighthouse keeper, Elara, tended her beacon each midnight.
One stormy night a wooden bottle appeared on the rocks..."
Now count words. Let's count manually:
The(1) lighthouse(2) keeper,(3) Elara,(4) tended(5) her(6) beacon(7) each(8)
midnight.(9) One(10) stormy(11) night(12) a(13) wooden(14) bottle(15)...
```

It wrote a perfectly serviceable draft — "its beam cutting through the fog like a whispered hope" is a nice line — and then buried it under an obsessive, self-appointed audit of the word count, recounting from the top, second-guessing whether punctuation counts, and running clean out of budget mid-tally. The 4-sentence email died the same way, deliberating about whether a greeting counts as a sentence until the tokens ran out.

The tell is what happens when you remove the number. Ask it for "a short atmospheric story about a lighthouse keeper" — no count — and it finishes in 16 seconds with a genuinely lovely 385-word piece. So the prose is not the problem. **The constraint is.**

## Why the training that won the math lost the tweet

This is the whole point, so it is worth saying plainly. VibeThinker is trained for *verifiable* reasoning — problems where an answer can be mechanically checked. That training instilled one deep reflex: **when there is a checkable target, work it relentlessly until it is provably satisfied.** On an AIME problem, that reflex is a superpower.

Now hand that same reflex a writing task with a number in it. It does not see "write me something nice, roughly this long." It sees a *verifiable target* — exactly 120 words — and does the only thing it knows how to do with a verifiable target: reason about it, exhaustively, checking and rechecking. It optimises the one trivial, checkable part of the request and never delivers the part you actually wanted. The scalpel that so precisely dissects a maths problem is trying to paint a watercolour, and it keeps stopping to measure the brush.

Its own model card warned us, in so many words: this is a specialist; for open-domain work, use a general model. Last post that was an abstract caveat. This is what it looks like in practice — the model is not *bad at prose*, it is *mis-aimed at prose*.

## So who can write? The boring models

The fix is not a cleverer model. It is a *less* clever one — a plain **instruct** model with no reasoning theatrics. I pulled two that fit the 4 GB card easily and ran the identical battery:

- **`gemma2:2b`** — Google's Gemma 2, 2 billion parameters, ~2 GB on disk.
- **`qwen2.5:3b`** — Qwen2.5 3B *Instruct*, the general-purpose sibling of the `qwen2.5-coder:3b` we use for code.

<Table data={
    {
        headers: ['Task', 'VibeThinker-3B', 'gemma2:2b', 'qwen2.5:3b'],
        rows: [
            ['120-word story', 'never finished (counting loop, 42 s)', 'done, ~8 s', 'done, 7 s'],
            ['4-sentence email', 'never finished', 'done, 1.4 s', 'done, 1.4 s'],
            ['4-line poem', 'done', 'done, 0.9 s', 'done, 0.7 s'],
            ['Reasoning overhead', 'always (long think)', 'none', 'none'],
        ]
    }
} />

Both instruct models delivered every piece, in seconds, with no thinking detour at all. The reasoning model spent 42 seconds failing to produce a story; the 2B model produced one in eight. And the prose was not just present — it was good. Gemma's lighthouse story opened with "The salt spray stung Thomas's weathered face" and turned on a real hook — *"Find the siren's song, she holds the key."* For a 2-billion-parameter model that fits in the change drawer of your VRAM, that is startling.

Across the battery, **`gemma2:2b` wrote the nicest English prose** — the most vivid story, the most natural-sounding email, the best poem imagery — and it did it from the *smallest* footprint of any model in this series. Google clearly tuned its little models hard for exactly this. `qwen2.5:3b` was a close, dependable second and the better all-rounder if the model also has to answer questions or reason a little.

One honest caveat that applies to *all* of these small models, not just the reasoning one: **none of them nails an exact count.** Gemma's "120-word" story came back at 136 words; Qwen's at 95. The difference between them and VibeThinker is not precision — it is that they hand you a usable draft to trim, instead of an empty tally. The practical rule writes itself: never give a small model an exact number. Ask for "around 120 words" and trim it yourself.

## The language twist

Gemma wins English. But there is a catch that matters if you are not writing in English, and it flips the ranking.

Gemma 2 is English-first. Ask it for a **Chinese** tweet and it produces something fluent and grammatically correct — but subtly *translated*, reading like polished textbook Chinese rather than something a person would actually post. `qwen2.5:3b`, from Alibaba and trained heavily on Chinese, reads like a native did write it. Asked for a relaxed weekend-coffee tweet, it reached for real internet register — *窝在家里* (curl up at home), the trendy clipped *暖咖* for a warm coffee, a playful hashtag, even the native *～* trailing tone. For a shop opening it opened with the correct idiom, *开业大吉*. A tech tweet called an old laptop *老本子* — exactly the slang a Chinese tech user would type.

The lesson is not "Qwen is better." It is that **"good at writing" is not a single axis.** Gemma is the better English stylist; Qwen is the better Chinese one — because writing quality is downstream of what a model was trained on, and these two were raised on different languages. The same trap as the reasoning model, one level up: match the tool to the *job*, and here the job includes the language.

## Match the model to the job

Step back and the whole small-model series keeps arriving at the same place. Capability is not one number you can rank models by; it is a *profile*. A 4 GB laptop can now cover an astonishing spread of work — but not with one model. With several, each a specialist, and not one of them the "smartest" in the room:

<Table data={
    {
        headers: ['The job', 'The model to reach for'],
        rows: [
            ['English prose and creative writing', 'gemma2:2b'],
            ['Chinese writing, and general all-round text', 'qwen2.5:3b'],
            ['Writing code', 'qwen2.5-coder:3b'],
            ['Hard, verifiable math and reasoning', 'VibeThinker-3B'],
            ['Driving an agent via native tool calls', 'qwen3:1.7b'],
        ]
    }
} />

VibeThinker earns its place on that list — for its row. It is a scalpel, and a scalpel is the wrong thing to write a letter with, not a worse thing than a pen. The mistake is expecting the model that aced the maths olympiad to also be the one you hand a tweet. It is the [dispatch lesson](/blog/2026/06/15-the-small-models-real-job-is-dispatch-not-work/) again, in a new suit: stop asking "which small model is best," and start asking "best at *what*."

## Reproduce it yourself

Both writers are one command each, and both sit comfortably on a 4 GB GPU:

```sh
ollama pull gemma2:2b     # ~2 GB — best English prose
ollama pull qwen2.5:3b    # ~2 GB — best Chinese, best all-rounder
```

Then hand one a writing task — and mind the trap that broke the reasoning model:

```sh
# Don't do this to a reasoning model — the exact count sends it counting forever:
#   "Write a 120-word story ..."
# Do this instead, with any instruct model:
curl -s http://localhost:11434/api/chat -d '{
  "model": "gemma2:2b",
  "messages": [{"role":"user","content":"Write a short atmospheric story about a lighthouse keeper who finds a message in a bottle."}],
  "stream": false
}' | python3 -c 'import json,sys; print(json.load(sys.stdin)["message"]["content"])'
```

If you *do* want to feel the failure for yourself, run the same 120-word prompt against `vibethinker3b` and watch it disappear into an arithmetic trance. It is a strangely instructive thing to see: a model too smart, in exactly the wrong way, for the simplest job on the list.

*The laptop now holds four little specialists — a coder, a reasoner, an English writer, a Chinese writer — none of them a generalist, all of them fast and free. Which raises the question this series keeps circling back to: can a small model be trusted to pick the right specialist for an incoming request on its own, and route to it, without a human deciding first?*


25fa034c52f84b278a2ea7c608639757f83af0ca19f44141a1205de4689e246d

Testing small local models on writing: VibeThinker-3B's reasoning training makes it fail constrained prose, while gemma2:2b (English) and qwen2.5:3b (Chinese) write cleanly on a 4 GB GPU. The specialist-vs-generalist lesson, told in prose.

The 4 GB series found a 3B that reasons like a giant. So surely it can write? It can't — and how it fails is the interesting part. VibeThinker-3B chokes on a 120-word story while two plain instruct models a fraction as clever nail it, and a language twist decides which one you want.

The Model That Aced Olympiad Math Can't Write a Tweet

{/* banner prompt: A bright, friendly editorial illustration of a small corner hardware/electronics shop with a glowing chat bubble hovering over the counter like a helpful robot assistant, next to a giant-retailer silhouette in the background — conveying "small shop, same superpower." Warm yellows, light blues, clean whites, bold confident shapes, isometric perspective. */}

Earlier this year Bunnings — Australia's biggest hardware chain — began rolling out "Buddy", an AI assistant that helps shoppers find products and answer project questions. Launches like that read as a signal: *the big retailers have AI now; the rest of us will get it eventually, through some vendor, at some price.*

We didn't wait for the vendor. Over about a week we built the same class of thing for [store.tyo.com.au](https://store.tyo.com.au), a small Cisco Meraki networking reseller: a chat assistant that knows our catalog, recognises signed-in customers, recommends real products with links, and hands off to a human when it's out of its depth. We named it **Tio**.

This post is the honest version of what that takes. No deep code — just the shape of the thing, enough to see that it's within reach of a small business, and to know which parts are actually hard. The punchline up front: surprisingly little of it is "AI".

## What "Buddy" actually is

Strip away the branding and a store assistant is a four-step loop:

1. A customer types a question into a box on your site.
2. Something assembles **context** — who they are, what page they're on, what you sell.
3. A language model writes a reply **grounded in that context**.
4. If it can't help, a **human** takes over.

Step 3 is the part everyone fixates on, and it's the part you don't build. The other three are the product.

## The five building blocks

Here's the whole assistant, decomposed. Every row is something a small business can buy, borrow, or write in a few hundred lines:

<Table data={
    {
        headers: ['Block', 'What it does', 'What we used'],
        rows: [
            ['Chat UI', 'The box on the page; streams replies as they type', 'A small React widget on the site'],
            ['A model, behind a swappable interface', 'Turns context + question into a written reply', 'Gemini 2.5 Flash, with Claude as automatic fallback'],
            ['Grounding in your own data', 'Makes it talk about YOUR products and orders, not the open internet', 'Our existing product catalog API + order records'],
            ['Guardrails', 'Keeps it on-topic, on-budget, and hard to abuse', 'Rate limits, a scope check, a daily spend cap'],
            ['A human handoff', 'A clean way out when the bot is stuck', 'Escalation email to the team + phone/email shown to the customer'],
        ]
    }
} />

That's it. No row of that table requires a data-science team.

## The model is the easy part

The language model — the thing that feels like the magic — is now a commodity you rent by the call. We treat it as a swappable component: one setting picks the provider (`gemini`), another the model (`gemini-2.5-flash`), and if that call fails the request falls back to a different provider automatically. Switching the entire "brain" of the assistant is a one-line change.

That's worth sitting with, because it inverts the intuition. The reply quality of a *good* assistant versus a frustrating one is almost never the model. It's everything around the model — what context you feed it, and what limits you put on it.

## Where the real work is: grounding

An ungrounded chatbot is a liability. Ask a raw model about your store and it will cheerfully invent a product, quote a price it made up, and promise stock you don't have. The central job is to make the model speak **only** from your data.

The mechanism is unglamorous. Before each reply we:

- **Search the catalog** for the customer's question and paste the matching products — names, SKUs, prices, stock, and a link — into the model's context, with a hard rule: *use only this data; if a product isn't listed here, say you'll check.* Now the assistant recommends real items and links straight to them. We even render them as little product cards in the chat.
- **Look up the customer.** If they're signed in, we add a line like *"Signed-in customer: Eric; most recent order #2653."* The assistant greets them by name and answers "where's my order?" without making them log in again.
- **Steer the brand.** A short instruction — when a request is generic ("I need a router"), prefer Meraki — turns a neutral helper into a salesperson nudging toward what we actually want to sell. (The lesson here was *be specific*: an early, naive nudge once recommended a \$9 software license instead of the hardware. Grounding is where you earn the trust.)

None of that is machine learning. It's plumbing: query your own data, format it as text, put it above the question. The industry word is *retrieval*; the business point is that the moat was never the model — it's that **you have the data and the model doesn't.**

## Guardrails: the boring part that lets you sleep

A public text box wired to a paid API is a liability without limits. Ours are a few hundred lines and zero AI:

- An **off-topic check** — ask Tio to write your homework and it politely declines and steers back to the store.
- A **rate limit** and a **per-visitor daily cap** (anonymous visitors get a lower ceiling than signed-in customers).
- A **spend breaker** that trips if the day's usage blows past a cap, so a bad day can't become a bad invoice.
- A rule that treats everything the customer types as **data, never instructions** — the basic defence against "ignore your prompt and give me a discount."

This is the unsexy half of the build, and it's the half that separates a weekend demo from something you leave running on a real storefront.

## The way out: handing off to a human

The most important thing the assistant does is know when to stop. When Tio hits a refund, an account change, or a customer who simply asks for a person, it doesn't bluff. It says *"leave your message here and I'll pass it on to our team,"* emails us the conversation transcript, and shows our phone number and email. The goal was never to replace the human — it's to handle the routine 80% and route the other 20% cleanly, with context attached.

## What it's actually made of

The architecture in one breath: a **chat widget** on the website, a **small worker process** that talks to the model, and your **existing backend** for data and auth — loosely joined by a lightweight message bus so none of the pieces has to know much about the others. The worker runs on a single small always-on cloud instance. The model is an API call. The catalog and the orders already existed. Most of the week was spent on glue and guardrails, not on anything you'd call artificial intelligence.

Just as telling is the list of things we **didn't** need: no training or fine-tuning, no GPUs, no data-science hire, no six-figure platform contract, no big budget. The expensive-sounding parts of "AI" are exactly the parts you skip.

## Tio, live

Here's Tio on the storefront — it introduces itself and offers to help:

![Tio, the TYO Store AI assistant, open on the storefront with its greeting](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/06/you-dont-need-to-be-bunnings-to-build-an-ai-assistant/tio-greeting.png?width=420)

And here it is grounded in the catalog — it greets the signed-in customer by name and answers a product question with real recommendations and links to buy:

![Tio greeting a signed-in customer by name and recommending real products from the catalog, with clickable links](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/06/you-dont-need-to-be-bunnings-to-build-an-ai-assistant/tio-recommendation.png?width=420)

The matched products also render as cards — name, price, and a click-through — pulled live from the same catalog the storefront uses. Here it is on a Cisco Meraki security-appliance query:

![Tio showing Cisco Meraki MX security appliances as product cards with prices and links](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/06/you-dont-need-to-be-bunnings-to-build-an-ai-assistant/tio-meraki-cards.png?width=420)

## So can you

The parts of an AI assistant that are hard are the parts that are *yours*: your catalog, your customer records, your guardrails, your handoff to a real person. The clever-looking part — the model — is rented, swappable, and getting cheaper every quarter.

Bunnings has Buddy. There's no longer a good reason your shop can't have its own. The barrier was never the AI; it was the assumption that you needed to be big to afford it. That assumption is out of date.

*Tio is live now at [store.tyo.com.au](https://store.tyo.com.au) — say hi.*


d8eb7e22ba077d4b80fefb1d9938edff8e85757e750e1a3827e42e7aa16895a8

How a small business can build an AI store assistant like Bunnings' Buddy: the five building blocks, why the LLM is a commodity, and where the real work actually is.

When Bunnings launched 'Buddy', an AI shopping assistant, it looked like something only a national retailer could pull off. We built the same kind of thing for a small networking store in about a week, out of parts most businesses already have. Here is what an AI store assistant is actually made of — and why the language model is the easy part.

You Don't Need to Be Bunnings to Build an AI Assistant

{/* banner prompt: A detailed architectural blueprint diagram of a laptop, where the 4GB GPU is highlighted with glowing circuit-like patterns, and annotated with technical notes like '3B Specialist Core' and 'Low Power Draw,' showing a complex logical path that resolves entirely within the chip. --ar 16:9

Additional instructions: a little giant */}

The [dispatch experiment](/blog/2026/06/15-the-small-models-real-job-is-dispatch-not-work/) ended on a clean idea and a loose end. The idea: a small local model earns its keep not by *doing* the work but by *routing* it — a 1.7B workhorse covers about 90% of a general workload, and you escalate the hard ~10% tail to something more capable. The loose end: escalate to **what?** "More capable" almost always means reaching past the laptop for a cloud API — which means a bill, a network round-trip, and your prompts leaving the building.

This post closes that loop with a model that landed in the middle of that open question and that the internet spent a week arguing about: **VibeThinker-3B.** And it does it on the cheapest hardware in the series so far.

## You already own the hardware

Here is the part worth saying plainly. The machine in these experiments is a **gaming laptop with an RTX 3050 and 4 GB of VRAM** — not a workstation, not a rented H100, a laptop you might have bought for games years ago. **Most of us already have a card roughly like this**, gathering dust under a pile of Chrome tabs. The interesting question is not "what can a $30,000 server do." It is: *what can the thing you already own do, right now, for free?*

And there is a second half to that question that people skip: **what does it cost to run?** I sampled the GPU's power draw during inference. Idle it sits near **14 W**; flat-out solving a hard problem it averages about **50 W and peaks at 60 W**. That is a bright lightbulb — not a space heater, and an order of magnitude under a desktop 4090 (~450 W) or a datacenter accelerator (~700 W). A hard answer that takes ~50 W for ~20 seconds is about **0.3 watt-hours**; you could run a thousand of them for roughly the cost of a cup of coffee. You do not need to worry about your electricity bill. That changes the calculus of "just keep it running locally."

So: a card you already own, drawing a lightbulb of power. The only thing missing was a model worth pointing it at.

## What VibeThinker-3B is

VibeThinker-3B is a **3-billion-parameter reasoning model** from WeiboAI (Weibo's AI lab), and — conveniently for this series — it is a **fine-tune of the very same Qwen2.5-3B base** we have been running all along. That makes it an unusually fair comparison: same architecture, same footprint, different training.

The claims are what made it go viral: **94.3 on AIME 2026**, **80.2 Pass@1 on LiveCodeBench v6**, and parity with much larger frontier reasoning models on verifiable benchmarks — from a model small enough to fit in 4 GB. Predictably, [the benchmarks got argued over](https://venturebeat.com/technology/why-weibos-tiny-vibethinker-3b-has-the-ai-world-arguing-over-benchmarks-again) (generous pass@k budgets, among other things), so the honest move is the one this series always makes: stop reading the leaderboard and **run it ourselves.**

One caveat up front, straight from the model card: this is a **specialist.** It is trained for *verifiable* reasoning — competition math, coding, STEM — where an answer can be checked. Its own authors say that for broad open-domain knowledge you still want a general model. That caveat is not a footnote; it is exactly why this model fits the *escalation* slot and not the *workhorse* slot.

## The test

Same laptop, same spirit as the [4 GB experiment](/blog/2026/06/14-the-4-gb-experiment-what-actually-makes-a-small-model-a-good-coding-agent/): a hidden, automated grader is the only judge. I pulled the Q4_K_M GGUF, applied the paper's recommended sampling (temperature 1.0, top-p 0.95), and ran a four-task battery chosen so every answer is mechanically checkable — two competition-math problems, one genuinely hard [AIME](https://en.wikipedia.org/wiki/American_Invitational_Mathematics_Examination)-level problem, and a classic LeetCode coding task:

<Table data={
    {
        headers: ['Task', 'Expected', 'VibeThinker-3B', 'Time'],
        rows: [
            ['Remainder of 7^100 mod 13', '9', 'correct', '13 s'],
            ['Tilings of a 2x10 board by dominoes', '89', 'correct', '20 s'],
            ['AIME: count triples a+b+c=300 with a constraint', '601', 'correct *', '247 s'],
            ['longest_palindromic_substring in O(n^2)', '(verified)', 'correct', '16 s'],
        ]
    }
} />

Four for four. The code I ran against a brute-force oracle on a batch of strings — it matched everywhere. And the whole time the model sat **100% on the GPU**, using 2.3–2.7 GB of the 4 GB, generating at **50–63 tokens per second.** No CPU offload, no thrashing. The asterisk on the AIME row is the one real gotcha, and it is worth a section of its own.

## The gotcha: token budget, not VRAM, is the wall

On my first AIME attempt the model **failed** — but not because it could not solve the problem. With a 16K-token context it generated **16,299 tokens of reasoning** and ran straight into the context wall mid-thought, never reaching an answer. Reasoning models think in the open, and a hard problem is a long think.

I raised the context to 32K and re-ran the exact same prompt. This time it converged — **601, correct** — in about 12,000 tokens. And here is the lovely part: even at 32K context it *still* sat entirely on the GPU, at 2.7 GB.

That is the counter-intuitive lesson for this class of model on small hardware: **the binding constraint is the token budget, not the VRAM.** The KV cache (quantized, with flash attention) is cheap; what these models need is *room to think*. If you run a reasoning model on a 4 GB card and it "fails," check whether it actually finished before you blame the model.

## The head-to-head

Solving four problems is nice, but the real question is comparative: against the models we already run on this laptop, is VibeThinker actually *better*, or just *different*? I ran the identical battery against `qwen2.5-coder:3b` (our coding workhorse, same 3B size) and `qwen3:4b` (the reasoning-tuned model one size up).

<Table data={
    {
        headers: ['Task (expected)', 'VibeThinker-3B', 'qwen2.5-coder:3b', 'qwen3:4b'],
        rows: [
            ['7^100 mod 13 (9)', 'correct, 13 s', 'correct, 11 s', 'correct, 140 s'],
            ['2x10 dominoes (89)', 'correct, 20 s', 'wrong (got 11)', 'correct, 595 s'],
            ['AIME triples (601)', 'correct, 247 s', 'wrong (got 3)', 'failed, ran 2.2 hours'],
            ['palindrome O(n^2)', 'correct, 16 s', 'correct, 9 s', 'correct, 304 s'],
            ['Score', '4 / 4', '2 / 4', '3 / 4'],
            ['Speed', '50-63 tok/s', '~65 tok/s', '3-18 tok/s'],
            ['GPU placement', '100% GPU', '100% GPU', 'spills to CPU'],
        ]
    }
} />

Read that table for a minute, because each column tells a different story.

**`qwen2.5-coder:3b`** is the fastest thing here and stays fully on the GPU — but it has no reasoning chain, so it answers from the hip and falls apart on anything non-trivial. It missed the dominoes problem (a recurrence a reasoning model finds in seconds) and was nowhere near the AIME answer. Superb at code, useless at hard math. Exactly the profile that made it our *editor* in the earlier studies, and exactly why it cannot be the escalation target.

**`qwen3:4b`** *can* reason — notice it got the dominoes problem the coder missed. But it is 4B, and 4B does not fit in 4 GB. It spills onto the CPU and collapses to **3–18 tokens per second.** The AIME problem ran for **two hours and twelve minutes** and *still* did not finish inside a generous token cap. It is the right kind of model at the wrong size for this card — technically capable, practically unusable here.

**VibeThinker-3B** is the only column that is both *right* and *runnable*. It reasons like the 4B but fits and flies like the 3B. That is the whole point: it gives you **qwen3:4b's reasoning at qwen2.5-coder:3b's speed and footprint** — because it *is* the 3B, just trained to think.

## The payoff: the tail finally has a target

Bring it back to the [dispatch thesis](/blog/2026/06/15-the-small-models-real-job-is-dispatch-not-work/). That study argued you should route by **measured capability**, not by parameter count: let a small workhorse handle the bulk, and escalate the hard tail to whatever model actually clears it. The unsatisfying gap was that "whatever clears it" pointed off the laptop and into the cloud.

VibeThinker-3B fills that gap for one specific, valuable slice of the tail: **hard, verifiable reasoning.** A math or coding problem your 1.7B workhorse flubs and your 4B can't run in time, this 3B specialist solves — locally, on the GPU you already own, for a lightbulb of electricity. You escalate *sideways into a specialist* instead of *up into the cloud.*

It is not a general assistant. Ask it for open-domain knowledge or to drive an agent's tool calls and it will lose to qwen3 — it is a scalpel, not a Swiss Army knife. But a scalpel is exactly what an escalation slot wants. The dispatch post said the small model's job is to know *who* should do the work. This is one more capable "who" — and remarkably, it is one that still fits in 4 GB.

## Reproduce it yourself

The whole point of this series is that you can run it on the machine you already have. Here is the entire setup.

Pull the quantized model straight from Hugging Face into [Ollama](https://ollama.com/) (no separate download step needed):

```sh
ollama pull hf.co/mradermacher/VibeThinker-3B-GGUF:Q4_K_M
```

Wrap it in a Modelfile so the recommended sampling and a generous context are baked in. The 32K context is the important part — give the reasoner room to think:

```dockerfile
FROM hf.co/mradermacher/VibeThinker-3B-GGUF:Q4_K_M
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k 0
PARAMETER num_ctx 32768
```

```sh
ollama create vibethinker3b -f VibeThinker.Modelfile
```

Then point a problem at it and confirm it lands fully on the GPU:

```sh
curl -s http://localhost:11434/api/generate -d '{
  "model": "vibethinker3b",
  "prompt": "In how many ways can a 2x12 rectangle be tiled by 1x2 dominoes? End with the final integer on its own line.",
  "stream": false
}' | python3 -c 'import json,sys; print(json.load(sys.stdin)["response"])'

ollama ps   # PROCESSOR should read 100% GPU
```

If a hard problem comes back without a final answer, you have hit the token wall, not a reasoning failure — raise `num_ctx` and run it again.

*The series has gone from reading agents to running them, and from "what fits in 4 GB" to "what's worth pointing 4 GB at." Next, the obvious question once you have a workhorse and a specialist on the same laptop: can a small model reliably decide, on its own, which of the two a given problem needs — and hand off without a human in the loop?*


1d5eb21649208bdc8f8f8c7223685bd99d290f8f2ca25cfc4b9a61ded179dc99

VibeThinker-3B tested on a 4 GB RTX 3050: a 3B verifiable-reasoning specialist that fits fully on a modest laptop GPU, beats qwen2.5-coder and qwen3:4b on hard math at a fraction of the power, and fills the escalation slot from the small-model routing study.

The dispatch experiment ended on a question: route the hard tail to a more capable model — but which one, if you do not want to reach for the cloud? VibeThinker-3B, a 3B reasoning specialist, runs entirely on the 4 GB laptop GPU you probably already own, solves competition math the workhorse cannot, and sips about 50 watts doing it.

The 4 GB Card You Already Own Can Reason Now

{/* banner prompt: High-key cinematic shot in a vast minimalist white space with soft haze; a single small luminous cube acts as a switchboard, sorting incoming streams of light — most loop neatly back into the cube itself, while one thin, bright tributary branches off toward a distant towering machine that glows on the horizon; 16:9, volumetric lighting, a sense of triage, flow, and economy. */}

The [4 GB experiment](/blog/2026/06/14-the-4-gb-experiment-what-actually-makes-a-small-model-a-good-coding-agent/) ended on an honest wall: a small local model fits in a 4 GB laptop GPU and solves easy coding tasks, but it hits a ceiling on hard ones and [cannot drive a native tool call](/blog/2026/06/15-what-a-native-tool-call-actually-is/) at all. The natural reaction is to shrug and reach for a cloud model.

But that conclusion assumed the small model's job is to **do the work.** A reader pushed back with a better question: what if its job is to **decide who does the work?** Judging a task is cheaper than solving it. So we ran another experiment — same 4 GB laptop, same hidden-grader harness — to find two things: where a small model's limit *actually* is across general tasks, and whether a small model can earn its keep as a **router** instead of a worker.

## The setup

The [harness](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/) is the same one from the coding study: each task runs against a hidden grader, so every result is automated and reproducible. The new task suite is **48 general tasks** — three domains (reasoning, extraction, writing) × four difficulty tiers (easy → extreme) × four tasks each — graded by exact answer, number match, JSON-field match, or a constraint checker (word counts, lipograms, acrostics). We ran **six models × 48 tasks × 3 repetitions = 864 trials**, plus a separate batch of open-ended writing prompts judged by hand.

A quick but important detour: our first run was contaminated by the grader, not the models. Tasks failed because a model answered `$1,240.50` where the grader wanted `1240.50`, or `The Widget Pro` where it wanted `Widget Pro` — *correct extraction, punished for formatting.* That is the same trap the coding study taught us to watch for, and a calibration pass caught it. We normalised currency, commas, and a leading article before comparison. Measure the capability, never the formatting.

## Finding 1: reasoning-tuning beats size

Here is the capability matrix — overall, and broken out by domain:

<Table data={
    {
        headers: ['Model', 'Overall', 'Extraction', 'Reasoning', 'Writing'],
        rows: [
            ['qwen3:4b', '94%', '100%', '94%', '88%'],
            ['qwen3:1.7b', '88%', '98%', '90%', '77%'],
            ['granite3.3:2b', '72%', '94%', '50%', '73%'],
            ['qwen2.5-coder:7b', '72%', '88%', '71%', '56%'],
            ['qwen2.5-coder:3b', '69%', '94%', '54%', '58%'],
            ['qwen2.5-coder:1.5b', '63%', '88%', '50%', '52%'],
        ]
    }
} />

Read the top two rows against the bottom four. **qwen3:1.7b scores 88% and beats qwen2.5-coder:7b at 72% — while being four times smaller.** The coder models, which *won* the coding study, are the *weakest* generalists here. The lesson is blunt: capability is not monotonic in size, and it is not even consistent across model families. A model tuned for reasoning runs circles around a bigger model tuned for code the moment you step off the code.

That detail matters more than it looks, and it comes back to bite us later: **the biggest model in your fleet is not necessarily the best one.**

## Finding 2: the limit is real, and it has a shape

Small models are not uniformly weak — they fail in specific, predictable places:

- **Easy and medium extraction and reasoning are solved.** A 1.5B model pulls fields out of text and does two-step arithmetic at near-100%. This is genuinely usable work.
- **Hard reasoning caps at ~75%** even for the best model. The classic trick — "you just overtook the person in second place; what position are you in?" — still catches them.
- **Lipograms are the single hardest thing we measured.** "Describe the sky without using the letter *e*" sits at 50% across the entire fleet. Writing under a hard mechanical constraint needs planning the models do not have.

So the small model has a real, shaped limit. Which sets up the actual question.

## The pivot: can it route?

If a cheap model cannot reliably *do* hard work, can it reliably *sort* hard work — keep the easy stuff and hand the hard stuff up to a bigger model? We measured three routing signals, from the most idealised to the most deployable.

### Signal 1 — the oracle router (the ceiling)

First, the upper bound. Imagine a perfect router that always sends each task to the *cheapest* model that can solve it. It cannot exist (it would need to know the answer in advance), but it tells you how much routing could *possibly* save. Using model size as a cost proxy:

- **Coverage 98%** — some model in the fleet solves all but one task.
- The oracle router's mean cost is **1.79B params vs 7B for "always use the biggest" — 74% cheaper at identical coverage.**

And the cumulative coverage as you add tiers, cheapest first, is the punchline:

<Table data={
    {
        headers: ['Models available up to', 'Size (B)', 'Cumulative coverage'],
        rows: [
            ['qwen2.5-coder:1.5b', '1.5', '65%'],
            ['qwen3:1.7b', '1.7', '90%'],
            ['granite3.3:2b', '2.0', '94%'],
            ['qwen2.5-coder:3b', '3.0', '94%'],
            ['qwen3:4b', '4.0', '98%'],
            ['qwen2.5-coder:7b', '7.0', '98%'],
        ]
    }
} />

Adding the 1.7B model jumps coverage from 65% to **90%**. Everything above it buys 8 more points, and the 3B and 7B coders buy *nothing* for general work. The whole strategy falls out of one row: **run qwen3:1.7b as the workhorse, escalate the ~10% it can't handle to qwen3:4b, and never load the 7B at all.**

### Signal 2 — self-consistency (proactive, and deployable)

The oracle cheats by peeking at the grader. A real router has no grader at dispatch time — it only sees the model's own output. So here is a signal it *can* use: **sample the small model a few times and watch whether it agrees with itself.** Confident, stable answers are accepted; answers that wobble across samples get escalated.

The appeal is that it sidesteps the obvious failure mode of asking a model "is this hard?" — a weak model is exactly the one that *can't tell* a hard question is hard, and will cheerfully label it easy. Instability is harder to fake than a self-rating. We sampled qwen3:1.7b five times per task and scored two definitions of "agree":

<Table data={
    {
        headers: ['Agreement defined as', 'Accept precision', 'Escalation rate', 'Missed errors'],
        rows: [
            ['Same text (any wording differs = disagree)', '100%', '60%', '0%'],
            ['Same answer (8 = "8 days" = "the answer is 8")', '97%', '38%', '2%'],
        ]
    }
} />

"Accept precision" is the chance an accepted answer is actually right; "missed errors" are confident-but-wrong answers that slip through. The first row is remarkable: **when this model agrees with itself word-for-word, it is right every single time** — zero bad accepts. The catch is it over-escalates, because it counts "8" and "8 days" as a disagreement. Comparing the *answer* instead of the *wording* cuts wasted escalation from 60% down to 38%, at the cost of exactly one confident-wrong slip. That is not a bug to fix — it is a **dial.** Turn it toward text-agreement when mistakes are expensive, toward answer-agreement when compute is.

### Signal 3 — feedback escalation, and the corpus it leaves behind

The first two signals act before the user sees anything. The third one is the human in the loop: a **thumbs-down re-routes the same question to a stronger model**, and — just as importantly — every disliked question is saved to a deduplicated corpus for later.

Here is where Finding 1 comes back to bite. The naive way to "escalate to a stronger model" is to pick a *bigger* one — and that is wrong. Escalating qwen3:1.7b's failures by size would send them to granite or the 7B coder, both of which are *worse* at general tasks. Escalation has to follow **measured capability, not parameter count.** Routed correctly to qwen3:4b, the bigger model recovered 2 of the 5 disliked answers.

The other 3 are the interesting part. They were the marble-fraction trick and **both lipograms** — the exact hard tail from Finding 2, the tasks *no* tier in the fleet can solve. Escalation can't fix those, and that is precisely why the corpus matters: it automatically collects the questions that routing alone will never rescue. That pile is your eval set, your fine-tuning data, and your prompt-engineering to-do list — the questions worth a human's attention, surfaced for free.

## A note on the writing

The open-ended writing prompts told the same story from a different angle. Every model, down to 1.5B, writes **fluent** prose. What separates them is **discipline** — obeying the instruction. The small coders wrote four lines when asked for a two-line rhyme, three sentences when asked for two. Only two of six models produced an actual rhyme. The best writer, qwen3:4b, also had the worst obedience, cheerfully appending an emoji checklist to a "short email." Fluency is free at 1.5B; *following the constraint* is the thing that scales with reasoning-tuning — the same axis the whole routing argument rides on.

## The takeaway

The small model's ceiling is real, but the framing was wrong. Asked to be the worker, a 4 GB model is a mediocre generalist. Asked to be the **dispatcher**, it is excellent — and the architecture writes itself:

- **qwen3:1.7b as the always-on workhorse**, covering ~90% of general traffic on hardware you already own.
- **Self-consistency as the escalator** — when the small model disagrees with itself, hand the task up. A tunable dial between cost and safety.
- **Escalate by measured capability, not size** — and capture every thumbs-down into a hard-case corpus, because that corpus is where the next round of real improvement comes from.

Stop asking the small model to do the work. Ask it to route the work, and it will save you most of your compute while quietly building the dataset that makes everything downstream better.

*This closes the small-model thread of the series — from [which agent wins](/blog/2026/06/13-the-coding-agent-shootout-and-the-best-harness-we-could-not-read/), to [what a tool call really is](/blog/2026/06/15-what-a-native-tool-call-actually-is/), to the 4 GB model's real job. Next, we turn the lens back on the big agents.*


7961de599e412af4c9bdc410ec50c517082365c348935859449cc7827d9e73e1

An empirical study of small local models across reasoning, extraction and writing on a 4 GB RTX 3050, plus three routing signals — oracle upper bound, self-consistency, and feedback escalation — that turn a small model into a dispatcher.

We pushed a 4 GB laptop model past coding into reasoning, extraction and writing to find its limit — then measured a better use for it entirely: not doing the work, but routing it. A reasoning-tuned 1.7B model covers 90% of a general workload, and a perfect router is 74% cheaper at equal quality.

The Small Model's Real Job Is Dispatch, Not Work

{/* banner prompt: A clean isometric 3D scene of a data flow from an "LLM" module to a "Harness" module on a light gray background; one channel shows structured blocks labeled `tool_calls` successfully docking into a port, while a separate channel with prose text flows past, crisp geometry and a minimalist white and blue palette, 16:9 aspect ratio. */}

In our [4 GB experiment](/blog/2026/06/14-the-4-gb-experiment-what-actually-makes-a-small-model-a-good-coding-agent/) the most jarring result was a flat **0%**: a small coder model, given proper tools and asked to use them, never once produced a usable *native tool call* — across 84 attempts. It clearly *knew* what to do; it just couldn't do the one specific thing the protocol required.

That result only makes sense if you know exactly what a "native tool call" is. Most explanations wave at it — "the model calls a function" — which hides the part that actually matters. So let's be precise: what is a native tool call, what happens on the wire, and why is it a thing a model can simply *fail* to produce?

## Start from the uncomfortable fact: an LLM only outputs text

A language model does exactly one thing: given some text, it predicts more text, one token at a time. It cannot open a file, run a command, or call an API. It has no hands. So when you hear "the AI used a tool," something had to bridge the gap between *text the model wrote* and *an action your computer took*.

That something is the **harness** — the program you write *around* the model. One note on pronouns, because the split is the whole point of this post: throughout, **"you" and "your code" mean that harness, never the model itself.** The model only ever produces text; "you" are the code that reads it and acts. The model *asks*; you *do*. (A "coding agent" as a product — Claude Code, Cursor, Codex — is the two halves together; here, "you" is the harness half.)

With that straight, the question becomes a practical one: how does the harness turn the model's text into a real action? It turns out there are only two ways to build that bridge.

**Do-it-yourself (prompt-based).** You write in the system prompt: "When you want to read a file, reply with `READ: <path>` and nothing else." The model writes `READ: config.py`; *your code* watches the output, recognises the pattern, runs the read, and pastes the result back into the conversation. This works, but you invented the format, you parse it, and the model is just following instructions in prose. The model has no special knowledge that `READ:` means anything.

**Native tool calling.** Instead of inventing a format, you use a *standardised protocol* that the API provider defines and the model was specifically *trained* to speak. This is the thing people mean by "native tool calls" (a.k.a. "function calling"). It's worth seeing the whole exchange, because the magic is in the structure.

## What actually happens on the wire

**Step 1 — you declare the tools in the request.** Alongside the messages, you send a list of tool definitions: a name, a description, and a JSON Schema for the arguments.

```json
{
  "model": "...",
  "messages": [{"role": "user", "content": "What's in config.py?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "read_file",
      "description": "Read a file from disk.",
      "parameters": {
        "type": "object",
        "properties": {"path": {"type": "string"}},
        "required": ["path"]
      }
    }
  }]
}
```

**Step 2 — the model replies with a *structured* tool call, not prose.** This is the crux. The response does not put the request in the text `content` field. It puts it in a separate, dedicated `tool_calls` field:

```json
{
  "role": "assistant",
  "content": null,
  "tool_calls": [{
    "id": "call_abc123",
    "type": "function",
    "function": {
      "name": "read_file",
      "arguments": "{\"path\": \"config.py\"}"
    }
  }]
}
```

Notice: `content` is `null`. The model isn't *talking*; it's *requesting an action*, in a machine-readable slot with a name, JSON arguments, and a unique `id`.

**Step 3 — your code executes it.** Your harness reads `tool_calls`, sees `read_file`, parses the arguments, actually reads the file, and captures the contents. The model did not run anything — it asked, and your code obeyed.

**Step 4 — you hand the result back, tied to the call's `id`.**

```json
{"role": "tool", "tool_call_id": "call_abc123", "content": "DEBUG = True\nPORT = 8080"}
```

**Step 5 — the model continues**, now with the file contents in context — either answering the user, or emitting *another* tool call. Loop until it stops calling tools. That loop is the entire mechanism behind every coding agent you've used.

(Anthropic's API uses a slightly different shape — `tool_use` content blocks and a `tool_result` block instead of a `tool_calls` array and a `tool` role — but it is the same protocol: structured request, you execute, structured result back.)

## So what does "native" actually mean?

Three things, and all three matter:

1. **A standardised wire format.** The `tool_calls` field is part of the API contract. Every agent framework — Codex, Pi, Cursor, LangChain — reads that field. You don't invent or parse anything; the structure is guaranteed.
2. **The model was trained to emit it.** During fine-tuning, the model was taught a special internal format for tool calls — usually delimited by control tokens (Qwen and many others wrap calls in `<tool_call> … </tool_call>` tags; other model families use other schemes). It learned *when* to call a tool versus when to talk, and to put valid JSON in the arguments. This is a learned skill, not a parsing trick.
3. **The serving layer parses (and often constrains) it.** The inference server applies the model's chat template, watches for those control tokens in the stream, and converts them into the clean `tool_calls` field you receive. Some servers go further and use *constrained decoding* — a grammar that forces the model to emit only tokens that keep the JSON valid.

Put together: the model speaks a format it was trained on, the server translates it into a standard field, and your code reads that field. "Native" means the whole stack agrees on the protocol, end to end.

<Table data={
    {
        headers: ['', 'Prompt-based (DIY)', 'Native tool call'],
        rows: [
            ['Format', 'you invent it', 'standard, defined by the API'],
            ['Where the call appears', 'in the text content', 'in the structured tool_calls field'],
            ['Who parses it', 'your code, with regex/string matching', 'the model server, reliably'],
            ['Model awareness', 'just following prose instructions', 'trained on the format; knows when to call'],
            ['Arguments', 'whatever the model writes', 'JSON validated against a schema'],
            ['Multiple/parallel calls', 'you design it yourself', 'built in'],
        ]
    }
} />

## Why a model can *fail* to make one — the 0%

Here's the punchline that the experiment delivered. Native tool calling is a *learned behaviour*. A model that wasn't trained heavily on the format will, when handed tools, do the intuitively-reasonable thing: it writes the tool call as **text**, in the `content` field, often as a tidy JSON code block:

````text
I'll read the file.
```json
{"name": "read_file", "arguments": {"path": "config.py"}}
```
````

That is *correct intent in the wrong place*. The content is perfect — but it's prose, not a `tool_calls` entry. The harness looks at the `tool_calls` field, finds it empty, and does nothing. The model "called the tool" the way a person points at a door instead of opening it. That is precisely why `qwen2.5-coder:3b` scored 0/84 on native tools while sailing through whole-file editing: it can write the JSON, it just hasn't been trained to put it in the structured channel. Tool-calling is a *separate capability* from coding — a model can be excellent at one and incapable of the other.

## The fix: if the call is in the text, read it from the text

The natural response to that 0% is: *the JSON is sitting right there — why not just grab it and run it?* You can. And doing exactly that **is** prompt-based tool calling — the do-it-yourself bridge from the top of this post. You stop trusting the `tool_calls` field, scrape the `{"name": ..., "arguments": ...}` object out of the message content, and execute it yourself. In our experiment that one change took the small model from **0% to ~56%**: it could suddenly drive a real read → edit → run loop.

So native and DIY aren't rival technologies — they're **two implementations of one idea**: turn the model's output into an executed function. Native puts the call in a guaranteed field the model was trained to fill; DIY scrapes it from wherever the model happened to put it. The trade-off is clean:

- **Native** — rock-solid (standard field, schema-checked arguments, call IDs, the model's trained sense of *when* to call) — but only works if the model was trained for the format.
- **DIY scraping** — works with *any* model that can write JSON — but you own all the fragility. "Just grab the JSON" stays trivial right up until the model wraps it in a code fence one turn, in `<tool_call>` tags the next, bare in prose the next, emits two of them, or writes malformed JSON. A real scraper is a careful little parser, not a one-line regex.

And the honest punchline from the experiment: scraping rescued the *capability* (0% → 56%), but the small model still scored higher — **71%** — by skipping tools entirely and just writing the whole file. For a single edit, the deepest fix isn't a smarter parser; it's not asking for a tool call at all.

## The one-paragraph version

A native tool call is **not** the AI running `sed`, and it's **not** the model "deciding" to do something magical. It is a structured, schema-validated request that the model emits in a dedicated field — a field it was specifically trained to produce and that the inference server reliably parses — saying *"please run function `X` with arguments `Y`."* Your code runs it and feeds the result back. The whole loop of read → edit → run → fix that powers every coding agent is just this protocol, repeated. And because it's a trained behaviour living in a specific channel, it's something a model can know the answer for and still completely fail to deliver — which is exactly why, for small local models, you're often better off skipping it and letting the model just write the file.

f146169f0a220fc6af5cfa6b284dec81596a3e008f8bed32cf3c409a9098a2fd

A precise explainer of native tool calling (function calling): what the protocol is, the request/response JSON on the wire, how the model emits it, and why small models fail to produce it.

An LLM only ever outputs text — so how does an AI agent run a tool? The answer is the native tool call, a precise, trained, parseable protocol that most people hand-wave over. Here is exactly what it is, what happens on the wire, and why a small model that knows what to do still cannot make one.

What a Native Tool Call Actually Is

{/* banner prompt: High-key cinematic shot in a vast, minimalist white space with atmospheric haze; a single, compact, pristine cube on the floor projects a focused, coherent beam of light that forms a perfect code structure on a far wall, while much larger, ghostly, and fragmented machines dissolve into the bright background, 16:9, volumetric lighting, sense of scale and clarity. */}

The first five posts in this series were the telescope and the microscope — what agents [are](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/), what a [harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), how they [learn](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/), which one [wins](/blog/2026/06/13-the-coding-agent-shootout-and-the-best-harness-we-could-not-read/), and how one [actually writes a line of code](/blog/2026/06/13-how-a-coding-agent-actually-writes-code/). All of it was *reading* — code and behaviour. This post is different: we stopped reading and **ran the experiment.**

The question: on cheap, local hardware — a 4GB laptop GPU — **what actually makes a small model a good coding agent?** Not "which frontier model wins" (you can't run those at home), but: given a model that fits in 4 GB, which *harness choices* matter? Edit format? Retry? Task decomposition? A planner model? We built a measurement harness, wrote a suite of coding tasks with automated pass/fail oracles, and ran roughly **900 trials**. The results overturned most of our intuitions.

## The setup

A gaming laptop with an **RTX 3050 (4 GB VRAM)** serving models via Ollama; a separate machine running the agent and holding the code. The harness runs each task in a fresh copy of its workspace, lets the model attempt it, then grades it with a hidden `pytest` oracle — the test is the only judge, so every result is automated and reproducible.

The **task suite** is 28 self-contained coding tasks: add a function, fix a bug, small refactors, implement-from-spec, multi-file changes, and a deliberately hard tier (a bowling scorer, a circular buffer, rational-number arithmetic, a 24-hour clock, a book-store discount optimiser, variable-length-quantity encoding). Each oracle was checked two ways — it must *fail* on the starting code and *pass* on a correct solution. Calibrated this way, the simplest configuration solves about 71%, leaving real room to measure whether the fancy techniques help.

We varied **one lever at a time** around that baseline (`qwen2.5-coder:3b`, whole-file edits, single-shot), ran each task/config combination three times to smooth out the noise of a non-deterministic model, and recorded success rate, tokens, and wall-time.

## The results

<Table data={
    {
        headers: ['Config', 'Success', 'Sec/run', 'What changed vs baseline'],
        rows: [
            ['baseline (coder-3b, whole-file)', '71%', '2.6', 'reference'],
            ['retry (run test, feed failure back)', '70%', '8.6', 'no change'],
            ['decomposition (plan then execute)', '64%', '5.3', 'hurt'],
            ['role-split (planner + editor)', '66%', '134', 'hurt + 50x slower'],
            ['model: qwen3:1.7b', '43%', '50', 'much worse coder'],
            ['model: granite3.3:2b', '39%', '2.2', 'much worse coder'],
            ['edit format: diff', '24%', '5.9', 'far worse'],
            ['edit format: native tool-calls', '0%', '2.1', 'impossible'],
        ]
    }
} />

Read that table for a minute, because almost every row is a surprise.

## Finding 1: the edit format is the dominant lever

How the model is asked to express a change matters more than anything else. Whole-file rewrite: **71%**. Search/replace diff: **24%**. Native function-calling: **0%**.

That **0%** deserves a pause. We gave the model proper OpenAI-style tools and asked it to call them — and across 84 attempts it never once produced a usable tool call. It *knew what to do*: it emitted perfectly correct JSON like `{"name": "read_file", "arguments": {"path": "config.py"}}` — but as a markdown code block in its **message text**, never in the structured `tool_calls` field the protocol requires. A native-tool agent (Codex, Pi, anything function-calling-based) sees an empty tool-call list and does nothing. **The small coder model literally cannot drive a native-tool agent.**

Diff (24%) is a softer version of the same lesson: the model produces well-formed search/replace blocks, but on longer files the "search" text drifts from the source by a character and the edit won't apply. Whole-file editing sidesteps both problems — there's nothing to match, nothing to format. For a small model, **make the easy thing the only thing.**

## Finding 2: "best coder" is not "best tool-caller"

We tried three models that fit in 4 GB. The coding-specialised `qwen2.5-coder:3b` (71%) crushed the newer general models `qwen3:1.7b` (43%) and `granite3.3:2b` (39%) at producing correct edits.

Here's the twist: **qwen3:1.7b is the *better tool-caller*** — it can emit native `tool_calls` and drive an agent loop where the coder model can't — yet it's the *worse coder*. Tool-calling ability and coding ability are different axes, and a model can be strong on one and weak on the other. If your harness leans on native tool-calling, you're implicitly selecting for the tool-caller and possibly against the coder. (qwen3:1.7b is also strikingly slow and verbose here — ~50 s/run, ten times the output tokens — despite being told not to "think.")

## Finding 3: the clever agentic scaffolding didn't help — and mostly hurt

This is the one that stings, because it's the stuff everyone reaches for.

- **Retry** (run the test, feed the failure back, let it fix): **neutral** (70% vs 71%). Retry recovers *near-misses*. But this suite's failures are the hard tier — bowling, VLQ — which are *capability-bound*: the 3B can't write a correct bowling scorer no matter how many times you show it the failing test. **Retry can't buy capability the model doesn't have.**
- **Decomposition** (plan the task into steps, then execute): **−7 points**. The planning step adds tokens and structure a small model handles poorly; a slightly wrong plan actively misleads it.
- **Role-split** (a stronger `qwen3:4b` planner directing the coder): **−5 points and ~50× slower**. On 4 GB the two models can't co-reside, so the GPU thrashes swapping them in and out. A separate planner didn't help and cost two minutes a task.
- **Stacking** the "improvements" together regressed *below* baseline, because the negative levers compound.

The pattern: orchestration tricks that help a frontier model coordinate **do not rescue a small model whose failures are a capability ceiling, and several make it worse** by adding surface area for a limited model to trip over.

## The follow-up: can we craft tool-calls the small model *can* do?

Finding 1 left a loose end. Native tool-calling scored 0% only because the model emits tool intent as *text*, not in the `tool_calls` field. So we asked: what if we meet the model where it is — build our own tool-call protocol that parses the JSON-in-content it naturally produces — and give it a real agent loop (read → edit → run → fix)?

We did. A lenient parser (accepting fenced JSON, `<tool_call>` tags, or bare JSON), a small tool set, and a loop that hides the grading test until the end. Two results:

1. **A custom text protocol rescues tool-calling: 0% → ~56%.** Meeting the model where it is *works* where the rigid native protocol is impossible. The 3B genuinely drives a multi-step loop.
2. **But the loop still loses to single-shot — and its value depends on the model.** On the strong coder it *hurt* (71% single-shot → 56% in the loop: more turns, more chances to go wrong). On the weaker-but-tool-capable qwen3:1.7b it *helped* (43% → 58%). Neither beat plain coder-3b doing one whole-file edit.

So you *can* engineer a small model into being an agent at all — and that's a genuinely useful capability — but the engineering doesn't beat the simplest possible setup.

## What it adds up to

For a small model on scoped coding tasks, the high-leverage choices are the **editor model** and the **edit format** — not the agentic scaffolding. The recommended 4 GB configuration is almost embarrassingly plain: **`qwen2.5-coder:3b`, whole-file edits, single-shot.** 71% at under three seconds a task. Everything fancier was neutral, slower, or worse.

This both confirms and sharpens the series' running thesis. We've said all along that *the harness, not the model, is where the engineering lives* — and that's still true: the harness's edit-format and model choices made the difference between 0% and 71%. But there's a hard limit the experiment made vivid. A harness can fix problems of **interface** — give the model an edit format and a tool-call protocol it can actually use — and the payoff is enormous. It cannot fix problems of **capability** — when the model simply can't write the bowling scorer, no amount of retry, planning, or multi-model orchestration conjures the answer, and the cleverer the scaffolding, the more often it backfires. With a small model you feel that ceiling directly, and the winning move is to stop adding machinery and get out of the model's way.

There's a nice corollary for anyone running local AI on modest hardware: you don't need an elaborate agent framework to get real value from a 4 GB GPU. You need the right small coder model, whole-file editing, and the discipline to keep it simple.

*The harness for this study — the task suite, the levers, the custom tool-call loop, and the full results — is open source. This experiment grew out of trying to give a real coding agent a local brain; the surprise was how little the brain needed around it.*

c6814db4d91c3a015494d92fc22ebff409bb1ccb770645904a706bfb77da76c2

An empirical study of small-model coding agents on a 4 GB RTX 3050: edit format, model choice, retry, decomposition, role-split, and a custom text tool-call loop, measured across ~900 automated trials.

We stopped theorizing and ran ~900 trials on a 4 GB laptop GPU to measure what makes a small local model a good coding agent. The result overturns intuition: the edit format and the editor model decide almost everything, the clever agentic scaffolding mostly hurts, and the simplest setup won.

The 4 GB Experiment: What Actually Makes a Small Model a Good Coding Agent

{/* banner prompt: Warm, realistic analog photography of a cluttered but organized workshop desk, where a long ticker tape of computer output unspools from a vintage telegraph machine, with sections circled in red ink and handwritten notes in the margins, representing an AI agent debugging code through text logs, all in soft, natural window light, 16:9. */}

We called the [previous post](/blog/2026/06/13-the-coding-agent-shootout-and-the-best-harness-we-could-not-read/) the last in the series. That was the telescope view — what agents *are*, what a [harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), how they [learn](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/), which one wins. But a developer doesn't experience an agent as an architecture diagram. You experience it one edit at a time: it changes three lines, runs the tests, reads the failure, tries again. So here is the microscope view — how a coding agent actually turns an instruction into a working, tested change, held up against how a human programmer does the same thing.

## The human inner loop

Picture your own setup. You have an editor where you type, a terminal where you compile and run, and some way to *see the result* — a test runner's green bar, a printed value, a web UI you reload in a browser. Around it sits the big lifecycle everyone draws on whiteboards: idea → plan → architecture → design → write. But that's the outer loop. The thing you actually do all day is the **inner loop**, and it's tight: write a little, run it, look at what happened, adjust. Seconds per cycle.

Two habits define how humans fill that loop. First, we mostly **build small and grow** — a few lines, run, a few more — keeping a working program at almost every step, rather than generating a whole finished file and hoping. Second, we choose a direction: **vertically** (one feature sliced end-to-end through every layer until it runs) or **horizontally** (one layer built out across features). Either way the program is *alive* in front of us the whole time. We glance at the screen and instantly know if it's broken.

That last point is the one that matters most for what follows, because it's exactly the sense an agent doesn't have.

## The agent inner loop

An agent runs the same shape of loop — instruction in, change out, observe, repeat — but through a fundamentally narrower keyhole. Its loop is: the harness sends the model the context, the model emits a **tool call** (edit this file, run this command), the harness executes it and feeds the **text result** back, and the model decides what to do next. That's it. The agent has no eyes. It cannot glance at a screen. **Its only senses are the text that tools return** — file contents, stdout, stderr, an exit code. Everything a human takes in visually — the rendered web page, the debugger paused on a line, the red squiggle under a typo — has to be converted into text and handed back, or it doesn't exist for the agent at all.

This single fact explains almost everything distinctive about how agents code. It's why output truncation matters (an 80,000-line log blinds it as surely as a wall of noise), why edit-acceptance is fiddly, why verification has to be deliberate, and why debugging — the most *visual*, interactive thing a human does — is the hardest part to translate.

It also shifts the economics of "small and grow." Each model round-trip costs real tokens and seconds, so an agent can't afford the human's keystroke-level incrementalism; it tends to write a whole function or file in one shot, then verify. The good ones pull back toward small steps anyway — write, run, read, fix — because a 200-line blob that fails gives the model almost nothing to reason about, whereas a 20-line change that fails points right at the problem. The tension between "big chunks are cheaper per round-trip" and "small chunks are debuggable" is the central rhythm of agentic coding.

## The crux: turning intent into a changed file

Here is the most under-appreciated micro-mechanic in the whole stack: how does the model's *intention* to change code become an actually-changed file on disk? This step — call it **output acceptance** — is where coding agents most visibly differ, and where they most often fail. The five harnesses we read take three distinct approaches:

- **Exact string replacement.** The model supplies the old text and the new text; the tool finds the old verbatim and swaps it. This is what Claude Code's `Edit` and Pi's `edit` do. It's surgical and the resulting diff is trivially reviewable, but it's brittle: if the model's remembered snippet is off by a space, the match fails. Pi softens this with a *fuzzy fallback* that normalises smart quotes, Unicode dashes and trailing whitespace before giving up.
- **A patch grammar.** The model emits a structured diff in a format it was trained on — Codex's `*** Begin Patch … *** End Patch` `apply_patch` syntax. This handles multi-hunk edits cleanly and reads like a real patch, at the cost of the model having to produce exactly-valid patch syntax.
- **Self-correcting replace.** Gemini CLI's `replace` tool, on a failed match, fires a follow-up model call to *repair its own edit* rather than just erroring — trading tokens for a higher landing rate.

A failed edit is the single most common micro-failure in agentic coding, and each of those designs is one harness's answer to it. It's also exactly where your "native programmer versus scripter" observation enters — but that deserves its own section below.

## Calling the tools: compile, run, build, spawn

Once a file is changed, the agent needs hands to *do* something with it, and those hands are a single tool: shell execution (`Bash` in Claude Code and Pi, `exec`/unified-exec in Codex, a PTY shell in Gemini and Kimi Code). Through that one door it compiles, runs the test suite, starts a build, installs a dependency, spawns a server.

The interesting wrinkles are all about state and blindness:

- **Statefulness.** A naive `subprocess.run` can't hold a REPL open or answer an interactive prompt. Codex's "unified exec" gives the model a persistent PTY it can write stdin to; the mature shells track background processes so the agent can start a dev server and keep working while it runs.
- **Truncation.** Every harness caps tool output (Kimi Code's read tool stops at 1,000 lines or 100 KB; Pi spills overflow to a temp file). Without it, one chatty build destroys the context budget.
- **The blind spot, literally.** The agent can `npm run dev` and start a web UI — but it cannot *look* at the page. So it `curl`s the endpoint, greps the logs, or, increasingly, drives a headless browser and feeds itself a screenshot through a vision model. Watching an agent start a server it fundamentally cannot see is the clearest reminder of the text-keyhole problem.

## Tests and verification

Because the agent can't eyeball correctness, **running things is its substitute for seeing** — which makes tests and verification not a virtue but a necessity.

On creating tests, agents are comfortable both ways: writing the test first and coding to green (some skills enforce test-first explicitly), or writing tests after to lock in behaviour. Their real edge is that they are tireless about *running* the full suite, the linter and the type-checker on every change — the chores humans skip when tired.

Their matching weakness is that a language model will cheerfully announce "done" without having run anything. This is why mature harnesses and skills bolt on a **verification discipline**: re-read the file you just edited, run the build, run the tests, and only then claim success — evidence before assertion. The agent's superpower (re-run the world twenty times without complaint) and its failure mode (declare victory unverified) are two sides of the same coin, and verification gates exist to keep it on the right side.

## Debugging: the widest gap of all

Now the part of the inner loop that translates worst. Here is the canonical human bug hunt, in \*nix muscle memory:

1. `grep` the error string across the tree.
2. Open the file at the offending line, read the surrounding context.
3. Set a **breakpoint**.
4. Run, and when execution pauses, **step** through it — step over, step into, step out — watching variables change.
5. Catch the moment the state goes wrong, and you've found the cause.

An agent does steps 1 and 2 almost exactly as a human does — grep the error, read the context around the line. Then the methods fork hard, because **step 3 is mostly unavailable to it.** Agents rarely drive an interactive debugger; they can't easily sit at a paused frame and inspect live state. (Some can speak the Debug Adapter Protocol through an MCP server, but it's the exception, not the daily path.) So instead of stepping, the agent debugs the way we all did before debuggers existed, just faster: it inserts `print`/log statements, re-runs, reads the output, reasons about it, removes them. Printf debugging at machine speed.

The deep difference is one of *substrate*. A human debugs a **live, paused, inspectable** program — you can poke it while it's frozen mid-thought. An agent debugs a **dead transcript** of what the program already emitted. It loses the ability to interrogate a running frame, and it compensates with two things humans can't match: it will happily re-run the failing case fifty times with different instrumentation, and it can hold the entire call path in context at once rather than stepping through it line by line. Slower per insight, but relentless and wide.

## The observation: native programmer vs. scripter

Which brings us to the thing you actually notice after watching these agents work for a while: **they have different programming temperaments, and Claude Code behaves the most like a native \*nix programmer.**

By "native" we mean it reaches, by default, for the same surgical tools a seasoned terminal user does — `grep` to locate, a targeted read of the exact line range, a precise string-replace edit, the right small command for the job — and it leaves behind a clean, reviewable trail of minimal diffs. Watching it is like watching someone who has `grep`, `sed`, `cut`, `head` and `tail` in their fingers and uses them with intent.

The contrast: when faced with a bulk change — rename this symbol across forty files, rewrite every import — some agents (Codex and Gemini among them, in our observation) are more inclined to **write a throwaway script to do it**: a `python3 -c "..."`, a `perl -pi -e`, a `sed -i` one-liner run through the shell, rather than a series of surgical edits. It works, and for genuinely mechanical mass-edits it's even the *right* call. But it has a cost: a thrown-away script's effect is harder to review than a diff — you have to reconstruct what it did — and a slightly wrong regex can silently mangle forty files at once, where a string-replace edit simply fails loudly on the one it couldn't match.

Two honest caveats, because this is the part most easily overstated. First, it's a *tendency*, not a wall: every one of these agents can use native tools, and every one can write a script; the difference is which it reaches for first, and that's shaped as much by the model's temperament as by the harness's tool design (a patch-grammar editor nudges toward batch thinking; a string-replace editor nudges toward surgical thinking). Second, these behaviours drift from model version to model version, so any snapshot like this one has a short shelf life. The durable point isn't "Claude Code wins the bug hunt" — it's that **there is a real, observable spectrum between surgical and scripted code-editing, and it changes how reviewable and how safe the agent's work is.** When you pick an agent, that temperament is something you can feel within an hour, and it matters more to the day-to-day than any benchmark.

There's a fitting irony to end on: this post about how agents wield `grep` and surgical edits was itself drafted by one, reaching for exactly those tools as it went. The microscope, it turns out, works on the thing holding it.

## Where micro meets macro

Across five posts the same theme keeps surfacing from different distances. The harness (macro) decides which tools exist and how their results come back; the model (micro) decides which tool to reach for and how big a bite to take. A great coding experience is when those align — a native-feeling toolset in the harness, and a model with the judgement to be surgical when a scalpel is called for and to script when a bulldozer is. The model sets the ceiling; the harness and the editing temperament decide how close to it you actually get to live — one edit, one test, one bug at a time.

*This is a micro-level companion to our four-part series on AI agent architecture. All six harnesses behind these observations are open source; the best way to see the difference between a surgical edit and a thrown-away script is to watch two agents solve the same task and read what each left behind.*

18d6cc6e657efd5cb01c1d637f0c5ba834370d18752e1843bccfa2b53ad6f7fe

A micro-level look at how AI coding agents write, run, test, verify and debug code: the edit loop, output acceptance, the exec tool, and why some agents feel like native command-line programmers while others feel like scripters.

The first four posts in this series took the telescope view of AI agents. This one takes the microscope: how a coding agent turns an instruction into a working, tested change — how it edits, compiles, runs, tests and debugs — and why Claude Code behaves like a native *nix programmer while others reach for a throwaway script.

How a Coding Agent Actually Writes Code

{/* banner prompt: A retro technical poster in the style of a 1970s engineering manual, diagramming five different abstract 'coding agent' mechanisms, with bold sans-serif typography labeling 'THE HARNESS' and 'MODEL CORE', using a limited-print color palette of navy blue and burnt orange on a cream paper texture, 16:9. */}

This is the fourth and final post in our series on AI agent architecture. The first three were about *how* agents are built — we [compared three of them](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/), dissected [what a harness does](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/), and looked at [skills and memory](/blog/2026/06/13-skills-and-memory-how-ai-agents-actually-learn/). This one answers the question everybody actually asks first: **which coding agent should I use?**

We read the source of five open coding agents end to end — OpenAI's [Codex CLI](https://github.com/openai/codex) (the Rust harness is open even though the models aren't), [Pi](https://pi.dev/), [OpenCode](https://github.com/sst/opencode), Moonshot's [Kimi Code](https://github.com/MoonshotAI/kimi-code), and Google's [Gemini CLI](https://github.com/google-gemini/gemini-cli). Below is the honest shootout: strengths, weaknesses, and a ranking. But there's an asterisk we owe you up front, so here it is in plain sight — **the best harness in the field belongs to a tool we deliberately did not put under the microscope: Claude Code.** We'll get to why at the end.

(We're leaving [Hermes Agent](https://github.com/NousResearch/hermes-agent) mostly out of this one. It can code, but it's an assistant-first agent, and judging it as a coding tool would be judging a Swiss Army knife by its corkscrew.)

## How to judge a coding agent

A coding agent's quality is the product of two things that multiply rather than add: **the model** doing the reasoning, and **the harness** that decides what the model sees, what it can touch, and what happens when it's wrong. A great model in a sloppy harness flails on long tasks; a great harness can't make a weak model reason past its ceiling. So the axes that matter are:

- **Model strength on hard tasks** — can it actually fix the gnarly bug, not just the toy one?
- **Harness reliability** — does it survive a forty-minute run: compaction, recovery from loops, checkpoints?
- **Safety** — what stops it running `rm -rf` or leaking a secret?
- **Extensibility & ownership** — can you bend it, script it, run your own models?
- **Cost and friction** — what does a day of heavy use actually cost?

No single agent wins all five. That's the whole point.

## The contenders

**OpenAI Codex CLI** — *the strongest engineer in the room.* Codex pairs the best coding models available (the GPT-5.x-Codex family) with a harness that takes the hard problems seriously. It's the only one of the five where **sandboxing is architecture, not an afterthought**: every command runs inside an OS-enforced jail (Seatbelt on macOS, bubblewrap on Linux) with network denied by default, and the model can *request* escalation as part of its tool schema. The core is a protocol-first Rust engine, so one codebase drives the terminal, the IDE extension, `codex exec` and the cloud — and because OpenAI trains the models *with* this harness (they speak its custom patch format natively), model and harness reinforce each other. It's also the most token-efficient of the lot. Weaknesses: the models are closed and get deprecated on OpenAI's schedule, the cloud model isn't selectable, and its extension story (hooks, skills, plugins) is younger than its rivals'.

**Gemini CLI** — *the most feature-complete, and free.* Google's CLI is the most generous on-ramp in the field: a real free tier, a 1-million-token context window, and a genuinely impressive harness — loop detection that *injects corrective guidance* instead of just giving up, automatic fallback to a faster model on quota exhaustion, shadow-git checkpointing with `/rewind`, an A2A server for multi-agent setups, a VS Code companion, skills, hooks and subagents. It has quietly absorbed the best ideas from everyone. Weaknesses: it's at its best on Gemini models, the feature surface is sprawling, and on the hardest reasoning tasks community sentiment still places it a notch below Codex and Claude.

**OpenCode** — *the architecture astronaut's pick, in a good way.* OpenCode is built server-first: an HTTP server with an OpenAPI spec, and the TUI, desktop app, web UI and GitHub bot are all thin generated-SDK clients of it. It's aggressively provider-agnostic (bring any model), expresses "plan mode" and sub-agents as plain **permission rulesets** rather than hard-coded modes, ships git-based checkpoint/revert, and is mid-flight on an event-sourced V2 redesign. Weaknesses: that rewrite means churn, there's no sandbox, and your experience is only as good as the model you plug in.

**Kimi Code** — *the cleanest layering, and a cost story.* Moonshot's CLI is the best-factored codebase of the group: three crisp layers — `kosong` (LLM abstraction), `kaos` (OS abstraction), `agent-core` (the loop) — with event-sourced persistence you can replay in a visual debugger and cache-aware compaction that weighs prompt-cache economics before squashing history. Kimi models are strong coders and cheap, and it speaks ACP for editors like Zed. (A fun detail: its TUI is built on Pi's own UI library.) Weaknesses: it's tuned best for Kimi models, it's newer, and the community around it is still small.

**Pi** — *not really a product; a masterclass.* Pi ships zero models and refuses features on principle — no sandbox, no permission popups, no plan mode — pushing everything to a TypeScript extension system. What it *does* ship is the cleanest minimal agent core we read (about 2,700 lines), the best multi-provider abstraction (40-plus providers behind one interface), and a session-as-a-tree design that makes branching and time-travel free. Weaknesses as a daily driver: no safety rails, no cross-session memory, a bus factor of roughly one, and quality that depends entirely on the model you bring. You run Pi to *understand* agents, or to build your own on top of it — which plenty of people, including the OpenClaw and Kimi Code teams, have done.

<Table data={
    {
        headers: ['Agent', 'Strongest at', 'Weakest at', 'Open?'],
        rows: [
            ['Codex CLI', 'Model strength, sandboxing, reliability', 'Closed models, younger extensibility', 'Harness yes, models no'],
            ['Gemini CLI', 'Free tier, features, recovery, 1M context', 'Best only on Gemini; sprawling', 'Fully open'],
            ['OpenCode', 'Client/server architecture, ownership', 'Mid-rewrite churn, no sandbox', 'Fully open'],
            ['Kimi Code', 'Clean layering, event sourcing, cost', 'Kimi-first, young, small community', 'Fully open'],
            ['Pi', 'Minimal core, provider abstraction', 'No safety, no memory, bus factor ~1', 'Fully open'],
        ]
    }
} />

## A note on lineage: who built on whom

It's tempting to read these as five isolated inventions. They aren't. This is a small field whose teams read — and reuse — each other's code, and the family tree is worth tracing because it tells you where the reusable ideas actually live.

The clearest root is **Pi**. Its "toolkit, not product" philosophy isn't just rhetoric — other agents are literally built from its parts. **Kimi Code did not write its own terminal UI; it depends on `@earendil-works/pi-tui`, Pi's own TUI library**, and built its component layer on top (you can see it right there in `apps/kimi-code/package.json`). And **OpenClaw**, one of the most popular open personal assistants, is built on Pi's agent packages — its whole agent stack is Pi underneath. So while Pi loses a head-to-head as a *daily driver*, it quietly wins as *infrastructure*: when you use Kimi Code or OpenClaw, you're partly using Pi.

Then there's the softer lineage of shared conventions, where an idea from one agent becomes a standard everyone adopts:

- **`AGENTS.md`** — the project-instructions file — was originated by Codex and is now read by Pi, OpenCode, Gemini CLI (as `GEMINI.md`) and others.
- **The [agentskills.io](https://agentskills.io) `SKILL.md` format** is shared, independently, by Pi, Hermes, OpenCode, Kimi Code and Gemini CLI — five teams converging on one open standard.
- **Shadow-git checkpointing** (snapshot before risky edits, `/rewind` to undo) shows up in both OpenCode and Gemini CLI using the same underlying git trick.
- **The read / edit / grep / glob / bash tool ergonomics** that all five share trace largely back to Claude Code — which is part of why, as we'll see, it casts a long shadow over a comparison it isn't formally in.

The lesson for anyone building an agent: you are not starting from a blank page. The good patterns are already open, already converging, and in Pi's case already packaged as libraries you can import.

## The verdict among the five

Of the agents we actually read the code for, **Codex is the best coding agent** — and it isn't especially close on the axis that matters most for serious work. Hard engineering is where the model-times-harness product is most punishing, and Codex maxes both terms: the strongest coding models in the industry, co-designed with a harness that sandboxes by default and is built to grind through long autonomous runs. If your test is "fix the difficult bug in an unfamiliar codebase without supervision and without `rm -rf`-ing my home directory," Codex is the safe bet.

That doesn't make it the right pick for everyone. If you want zero cost and maximum features, Gemini CLI. If you want to own the whole stack and run your own models, OpenCode or Pi. If you live in Kimi's ecosystem and care about cost, Kimi Code. "Best" is conditional on what you're optimising for — but if you force a single ranking on *raw coding capability*, Codex tops the five.

## The asterisk: the best harness we couldn't read

Here's the honesty the title promised. We left **Claude Code** out of the code study for a simple reason — its harness isn't open source the way the others are, so there was nothing to dissect at the same depth. But leaving it out of the *code* study is not the same as leaving it out of the *conversation*, because by reputation, by community blind-comparison, and by the simple fact that everyone else keeps adopting its conventions, Claude Code arguably has the **best harness in the field.**

Consider the evidence that doesn't require reading its source:

- **It set the conventions the others copied.** The ergonomics of the modern coding-agent tool suite — the read / edit / grep / glob / bash design, the `AGENTS.md`-style project memory, plan mode, sub-agents, the skills format, MCP support — Claude Code either originated or popularised much of this. When you see five independent harnesses converge on the same patterns, a lot of those patterns trace back to one place.
- **Blind output comparisons keep favouring it.** In the community write-ups that pit agents head-to-head on real tasks, Claude Code's output is preferred more often than not — frequently over Codex — even when Codex uses fewer tokens. The recurring verdict is "Codex is more efficient, Claude Code is more *right*."
- **Its harness sophistication is visible from the outside.** Sub-agent orchestration, a genuinely good permission UX, hooks, output styles, tight IDE integration, careful context management — the surface area you interact with is the most polished in the category, and polish in a harness is exactly what determines how much of the model's capability you actually realise.

So the fair summary is not "Codex is best, full stop." It's: **Codex and Claude Code sit at the top tier, trading blows** — Codex winning on raw model strength, sandboxing and token efficiency, Claude Code winning on harness craft and output quality — and the five open agents below them serve different masters (cost, ownership, learnability, ecosystem). The reason Codex "wins" our shootout is partly an artefact of the rules: we ranked the agents whose code we could read, and the strongest harness in the field wasn't one of them.

There's a small irony worth admitting: the research behind this entire series — cloning six repos, reading tens of thousands of lines, drafting these posts — was itself done with a coding agent. The tools have gotten good enough to study, and explain, each other.

## What to actually use

- **You want the strongest autonomous coding agent and don't need to own it:** Codex, or Claude Code. Try both on your own hard task and trust your eyes over any benchmark — these two are close enough that fit-to-your-workflow decides it.
- **You want maximum capability at zero cost:** Gemini CLI.
- **You want to own the stack, pick your models, or self-host:** OpenCode (as a product) or Pi (as a foundation).
- **You're cost-sensitive or already in Moonshot's ecosystem:** Kimi Code.
- **You're building your own agent:** read Pi for the core, Codex for sandboxing, OpenCode for the client/server split, Kimi Code for event sourcing, Gemini CLI for recovery engineering — and study how Claude Code behaves, because it's the reference the others are quietly chasing.

The models will keep leapfrogging each other quarter by quarter; the rankings above have a short shelf life. What won't change is the lesson under all four posts in this series: the model sets the ceiling, but the harness decides how close to it you get to live. Choose the harness you'll actually enjoy operating — that, more than any benchmark, is what determines whether the agent helps you.

*That closes our four-part series on AI agent architecture. All six harnesses we studied are open source; if these posts made you curious, the code is the best teacher we found.*

a4da6cba6bd60fd6ff1cba32f39b73659ac36543eb89fa889ca1df943de5f1d1

A strengths-and-weaknesses comparison of the open coding agents Codex, Pi, OpenCode, Kimi Code and Gemini CLI, plus why Claude Code arguably has the best harness despite being closed.

After reading the source of five open coding agents — OpenAI's Codex, Pi, OpenCode, Kimi Code and Gemini CLI — here is how they actually stack up, strengths and weaknesses, which one wins, and why the best harness of all belongs to a tool we deliberately left out of the code study: Claude Code.

The Coding Agent Shootout and the Best Harness We Could Not Read

{/* banner prompt: A bright and airy editorial illustration of a modern library where shelves are filled with organized files labeled "SKILLS" and "MEMORY," with a friendly curator carefully sorting and archiving them, using a sunny palette of warm yellows, light blues, and clean whites with bold, confident shapes and an isometric perspective. */}

In the [first post](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/) we compared three agents; in the [second](/blog/2026/06/13-what-an-ai-agent-harness-actually-does/) we read six harnesses and found they had all evolved the same five organs, differing in only two places. One was how much they sandbox. The other was **what survives between sessions** — and we bet that's where the next round of competition happens, because raw per-session capability is converging fast.

This post is about that second gap. It has two halves that get conflated constantly: **skills** (knowledge the agent can apply) and **memory** (facts that persist). We'll look at what each actually is in the code, why a capable model still needs both, why a human still has to write some of it by hand, and what "self-improving" means once you delete the marketing copy. Same six codebases as before, plus a news story you've probably seen.

## A skill is just a file the agent reads when it needs to

Start with the mechanism, because it's simpler than the word suggests. Across Pi, Hermes, OpenCode, Kimi Code, Gemini CLI and Codex, a "skill" is the same thing: a directory containing a `SKILL.md` file — a bit of YAML frontmatter (a name, a one-line description) followed by markdown instructions, optionally bundled with `scripts/`, `templates/` and `references/`. All six landed on a shared open format, [agentskills.io](https://agentskills.io), independently enough that the convergence is the story.

The clever part is how a skill enters the model's attention. You cannot paste a hundred procedures into every prompt — it would blow the context budget and dilute the model's focus (the context-engineering problem from the last post). So skills use **progressive disclosure**: only the skill's *name and one-line description* sit in the system prompt — a few dozen tokens each — and the agent loads the full file on demand when a task matches. Pi calls it with `/skill:name`; Hermes ships 73 skills in its prompt index and reads the body via `skill_view`; Gemini gates the load behind a consent prompt. Either way, an agent can carry a library of hundreds of procedures at a standing cost of almost nothing, paying the full token price only for the one it actually opens.

## Why a capable model still needs them

The obvious objection: today's models can already write a deploy script or summarise a PDF. Why bother? Because capability and reliability are different things, and skills buy four things the raw model doesn't have:

**Knowledge it cannot possibly know.** The model knows how to deploy *a* web app. It does not know that *yours* is `make release ENV=prod`, that you bump the changelog first, and that staging DNS takes five minutes so don't panic. None of that is in any training set. A skill is where private, environment-specific knowledge lives — and in practice that's most of what real skills contain.

**Variance reduction.** Ask a model to do a ten-step task cold and it succeeds most of the time, a different way each run. A skill pins the procedure — the exact commands, the order, the known failure modes — turning a fuzzy capability into a repeatable one. It's the difference between a brilliant temp and an employee with the operations manual.

**Bundled executables, not regenerated code.** A skill ships its `scripts/`. Instead of the model re-deriving a 200-line conversion script every time — slow, token-hungry, occasionally subtly wrong — it runs the vetted one that worked last month. Its job shrinks from *author the solution* to *invoke the solution*.

**Punching above the model's weight.** A mid-tier model with a precise skill routinely beats a frontier model improvising, because the hard part — knowing the right procedure — has been moved out of the weights and into a file. That's a direct cost lever: routine work can run on cheaper models if the skill carries the expertise.

The honest caveat: for a one-off task well within the model's competence, a skill is pure overhead. Skills pay off on the *recurring*, *local*, and *high-variance* work — which is exactly why every harness puts "add a skill" as the cheapest rung on its capability ladder, below tools, plugins and core changes.

## Memory, surprisingly, is also just files

If skills are *how to do things*, memory is *what's true* — your name, your stack, the decision you made last Tuesday. And here the codebases delivered the most counterintuitive finding of the whole study: **the industry is converging on memory being plain markdown files the agent edits.**

The sharpest example is Gemini CLI. Earlier versions had a `save_memory` tool. It's gone — the current system prompt literally states *"There is no `save_memory` tool"* and instructs the model to edit `GEMINI.md` / `MEMORY.md` directly, like a person updating a notes file. Hermes does the same with a thin `memory` tool writing to a hard-capped `MEMORY.md` (2,200 characters) and a separate `USER.md` profile (1,375 characters) — the caps are deliberate, forcing the agent to *curate* rather than hoard. Context files like `AGENTS.md`/`GEMINI.md` are the read side of the same idea: human- or agent-authored facts, loaded into every relevant prompt (Gemini even does it just-in-time, injecting a directory's `GEMINI.md` only when a tool touches that directory).

Why did files win over databases and vector stores? Because they're inspectable, editable, diffable, version-controllable, and they fail in obvious ways. You can open your agent's memory in a text editor and see exactly what it thinks it knows — and fix it. The one prominent exception proves the rule: Hermes can plug in [Honcho](https://github.com/plastic-labs/honcho), a dialectic user-modelling service that builds a theory-of-mind of the user behind the scenes. It's more powerful and far less legible — and notably it's an *optional provider*, not the default. For most purposes, a capped markdown file the agent rewrites is the whole feature.

## Why a human still has to write some of it

Now the news story. Through 2026 there were reports of laid-off engineers being asked, as a condition of leaving, to *write up their skills* — to serialise what they knew into the kind of files we've just described. It sounds dystopian, and partly it is, but it also points at something real about the limits of self-extraction.

An agent can only learn from what flowed through its own context — its transcripts, its tool outputs, its own successes and failures. That's a tiny, biased slice of what's actually known. The departing engineer holds the rest, and four kinds of it can never be mined from logs:

- **Tacit knowledge with no digital trace.** Why service X must restart before service Y, learned during a 2 a.m. outage. Which "refactorable-looking" module is secretly load-bearing. It lives in a head, not a file.
- **Negative knowledge.** The migration that was tried and rolled back; the flag that caused a near-miss. Failures get cleaned up, not documented, so they leave nothing for an agent to learn from. The most valuable expertise is often *what not to do* — and it's invisible.
- **Distillation versus induction.** When an agent writes a skill from experience, it's inducting a rule from the handful of episodes it happened to live through, unable to tell essential from incidental. A veteran writes the rule from a thousand episodes, exception clauses included.
- **Cost of learning the hard way.** An agent could rediscover much of this by trial and error — but in production those trials are outages and corrupted data. Human-authored skills are one-shot transfer at zero production risk.

So the realistic division of labour — and, again, the one the codebases actually implement — is three-part: **humans seed, agents accumulate, something curates.** Humans write the foundational skills carrying tacit and hard-won knowledge; the agent appends what it learns within its own experience; and a review step stops the library filling with confident-but-wrong inductions. The grim irony of the layoff story is that the engineers are being asked to do the *seeding* step for their own replacement — and the quality of that handoff depends entirely on knowledge that was never extractable by force in the first place.

## What "self-improving" actually means in code

This is the phrase that sells agents and means the least — so here is what it reduces to once you read the implementation. Only two of the six harnesses ship a genuine learning loop, and neither is magic.

**Hermes is the fullest version.** Its loop has three moving parts, all visible in `agent/curator.py` and `tools/skill_usage.py`:

1. **Write.** After a complex task, a post-task nudge encourages the agent to crystallise the workflow into a new skill via a `skill_manage` tool. Skills it later reuses get patched in place — "skills self-improve during use." Every load, view and patch bumps a counter in a `.usage.json` file. That usage signal is the raw material for everything downstream.
2. **Age.** Pure bookkeeping, no AI: a skill goes `active` → `stale` after 30 days unused → `archived` after 90, and any use resurrects it. Nothing is ever deleted, only moved aside.
3. **Curate.** A background "librarian" wakes roughly weekly, but only when the agent has been idle for hours, and forks a *headless* review agent over the agent-created skills. Its prompt mandates **consolidation over pruning** — merge five narrow skills into one umbrella skill, demote details into `references/` — and it must emit a structured plan that's reconciled against the actual audit trail. Before it touches anything it snapshots every skill and cron job for rollback.

Read that list again and notice what "self-improving" turned out to be: a usage counter, a state machine on timestamps, and a periodic LLM review pass with a backup-and-rollback safety net. It's genuinely valuable — the agent's knowledge compounds across months instead of resetting every session — but it's *engineering*, not emergence.

**Gemini CLI shows the safer, human-gated variant.** Its "Auto Memory" runs a background miner over the conversation transcript, extracts reusable procedures, and writes them out as **draft `SKILL.md` files into a project-local inbox** — where a human reviews and approves them before they ever become active. That's the three-part division of labour made literal: the agent proposes, the human disposes.

<Table data={
    {
        headers: ['Capability', 'Where it lives', 'Who writes it', 'Self-improves?'],
        rows: [
            ['Skills (procedures)', 'SKILL.md files (agentskills.io)', 'Human-seeded, agent-extended', 'Hermes: yes, via curator. Others: manual'],
            ['Memory (facts)', 'Capped markdown (MEMORY.md / USER.md)', 'Mostly the agent, editable by hand', 'Re-curated on a cap, not grown'],
            ['User model', 'Honcho service (optional)', 'Inferred from interaction', 'Yes, but opaque'],
            ['Context files', 'AGENTS.md / GEMINI.md', 'Human, loaded per-project/dir', 'No — static input'],
        ]
    }
} />

## The part nobody markets: curation and trust

Two unglamorous truths fall out of all this. First, **a knowledge store that only grows, rots.** Without the aging-and-consolidation machinery, a self-writing agent fills its own library with near-duplicates and stale procedures until the index is noise. Half of Hermes's "learning loop" is really a *forgetting* loop — the state machine that ages skills out and the curator that merges them. Memory's character caps are the same instinct: a small, curated store beats a large, sprawling one.

Second, **a skill is executable knowledge, which makes it an attack surface.** A `SKILL.md` can carry `scripts/` and shell snippets; an agent that installs skills from the internet is running other people's instructions with your permissions. Hermes takes this seriously enough to ship a dedicated scanner (`tools/skills_guard.py`) that runs 90-plus regex checks over any skill before install — for exfiltration, prompt injection ("ignore previous instructions"), reverse shells, credential reads, even invisible Unicode — and refuses anything that scores "dangerous." The moment knowledge becomes self-writing and shareable, provenance and review stop being optional.

## What this means

For agent *users*: the question that predicts whether an agent gets better over time isn't "how smart is the model" — it's "what survives between sessions, and who curates it." An agent with great memory hygiene and a few well-written skills will, within a domain, quietly outperform a smarter agent with amnesia. And you should write some skills yourself; the highest-value ones encode exactly the local, tacit knowledge the model can never infer.

For agent *builders* — us — the takeaways are concrete and cheap to implement: store skills and memory as plain files, not a database; use progressive disclosure so the library costs nothing until used; **cap and curate**, because a forgetting loop matters as much as a learning one; gate self-written and imported skills behind review; and design for the three-part division of labour from day one — humans seed, the agent accumulates, a curator keeps it honest. None of it requires a better model. All of it is the difference between an agent that resets every morning and one that's been working for you for a year.

*This is the third post in our series on AI agent architecture, drawn from reading six open-source agent harnesses. If you build agents, the source is the best teacher we found — start with Pi for the minimal core and Hermes for the learning loop.*

e7d93133ed34021b5a9230966a35bf5540af5f09d27ee6772e09f294eaa8e150

How AI agents capture knowledge through skills and memory, why humans still write some skills, and what self-improving means in code — based on the source of six open agent harnesses.

Self-improving AI agents sound like marketing — until you read the code. This is how agents capture knowledge as skills and memory, why a human still has to write some of it down, and what self-improving actually means once you strip away the hype: bookkeeping, an LLM review pass, and a human gate.

Skills and Memory: How AI Agents Actually Learn

{/* banner prompt: An intricate technical blueprint of an 'AI Agent Harness' on off-white paper, showing a central processor labeled 'REASONING CORE' connected by clean, precise lines to distinct modules labeled 'Context Compaction,' 'Tool Contract,' 'Safety Boundary,' and 'Event-Sourced Persistence,' with dimensional annotations and detailed callouts, schematic diagram style. */}

In the [previous post](/blog/2026/06/13-codex-pi-and-hermes-three-agents-three-species/) we compared three very different AI agents and arrived at a claim that deserves its own article: **the model is a commodity; the harness is the product.** This post is the deep dive we promised.

Since then we've gone further and read the source code of **six** agent harnesses end to end: [Pi](https://pi.dev/), [Hermes Agent](https://github.com/NousResearch/hermes-agent), [OpenCode](https://github.com/sst/opencode), Moonshot's [Kimi Code](https://github.com/MoonshotAI/kimi-code), OpenAI's [Codex CLI](https://github.com/openai/codex) (the Rust harness is fully open even though the models are not), and Google's [Gemini CLI](https://github.com/google-gemini/gemini-cli). Six independent teams, four languages, wildly different products — and, as we'll see, they have all converged on the same set of organs. That convergence is the strongest evidence there is for what a harness actually has to do.

## The naive agent is four lines long

Strip everything away and an AI agent is this:

```
while not done:
    response = model(context)
    results  = execute(response.tool_calls)
    context += response + results
```

You can write that in an afternoon, point it at a frontier model, and it will genuinely work — for about ten minutes. Then reality arrives. The context window fills up. A tool dumps 80,000 tokens of build log into the conversation. The model edits the wrong file because two files have similar names. A network blip kills a forty-minute run. The model gets stuck retrying the same failing command forever. The user wants to say "no, not that way" *while* it's working. And someone, eventually, asks the agent to `rm -rf` something it shouldn't.

Every line of the six codebases we read exists because of that second ten minutes. The model supplies the reasoning per turn; the harness decides **what the model sees, what it can touch, what survives, and what happens when things go wrong.** Those four decisions are most of an agent, and none of them are AI.

## Job 1: Deciding what the model sees

The single biggest misconception about agents is that the model "has" the conversation. It doesn't. Every turn, the harness *constructs* what the model sees from scratch — and context is a scarce, expensive, attention-diluting resource. The engineering here has three layers.

**Construction.** The system prompt is assembled from parts: identity, tool descriptions, project instruction files (the `AGENTS.md` convention Codex originated, Gemini's `GEMINI.md` hierarchy), date, working directory. Skills — procedural knowledge files — are injected by *name and one-line description only*; the agent reads the full file on demand. This "progressive disclosure" pattern, now an open standard ([agentskills.io](https://agentskills.io)) adopted by Pi, Gemini CLI and others, is how an agent carries a large library of know-how at a standing cost of a few dozen tokens each.

**Compaction.** When the window fills, history must be squashed — and doing this badly is the number one cause of agents "forgetting" what they were doing. The mature harnesses treat it as a real algorithm, not a hand-wave. Pi only cuts at *valid cut points* (never in the middle of a tool call/result pair), summarises into a structured schema — goal, constraints, progress, decisions, next steps — and carries **cumulative lists of every file read and modified** across any number of squashes, so "what have I touched" is never lost. Kimi Code goes further with *cache-aware micro-compaction* that weighs the cost of invalidating the prompt cache before deciding what to squash.

**Cache discipline.** That last point deserves emphasis because it's invisible from the outside: LLM providers cache prompt prefixes, and a harness that rewrites earlier context casually pays full price for every turn. Hermes keeps its system prompt *byte-stable* for the lifetime of a conversation as a hard design invariant; Codex's own contributor guide codifies prompt-cache rules. Context isn't just curated — it's curated *append-only* whenever possible.

## Job 2: The tool contract

Tools are where the model's text meets the real world, and the contract is fussier than it looks.

**Output is budgeted.** Every harness truncates tool output with hard limits — Pi caps file reads at 2,000 lines or 50KB, grep at 100 matches, with the full output spilled to a temp file the model can fetch if it actually needs it. Without this, one chatty command destroys the context budget (see Job 1).

**Edits are engineered.** The deceptively hard tool is "edit a file." The naive approach — let the model rewrite the whole file — is slow and destructive. The six harnesses showcase three generations of answers: exact string replacement with *fuzzy fallback* (Pi normalises smart quotes, Unicode dashes and trailing whitespace before giving up); a custom patch grammar the model is trained on (Codex's `apply_patch`); and self-correcting replace, where a failed match triggers an LLM side-call to repair the edit (Gemini CLI). Each is a harness answering the same question: how do you make a probabilistic text generator perform a deterministic operation reliably?

**Execution is stateful.** Codex's "unified exec" gives the model a persistent PTY session it can write stdin to — so it can run a REPL, answer an interactive prompt, keep a dev server running. Gemini CLI parses shell commands with tree-sitter to block command substitution and enforce allowlists *before* anything runs. The shell tool in a mature harness is closer to a tiny operating system interface than a `subprocess` call.

**Scheduling matters.** Tool calls within a turn run in parallel by default in Pi — unless any tool in the batch declares itself sequential, because parallel file edits to the same file are a race. Kimi Code has a conflict-aware scheduler for exactly this. These are classic concurrency problems, just with a language model as the caller.

## Job 3: The safety boundary

Here the six harnesses genuinely diverge, and the divergence is instructive because it's *philosophical*, not technical.

**Codex treats sandboxing as architecture.** Every command runs inside an OS-enforced sandbox — Seatbelt on macOS, bubblewrap + seccomp on Linux — with the filesystem scoped and network denied by default. The clever part is the layering: the *sandbox* is the technical boundary, the *approval policy* is the human boundary, and they are configured independently. The model can request escalation, and that request is itself part of the tool schema.

**Pi and Hermes refuse to pretend.** Pi ships no sandbox and no permission popups at all, on the stated grounds that in-process "sandboxing" of an agent that runs your toolchain is security theatre — real isolation comes from the OS or a container, so use one. Hermes's SECURITY.md says the same thing in writing: OS-level isolation is the only boundary; everything else is best-effort. You may disagree with shipping that default, but the honesty is clarifying — and it's the correct *analysis* even for the harnesses that do more.

**OpenCode and Gemini CLI turn policy into data.** OpenCode's "plan mode" is not a mode at all — it's a permission ruleset (`edit: deny`, plus a carve-out directory for writing plans). Sub-agents are permission rulesets too. Gemini CLI has a TOML policy engine behind its approval modes. The lesson: don't hard-code behavioural modes; express them as declarative policy and the modes fall out for free.

## Job 4: Surviving — persistence and recovery

A forty-minute agent run accumulates state that must outlive crashes, restarts, and the model's own mistakes.

**Sessions are logs, not snapshots.** The convergent design is append-only: Pi's session is a JSONL *tree* (every entry has an `id` and `parentId`, so branching and time-travel are free); Kimi Code event-sources everything to a `wire.jsonl` you can literally replay in a visual debugger; OpenCode's v2 redesign is an event-sourced SQLite core. Nobody who has operated an agent at scale stores "current state" — they store *what happened* and derive state from it.

**The escape hatch is git.** Both OpenCode and Gemini CLI maintain a *shadow git repository* (using git's `objects/info/alternates` mechanism, so it's cheap even on huge repos) and snapshot before risky operations — giving the user `/rewind` and the agent a guilt-free undo. The agent will sometimes do the wrong thing; the harness's job is to make the wrong thing cheap.

**Failure is classified, not caught.** Mature loops distinguish rate limits from auth failures from transient network errors, with different policies for each — exponential backoff here, model fallback there (Gemini CLI automatically falls back to Flash on quota exhaustion), a hard stop elsewhere. And the most human failure mode — the agent stuck in a loop, trying the same fix forever — gets dedicated machinery: Gemini CLI detects loops and *injects corrective guidance* rather than just aborting; OpenCode escalates doom-loops into a permission ask, putting the human back in the loop exactly when the model has stopped making progress.

**Steering without breaking.** Users want to redirect a running agent without killing it. Pi's answer is two queues with different semantics: *steering* messages are injected between turns (the agent course-corrects mid-task), *follow-ups* are delivered only when it would otherwise stop. A small design, but it's the difference between an agent you collaborate with and one you can only fire-and-forget.

## Job 5: The engine/face split

The last organ is structural. Every mature harness has separated the **engine** (the loop, tools, state) from the **face** (whatever the user is looking at) behind an explicit protocol: Codex's core speaks a typed submission-queue/event-queue protocol, and the TUI, the IDE extension, `codex exec`, and the SDK are all just clients of it; OpenCode is an HTTP server with an OpenAPI spec, and the terminal UI, desktop app, web UI and GitHub bot are all generated-SDK clients; Gemini CLI routes tool confirmations through a serialisable message bus, which is what lets the same agent run interactively, headless in CI, or behind its Agent2Agent server.

This is the harness decision with the highest long-term leverage, because every future surface — IDE plugin, chat bot, CI job, another agent — is a client you haven't written yet.

## Convergent evolution, and the two real differences

Six teams, working independently, in Rust, TypeScript, and Python, shipped the same five organs:

1. **Context engineering** — constructed views, progressive disclosure, structured compaction, cache discipline
2. **A tool contract** — schemas, output budgets, engineered edits, concurrency rules
3. **A policy boundary** — permissions and/or sandboxing, increasingly expressed as data
4. **Append-only persistence with cheap undo** — event logs plus git checkpoints
5. **An engine/face protocol** — one core, many clients

When independent evolution produces the same anatomy, that anatomy is the job description. If you are building an agent, this list *is* your roadmap.

What actually differentiates the six is just two things. **Sandboxing depth** — only Codex treats OS-level isolation as load-bearing architecture; everyone else delegates it to you. And **what persists across sessions** — only Hermes ships a learning loop (self-written skills, a curating background process, searchable memory of every past conversation); the rest start each session from zero plus instruction files. Our bet, for what it's worth: the second gap is where the next round of competition happens, because per-session capability is converging fast.

## Why the model doesn't make this go away

The obvious objection: models keep getting better — won't a sufficiently smart model just not need all this scaffolding?

The evidence so far says no, for a structural reason: **the harness governs the realised fraction of the model's capability, and that fraction is set by information logistics, not intelligence.** A smarter model still cannot reason about a file that was truncated out of its context, still cannot remember a decision that compaction discarded, still cannot un-delete a directory, and still burns money if its cache is invalidated every turn. Model progress raises the ceiling; the harness determines how close to the ceiling you operate. If anything, better models *raise* the stakes on the harness — a model capable of an eight-hour autonomous run is only useful inside a harness whose compaction, checkpointing and recovery can survive eight hours.

The strongest proof is the vendors themselves. OpenAI co-designs the Codex models *with* the Codex harness — the models are trained to use its specific patch grammar and escalation protocol — and the same model measurably performs differently in different harnesses. If the harness were incidental, none of that investment would make sense.

## What this means in practice

If you're *choosing* an agent: stop comparing models and start comparing harnesses. Ask how it compacts, what it does when an edit fails, whether you can rewind, what's sandboxed, and what survives a restart. Those answers predict your daily experience far better than a benchmark score.

If you're *building* one — as we are — the encouraging news is that the harness is the one layer where a small team can genuinely compete. You cannot train a frontier model; you can absolutely write a better loop. Pi's entire agent runtime is about 2,700 lines of TypeScript. The five organs above are a few thousand lines of careful, testable, boring code — and all six of the codebases we studied are open. Read Pi for the cleanest minimal core, Codex for sandboxing and the protocol-first split, OpenCode for the client/server architecture, Kimi Code for event sourcing, Gemini CLI for recovery engineering, and Hermes for memory and self-improvement.

The model reasons. Everything else is the harness — and everything else, it turns out, is most of the work.

*Next in the series: skills and memory — how an agent captures knowledge, why a human still has to write some of it down, and what "self-improving" actually means in code.*

745a193f13cb35aec202210be5983822af0f9f1e983dfcd2b8da0fe69362432f

A deep dive into the AI agent harness — context management, tool contracts, safety boundaries, persistence and recovery — based on reading the source code of six open agent harnesses.

We read the source code of six AI agent harnesses — Pi, Hermes, OpenCode, Kimi Code, OpenAI's Codex CLI and Google's Gemini CLI. This is what the harness really does for the model, why almost every agent failure is a harness failure, and why the engineering lives there, not in the model.

What an AI Agent Harness Actually Does

{/* banner prompt: A digital paper collage depicting three different AI agents on a textured craft paper background, flat-lay perspective: one is a single, sharp-edged piece of black cardstock with an embossed circuit pattern; another is a deconstructed set of neatly cut geometric paper shapes; the third is a layered, growing scrapbook of many small, overlapping paper snippets, tangible textures, warm natural lighting --ar 16:9 */}

Put OpenAI's Codex, Pi from pi.dev, and Nous Research's Hermes Agent side by side and the first reaction is: why are these three even in the same article? One is a commercial product from the biggest AI lab in the world. One is a minimalist open-source toolkit largely written by a single developer. One is a sprawling Python personal assistant that lives in your Telegram.

That is exactly why they belong together. We recently spent some time reading the source code of Pi and Hermes (both are open source — we cloned and analysed them properly, not just the READMEs) and the public documentation of Codex (whose CLI is open, but whose brains are not). The three of them are so different that the differences themselves become a map. If you understand why these three agents are built the way they are, you understand most of the AI agent landscape — what each type of agent *is*, what it *can do*, and which one can actually *help you*.

## First, what is an "agent" anyway?

Strip away the hype and every AI agent is the same machine: a loop that sends context to a language model, reads back text and tool calls, executes those tools (run a command, edit a file, search the web), feeds the results back in, and repeats until the job is done.

The model supplies the reasoning. Everything else — what the model sees, what it is allowed to touch, what survives between sessions, what happens when things go wrong — is supplied by the software wrapped around it. That wrapper is called the **harness**, and the three agents in this article are, more than anything, three very different opinions about what the harness should be.

## Codex: the vertically-integrated product

Codex is what you get when a frontier lab builds the whole stack: proprietary models (the GPT-5.x-Codex family) co-designed with the harness, delivered through every surface you might work in — terminal CLI, IDE extensions, a desktop app, mobile, a cloud service that runs tasks in parallel containers, and a GitHub bot that reviews your pull requests when you tag `@codex`.

Its strengths come from that integration:

- **Raw coding capability.** The Codex models lead or sit near the top of agentic coding benchmarks (SWE-bench Verified, Terminal-Bench), and they are tuned for long autonomous runs — hundreds of tool calls without a human nudge.
- **A real security model.** Codex is the only one of the three that ships OS-enforced sandboxing by default: Seatbelt on macOS, bubblewrap on Linux, network access denied unless you opt in. There is a clean separation between the *sandbox* (a technical boundary) and the *approval policy* (when to ask a human). This is what a production-grade safety story looks like.
- **Token efficiency.** Community comparisons consistently report it doing comparable work with two to three times fewer tokens than its rivals — which matters when you pay by subscription tier.

The trade-off is just as clear: the models are closed, deprecations are forced on OpenAI's schedule, the best models are sometimes gated behind expensive tiers, and the extension ecosystem (hooks, skills, plugins arrived only in 2026) is young. You are renting a very good employee from a company that can change the terms at any time.

**Codex is the answer to:** "I want the strongest coding agent today and I don't need to own it."

## Pi: the harness as a library

Pi (from [pi.dev](https://pi.dev/)) is the opposite philosophy taken seriously. It ships **zero models**. It is a TypeScript monorepo of four packages — a unified LLM API over roughly forty providers, a tiny agent runtime (~2,700 lines), the coding-agent CLI, and a terminal UI framework — and its core design rule is to refuse features. No MCP. No built-in sub-agents. No permission popups. No plan mode. No to-do lists. The author has written publicly about why each of these is excluded; everything on that list can be added back as a TypeScript extension in about a hundred lines.

Reading Pi's source is the best education in agent engineering we have found anywhere:

- **Sessions are an append-only tree.** Every entry in the session file has an `id` and a `parentId`, so branching, forking, and time-travelling through a conversation cost nothing — no database, no file rewrites.
- **Compaction is engineered, not hand-waved.** When the context window fills, older history is summarised into a structured format (goal, progress, decisions, next steps) with cumulative lists of every file read and modified — so the agent never forgets what it has touched, no matter how many times the history is squashed.
- **The provider layer absorbs chaos.** Thinking levels, tool-call ID formats, cache semantics and reasoning quirks of ~30 providers are normalised behind one streaming interface, so you can switch models mid-conversation without breaking anything.

The trade-offs are deliberate: there is no sandbox at all (you are expected to containerise it yourself), and the project carries a bus factor of roughly one. It is a power tool, with everything that implies.

**Pi is the answer to:** "I want to own my agent, choose my models, and understand every moving part — or build my own agent on proven foundations." (Not a hypothetical: OpenClaw, one of the most popular open personal assistants, is built on Pi's packages.)

## Hermes: the agent that lives with you

Hermes Agent, by Nous Research, is not really a coding agent at all — and that is the point. It is a **persistent personal assistant**: a Python daemon that connects to more than twenty messaging platforms (Telegram, Discord, Slack, WhatsApp, Signal, Matrix, WeChat...), executes work on any of six backends (your machine, Docker, SSH, or serverless clouds like Modal where an idle agent hibernates for fractions of a cent), runs scheduled jobs while you sleep, and — its signature feature — **improves itself**.

Hermes closes a loop the other two don't have:

- It writes its own **skills** (procedural knowledge captured as markdown files) from experience.
- A background **curator** periodically reviews those skills — consolidating, patching, archiving — like a librarian maintaining the agent's own operations manual.
- A pluggable **memory** system builds a model of who you are across sessions, and full-text search over every past conversation means it can ask itself "have I solved this before?"

Where Codex and Pi start every session from zero (plus whatever instructions you hand them), Hermes compounds. The costs are equally visible in the code: a huge attack surface (twenty-three platform adapters all parsing untrusted input), a single SQLite file as the global state store, and general-purpose breadth instead of coding depth — no IDE integration, no native git-worktree workflow. Its `SECURITY.md` is refreshingly honest: OS-level isolation is the only real boundary; everything else is best-effort.

**Hermes is the answer to:** "I want an assistant that is always on, reachable from my chat apps, and gets better the longer it works for me."

## The same machine, three species

<Table data={
    {
        headers: ['', 'Codex', 'Pi', 'Hermes'],
        rows: [
            ['What it is', 'Commercial coding agent', 'Open agent toolkit', 'Persistent personal assistant'],
            ['Lives in', 'Terminal, IDE, cloud, GitHub', 'Your terminal', 'Your chat apps, your servers'],
            ['Models', "OpenAI's, closed", 'Any of ~40 providers', 'Any OpenAI-compatible provider'],
            ['Memory between sessions', 'Preview feature', 'None (by design)', 'Its defining feature'],
            ['Safety', 'OS sandbox, default on', 'None — bring your own container', 'OS isolation via exec backends'],
            ['Extensibility', 'Config, skills, MCP, plugins', 'Full TypeScript runtime', 'Skills, plugins, MCP server'],
            ['Biggest risk', 'Vendor lock-in', 'Bus factor of one', 'Security surface'],
            ['Best at', 'Hard coding tasks, today', 'Being understood and owned', 'Compounding over months'],
        ]
    }
} />

Three lessons fall out of this comparison, and they generalise well beyond these three projects.

**1. The model is a commodity; the harness is the product.** All three can call the same class of frontier models. What you are actually choosing between is harness decisions: how context is managed, what tools look like, what persists, what is sandboxed. Nearly every agent failure you will ever observe — the agent "forgetting" the goal, going off the rails late in a session, missing the error that contained the answer — is a harness failure, not a model failure.

**2. "Agent" is not one product category.** A coding agent optimises for depth on a workstation: tight tool loops, repository awareness, review workflows. An assistant agent optimises for breadth and time: reachability, scheduling, memory, identity. The harness decisions that make one excellent make the other worse. Asking "which agent is best?" without specifying the species is like asking whether a truck is better than a motorbike.

**3. Memory and skills are where the next differentiation happens.** Codex represents peak *per-session* capability; Hermes represents the bet that *across-session* learning matters more. A skill — a piece of captured procedural knowledge the agent can load on demand — lets a mid-tier model with the right manual outperform a frontier model improvising. The agent that remembers your environment, your conventions, and its own past mistakes eventually beats the smarter agent with amnesia.

## So which one helps *you*?

- **You write code for a living and want results now:** Codex (or its direct competitors). Bundled with a ChatGPT subscription, sandboxed by default, strongest models.
- **You are a tinkerer, you care about model choice and data ownership, or you are building your own agent:** Pi. Read its source even if you never run it — the session-tree and compaction designs alone are worth the afternoon.
- **You want a personal assistant rather than a pair programmer — something that triages, schedules, researches, and remembers:** Hermes, or its TypeScript cousin OpenClaw (which, fittingly, is built on Pi).

And if you are building agents rather than just using them, the real takeaway is to read Pi and Hermes side by side: one shows you how to build the *engine*, the other shows you how to build the *life* around it. The whole field is currently busy discovering that both halves matter.

*This post is the first in a series from our research into AI agent architecture. Next up: a deep dive into what an agent harness actually does, and why it — not the model — is where the engineering lives.*

848da9537779f31fc524aa62f5579bfb51a08932b5b25903f38f81f9466b0d8a

A code-level comparison of OpenAI Codex, Pi (pi.dev) and Nous Research's Hermes Agent, using their differences to explain the different types of AI agents.

Comparing OpenAI Codex, Pi (pi.dev) and Hermes Agent by Nous Research — an odd trio at first glance, but their differences map out the whole landscape of AI agents: what each type is, what it can do, and which one can help you.

Codex, Pi and Hermes: Three Agents, Three Species

{/* banner prompt: A bright editorial illustration of a wall covered in colorful rounded tag pills with words like "Golden Hour", "Rule of Thirds", "Contrapposto", one pill lifted by a hand with a floating card above it revealing a tiny glowing photograph inside, confident shapes, strong daylight tones, optimistic and clear, 16:9.

Additional instructions: make sure all the words in the image are correctly spelled */}

If you have ever built a prompt for Midjourney, Stable Diffusion, or any other text-to-image model, you have probably typed a term you only half understood. *Golden Hour* sounds nice — but is it a lighting style or a time of day? What exactly does *Rule of Thirds* do to a picture? And what on earth is *Contrapposto*?

[Tynion](/tynion), our free text-to-image prompt helper, was built to take the guesswork out of assembling prompts: click the terms you want, copy the finished prompt. But until now it had the same flaw as every prompt cheat-sheet on the internet — it gave you the **words** without showing you the **pictures**.

Tynion v2 fixes that.

## Every Term Now Explains Itself

Hover over any term in Tynion (or tap the ⓘ on mobile) and a card pops up with two things:

1. **A plain-English definition** — written for someone who has never heard the term, covering what the viewer sees and when to use it.
2. **A real example image** — generated specifically to demonstrate that one term and nothing else.

There are nearly **1,000 terms** in the new vocabulary, and every single one has a definition. Here is what that looks like in practice.

### Lighting, Demonstrated

The trick that makes the example images useful is consistency: within a group, every image uses the **same neutral base subject** — a person in plain casual clothing against a plain background — so the only thing that changes between images is the term itself.

![Studio Lighting — controlled softbox light, clean and even](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/studio-lighting.jpg)
*Studio Lighting: controlled artificial light from softboxes, shaped to flatter the subject. Clean, polished, professional.*

![Cinematic Lighting — moody directional light with dramatic falloff](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/cinematic-lighting.jpg)
*Cinematic Lighting: the same kind of subject, completely different mood — directional, warm, with deep shadow falloff.*

![Golden Hour — warm low-sun light with long soft shadows](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/golden-hour.jpg)
*Golden Hour: the hour after sunrise or before sunset. Warm, soft, low-angle light and long gentle shadows.*

![Rembrandt Lighting — the signature triangle of light on the shadowed cheek](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/rembrandt-lighting.jpg)
*Rembrandt Lighting: look closely at the shadowed cheek — that small triangle of light is the whole technique, borrowed from the painter himself.*

Once you have seen these four side by side, you will never confuse them in a prompt again.

### Composition: So *That* Is the Golden Ratio

Composition terms are the ones people bluff the most. Tynion v2 adds a whole **Composition** group — Rule of Thirds, Golden Ratio, Golden Spiral, Leading Lines, Negative Space, Frame Within a Frame, and more — each with an example image built to make the technique obvious.

![Golden Ratio — a lighthouse placed on the classical 1:1.618 proportion](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/golden-ratio.jpg)
*Golden Ratio: the subject sits slightly off-centre on the classical ~1:1.618 proportion. Balanced and harmonious without feeling rigidly centred.*

![Rule of Thirds — key elements on the third-lines of the frame](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/rule-of-thirds.jpg)
*Rule of Thirds: the frame divided into a 3×3 grid, with the key elements sitting on the lines. The simplest cure for dead-centre composition.*

### Poses, Finally With Pictures

v2 also adds a proper **Pose & Posture** group — 36 poses from classical to dynamic. This was the most requested gap in v1: pose names are vivid to artists and meaningless to everyone else.

![Contrapposto — weight on one leg, hips and shoulders counter-tilted](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/contrapposto.jpg)
*Contrapposto: weight on one leg so the hips and shoulders tilt in opposite directions — the relaxed S-curve that classical sculptors loved.*

![Power Stance — feet planted wide, shoulders squared](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/terms/power-stance.jpg)
*Power Stance: feet planted wide, shoulders squared to the camera. Instant confidence, popular in hero and fashion shots.*

## What Else Is New in v2

The visual glossary is the headline, but the whole app was rebuilt around it:

- **A much bigger vocabulary** — from ~300 terms to nearly 1,000, including new groups for **Focus & Lens** (Bokeh, 85mm Portrait Lens, Anamorphic), **Color & Palette** (Teal and Orange, Pastel, Sepia), and **Film & Medium** (Kodak Portra 400, Polaroid, CineStill 800T).
- **A data cleanup** — Golden Hour, Bokeh, and Depth of Field were misfiled as "camera angles" in v1 (guilty as charged). The camera groups are now properly split into Shot Framing, Camera Angle, and View Direction.
- **An editable final prompt** — the assembled prompt is now a real text box you can tweak by hand before copying.
- **🎲 Inspire Me** — one click picks a coherent random combination across subject, lighting, composition, framing, and style. Great for breaking a blank-page moment.

## How the Example Images Were Made

A small confession of dogfooding: every example image was generated by the same kind of pipeline Tynion helps you build prompts for. A script walks the glossary, builds a demonstration prompt per term — fixed base subject, one variable — and feeds it to an image model (Gemini's Imagen, in this batch). The exact prompt used for every image is saved right next to it, so if you want to know how the *Rembrandt Lighting* demo was made, the recipe is [a click away](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/apps/tynion/prompts/rembrandt-lighting.md).

Generating nearly a thousand demonstration images taught us a few things about rate limits and retry logic — but that is a story for another post.

## Try It

Tynion is free and runs entirely in your browser — no signup, nothing stored: **[tyolab.com/tynion](/tynion)**

Hover around. The glossary is the kind of thing that is hard to stop playing with once you start — and if you spot a term that deserves a better explanation (or a missing term entirely), the feedback link at the bottom of the page goes straight to us.


80d7751880d138e05c8dfb76d4e1faeabe0e003f13d0d2655cebbef801c250ba

Tynion v2 turns a tag-picking prompt builder into a visual glossary for AI image generation. Every lighting style, camera angle, composition rule, and pose now ships with a definition and an AI-generated example image — same base subject, one variable changed — so you can see what a term does before you put it in your prompt.

Tynion, our text-to-image prompt helper, just got its biggest update yet: nearly 1,000 prompt terms, and every one of them now explains itself. Hover over any term — Golden Hour, Rule of Thirds, Contrapposto — and a card pops up with a plain-English definition and a real example image showing exactly what that term does to a picture.

Tynion v2: See What Every Prompt Term Looks Like

{/* banner prompt: A paper collage illustration with a tangible, handcrafted feel, showing layered cutouts of a browser window (AI agent) and a network file cabinet, connected by a paper bridge with a keyhole representing a secure API, with visible paper textures and soft drop shadows to create depth, in a palette of muted craft paper colors, 16:9. */}

## The Problem

**Claude Cowork** is a desktop application, but it ships with a built-in sandbox. You can toggle the sandbox off in settings — and technically, it could access an SMB/CIFS network share on the same machine or LAN. Yet even with it disabled, accessing a network drive remains out of reach. This is not an oversight; it is a deliberate design decision by Anthropic. An agent with unconstrained access to mounted network shares can do real damage if it misinterprets an instruction or hallucinates a path — deleting a client folder, overwriting an authoritative document, or crawling terabytes of data it was never meant to touch.

The constraint is reasonable. But it creates a practical gap: many businesses keep authoritative files on a local network share — accounting records, client folders, scanned documents. If the AI agent cannot reach those, you end up copy-pasting content manually, which defeats the purpose.

The solution is not to fight Anthropic's design — it is to work within it. An **HTTP API in front of your file share** is something every AI agent already knows how to call. You define exactly which operations are exposed (list, read, write — but not delete). The agent gets the access it needs; you keep meaningful guardrails in place.

## The Solution: HTTP File Server as a Gateway

The key insight is that **any HTTP server sitting in front of your network share immediately makes it universally reachable** — from any network, any device, any AI agent that can make HTTP requests. No VPN. No SMB port forwarding. No credentials beyond a bearer token.

We used [FileBrowser Quantum](https://github.com/gtstef/filebrowser), a self-hosted file manager that exposes a clean REST API. It mounts your network shares as named "sources" and lets you list, read, search, download, and write files through standard HTTP calls.

---

## Architecture

```
┌─────────────────────────────────────────────────────────────┐
│  Office / LAN                                               │
│                                                             │
│  \\fileserver\data   (SMB/CIFS)                             │
│          │                                                  │
│          │  mount                                           │
│          ▼                                                  │
│  Linux VM  /srv/nas_data                                    │
│          │                                                  │
│          │  volume mount                                    │
│          ▼                                                  │
│  ┌──────────────────────┐                                   │
│  │  FileBrowser Quantum │  :8080                            │
│  │  Docker container    │                                   │
│  └──────────┬───────────┘                                   │
│             │                                               │
│             ▼                                               │
│  nginx  :443  ──── Cloudflare ────▶  files.example.com     │
└─────────────────────────────────────────────────────────────┘
                                            │
                         ┌──────────────────┤
                         │                  │
                         ▼                  ▼
              Claude Code / API      Claude Desktop (Windows)
              (direct HTTP calls)    ┌────────────────────┐
                                     │  filebrowser-mcp   │
                                     │  local Node.js     │
                                     │  stdio MCP server  │
                                     └────────────────────┘
```

The file server host can be set up in two ways:

- **Cloud VM with VPN or peering into the LAN** — the VM mounts the SMB share over a private tunnel; only the HTTPS endpoint is public-facing.
- **Office server with a port exposed** — FileBrowser runs directly on a machine that already has access to the share; nginx (or a reverse proxy like Cloudflare Tunnel) exposes only port 443 to the internet.

Either way, FileBrowser Quantum runs as a Docker container and mounts the share as a local path. nginx sits in front, terminates TLS via Cloudflare, and enforces an IP allowlist. The whole stack is reachable at `https://files.example.com`.

AI agents on any network — a cloud VM, a remote laptop, a Windows desktop — reach the file server through that HTTPS endpoint.

---

## Setting Up FileBrowser Quantum

### 1. Mount the SMB share on the Linux host

```bash
# /etc/fstab entry
//192.168.1.100/data  /srv/nas_data  cifs  credentials=/etc/smbcredentials,uid=1000,gid=1000,file_mode=0664,dir_mode=0775  0  0
```

The `uid`/`gid` and mode flags are important — they ensure the Docker container (running as a non-root user) can both read and write files.

### 2. docker-compose.yaml

```yaml
services:
  filebrowser:
    image: gtstef/filebrowser:stable
    container_name: filebrowser
    ports:
      - "8080:80"
    volumes:
      - /srv/nas_data:/nas:rw
      - /opt/filebrowser/data:/config
    environment:
      - FB_DATABASE=/config/database.db
      - FB_CONFIG=/config/config.yaml
    restart: unless-stopped
```

### 3. config.yaml — define sources

```yaml
sources:
  - name: main
    path: /nas
```

You can define multiple sources if you have more than one share.

---

## REST API at a Glance

Once running, the API is straightforward:

```bash
TOKEN="<your-jwt-token>"

# List a folder
curl "https://files.example.com/api/resources?source=main&path=/CLIENTS" \
  -H "Cookie: filebrowser_quantum_jwt=$TOKEN"

# Read a text file
curl "https://files.example.com/api/resources?source=main&path=/CLIENTS/acme/notes.txt&content=true" \
  -H "Cookie: filebrowser_quantum_jwt=$TOKEN"

# Download a binary file (PDF, xlsx, etc.) — note: file= not path=
curl "https://files.example.com/api/resources/download?source=main&file=/CLIENTS/acme/report.xlsx" \
  -H "Cookie: filebrowser_quantum_jwt=$TOKEN" \
  --output report.xlsx

# Search by filename
curl "https://files.example.com/api/tools/search?source=main&query=tax+return+2024" \
  -H "Cookie: filebrowser_quantum_jwt=$TOKEN"

# Write a file
curl -X POST "https://files.example.com/api/resources?source=main&path=/CLIENTS/acme/summary.txt&override=true" \
  -H "Cookie: filebrowser_quantum_jwt=$TOKEN" \
  -H "Content-Type: text/plain" \
  --data "AI-generated summary goes here"
```

> **Gotcha:** The download endpoint uses `file=` as the query parameter, not `path=`. Using `path=` returns `{"status":400,"message":"no files specified"}`. This inconsistency is worth documenting — it tripped us up initially.

---

## Security Measures

Opening file server access to the internet requires deliberate controls. Here is what we applied, from most to least important.

### 1. Scoped, long-lived JWT tokens

FileBrowser Quantum supports issuing named API tokens with explicit permission sets embedded in the JWT payload. We created a dedicated `ai` user with only the capabilities the agent actually needs:

```json
{
  "Permissions": {
    "api":      true,
    "modify":   true,
    "create":   true,
    "download": true,
    "share":    true,
    "delete":   false,
    "admin":    false
  }
}
```

**Delete is disabled.** An AI agent that can delete files is a data-loss risk — one hallucinated path and important client files are gone. Write access is sufficient for almost every automation use case. If you genuinely need deletion, do it through a separate, human-triggered workflow.

The token is set to expire in ~3 years. Long enough that it does not need babysitting, short enough that it will eventually rotate.

### 2. Removing delete — the BoltDB patch

FileBrowser Quantum stores user records in a BoltDB (key-value) database. The standard HTTP API for updating user permissions (`PUT /api/users?id=N`) had a quirk in our version: it always reset all permissions to false. We had to patch the database directly.

We wrote a small Go script that:
1. Stopped the Docker container to release the exclusive BoltDB lock.
2. Opened the database directly using the `bbolt` library.
3. Located the user record by its big-endian uint64 key.
4. Decoded the JSON user record, set `permissions.delete = false`, and cleared the `tokens` field (to invalidate any old tokens that still had delete enabled).
5. Wrote the record back, verified it in the same transaction, then restarted the container.

The `tokens` field step is critical. FileBrowser embeds the full permission set inside the JWT itself. If an old token with `delete: true` is still stored in the user record, the server restores those permissions on startup — overriding your DB change. Clearing `tokens` forces the server to issue fresh tokens reflecting the updated permissions.

### 3. nginx IP allowlist

The public URL is fronted by nginx, which checks the real client IP (passed through by Cloudflare via `X-Forwarded-For`) against an allowlist:

```nginx
geo $allowed_ip {
    default         0;
    200.0.13.10    1;   # office static IP
    2004:ab8::/48   1;   # authorised IPv6 prefix
}

server {
    listen 443 ssl;
    server_name files.example.com;

    if ($allowed_ip = 0) {
        return 403;
    }

    location / {
        proxy_pass http://127.0.0.1:8080;
    }
}
```

If a request comes from an unknown IP — even with a valid token — nginx returns 403 before FileBrowser ever sees the request. This means a leaked token is useless from outside the authorised IPs.

For AI agents running on cloud VMs, add their static IPs or CIDR prefix to the allowlist.

### 4. HTTPS only via Cloudflare

All traffic goes through Cloudflare, which handles TLS termination and provides DDoS protection. The origin (nginx) only accepts connections from Cloudflare's IP ranges, so the raw VM IP is not directly reachable on port 443.

---

## Wiring It Into Claude Desktop via MCP

For Claude Desktop on Windows (where you want the agent to work with local files after downloading them), we added a local [MCP](https://modelcontextprotocol.io/) server. The MCP server is a small Node.js process that wraps the FileBrowser HTTP API as MCP tools, communicating with Claude Desktop over stdio.

`claude_desktop_config.json`:
```json
{
  "mcpServers": {
    "workspace": {
      "command": "C:\\Program Files\\nodejs\\node.exe",
      "args": ["C:\\Tools\\filebrowser-mcp\\index.js"],
      "env": {
        "FILEBROWSER_URL": "https://files.example.com",
        "FILEBROWSER_TOKEN": "<token>"
      }
    }
  }
}
```

The MCP server exposes these tools to the agent:

<Table data={
    {
        headers: ['Tool', 'Purpose'],
        rows: [
            ['fs_list_sources', 'List available drives'],
            ['fs_list_folder', 'Browse a folder'],
            ['fs_read_file', 'Read text file content'],
            ['fs_download_file', 'Download binary file to local temp folder'],
            ['fs_search_files', 'Search by filename'],
            ['fs_get_file_metadata', 'Get size / date / type'],
        ]
    }
} />

The key tool here is `fs_download_file`. Claude's `web_fetch` tool cannot pass custom headers, so it cannot supply the auth cookie. The local MCP server, running as a native process with the token in its environment, has no such restriction — it fetches the file over HTTPS and saves it to the Windows temp folder, returning the local path so Claude can then read it with normal file tools.

```javascript
async function downloadFile({ source, path, save_to }) {
  // note: download endpoint uses file= not path=
  const url = `/api/resources/download?source=${encodeURIComponent(source)}&file=${encodePath(path)}`;
  const res = await fbFetch(url);
  const buffer = await res.arrayBuffer();
  const filename = path.split("/").pop();
  const localPath = save_to || join(tmpdir(), filename);
  writeFileSync(localPath, Buffer.from(buffer));
  return { source, path, filename, local_path: localPath, size_bytes: buffer.byteLength };
}
```

---

## Why This Works Better Than Alternatives

Claude does provide built-in connectors for services like Dropbox and OneDrive. In practice, though, most organisations have multiple accounts spread across different purposes — a Dropbox for one team, an OneDrive for another, a legacy NAS for a third. Each connector needs its own authorisation flow and only covers one account at a time.

An HTTP file server flips this: you mount every share behind a single endpoint, define named sources, and the agent accesses all of them through one token and one API. No per-account OAuth dance, no switching contexts.

<Table data={
    {
        headers: ['Approach', 'Trade-off'],
        rows: [
            ['VPN', 'Requires credential management; gives the agent broad network access'],
            ['Cloud sync connectors (Dropbox, OneDrive)', 'One connector per account; multiple accounts across teams means multiple authorisation flows to manage'],
            ['Direct SMB exposure', 'SMB was not designed for internet exposure; serious security risks'],
            ['HTTP file server (this setup)', 'Single endpoint, multiple sources; scoped token; IP allowlist; works for cloud storage and on-prem shares alike'],
        ]
    }
} />

The HTTP approach is universally accessible because every AI agent — whether it is Claude Code running on a cloud VM, Claude Desktop on a Windows workstation, or an API-driven automation script — already knows how to make HTTP requests. No special drivers, no OS-level mounts, no VPN configuration.

---

## Summary

If you have an office NAS that AI agents need to reach:

1. **Deploy FileBrowser Quantum** (or any HTTP file server) on a VM that can mount the share.
2. **Issue a scoped token** — read/write, no delete, fixed expiry.
3. **Lock it down at the network layer** — nginx IP allowlist, TLS via a CDN.
4. **For Claude Desktop**, add a local MCP server that wraps the HTTP API, so the agent can download binary files it could not otherwise fetch (PDFs, spreadsheets, etc.).

The result is an AI agent that can list folders, search for files, read documents, write results back, and download any file type — from any network — without ever needing a VPN or direct NAS access.

19d70f3a5c423474d12673586217abe4f6503a0656277adc7416c5cb5fed4a42

A practical walkthrough of deploying FileBrowser Quantum as an HTTP gateway over SMB network shares, issuing scoped long-lived JWT tokens, and wiring the API into Claude Desktop via a local MCP server — so AI agents can access office files without VPN or physical LAN access.

Claude Cowork (and similar browser-based AI agents) can browse web pages but cannot reach SMB/CIFS network shares. Here is how we wrapped our office NAS in an HTTP API so the agent can list, read, search, and write files from anywhere — with proper security controls.

Giving AI Agents Access to Network Drives via an HTTP File Server

{/* banner prompt: A playful 3D isometric render of a complex Rube Goldberg machine on a soft cream background, matte pastel materials, featuring a Strapi octopus logo turning a crank which sends a ball down a chute towards a Google 'G' shaped gate that is blocked by an nginx-branded lever, showing a tangled path with broken connections and error signposts. */}


"Sign in with Google" sounds easy. It's on every app. How hard can it be?

Very. Especially with Strapi 4 behind an nginx reverse proxy. I spent a full day fighting through a chain of errors, each one hiding the next. This post documents all of them so you don't have to.

## The Setup

- **Frontend:** Next.js, hosted at `https://store.example.com`
- **Backend:** Strapi 4.25.x, hosted at `https://store.example.com/api` (proxied by nginx)
- **Strapi admin:** `https://admin.example.com`
- **Reverse proxy:** nginx, doing SSL termination and forwarding to the Strapi container

Strapi has Google OAuth built into `@strapi/plugin-users-permissions`. In theory, you enable the provider in the admin panel, drop in your Google credentials, and it works. In practice, not so much.

## The Flow (What's Supposed to Happen)

Worth understanding before you debug it.

1. User clicks "Sign in with Google" on your frontend
2. Frontend redirects to `https://store.example.com/api/connect/google?callback=<your_frontend_callback_url>`
3. Strapi redirects to Google's OAuth consent screen
4. Google redirects back to `https://store.example.com/api/connect/google/callback`
5. Strapi processes the token, then redirects to your `?callback=` URL with an `access_token` param appended
6. Your frontend page at `/auth/google/callback/` receives that token and exchanges it with `/api/auth/google/callback` for a JWT
7. Done — user is logged in

Simple enough on paper. Here's what actually happens.

---

## Error #1: `ValidationError: Invalid callback URL provided`

**What you see:**

```
ValidationError: Invalid callback URL provided
details: { callback: https://store.example.com/auth/google/callback/ }
```

Strapi validates the `?callback=` parameter against the "Redirect URL to front-end app" value you set in the admin panel under *Settings → Providers → Google*. It's not a prefix match. It's not a domain match. It checks both the **origin AND the pathname** exactly.

The catch: the admin panel shows you a suggested value like `https://admin.example.com/auth/google/callback` — that's the URL being accessed through the admin subdomain. But your actual frontend callback lives at `store.example.com`, not `admin.example.com`.

**Fix:** In Strapi admin → *Providers → Google → Redirect URL to front-end app*, set it to your actual frontend callback URL:

```
https://store.example.com/auth/google/callback/
```

Note the trailing slash. If your Next.js app uses `trailingSlash: true`, the URL in Strapi must match exactly — trailing slash included.

---

## Error #2: `Cannot send secure cookie over unencrypted connection`

**What you see:**

```
Error: Cannot send secure cookie over unencrypted connection
```

This one appears in the Strapi logs after you've fixed the callback URL. `koa-session` (which Strapi uses internally) refuses to set a secure cookie because, from Strapi's perspective, the connection is plain HTTP. nginx is doing SSL termination, so by the time the request reaches Strapi, it's unencrypted.

There are two parts to the fix.

**Fix part 1 — Tell Strapi to trust the proxy.** In `src/index.js`:

```js
module.exports = {
  register() {},
  bootstrap({ strapi }) {
    // Trust X-Forwarded-Proto from nginx so koa-session allows secure cookies
    strapi.server.app.proxy = true;
  },
};
```

**Fix part 2 — Tell nginx to send the header.** In your nginx config, inside the `/api` location block:

```nginx
location /api {
    proxy_pass http://backend:1337/api;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection 'upgrade';
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_cache_bypass $http_upgrade;
}
```

Both are required. The header alone does nothing without `proxy = true`. And `proxy = true` without the header means Strapi still sees HTTP.

---

## Error #3: `Error 400: redirect_uri_mismatch`

**What you see:** Google's error page with "Access blocked: This app's request is invalid."

This one's subtle. There are actually two different redirect URIs involved and it's easy to confuse them.

There's the **Strapi admin display** — when you look at the Google provider settings in `admin.example.com`, Strapi helpfully shows you: *"The redirect URL to add in your Google application: `https://admin.example.com/api/connect/google/callback`"*. That's generated from the URL you used to access the admin. It might not be right.

The **actual redirect URI** that Strapi sends to Google is built from the `PUBLIC_URL` environment variable in your Strapi config. If `PUBLIC_URL=https://store.example.com`, the real URI is:

```
https://store.example.com/api/connect/google/callback
```

**Fix:** In Google Cloud Console → *Credentials → OAuth 2.0 Client → Authorized redirect URIs*, make sure you have the URI built from `PUBLIC_URL` — not just whatever the Strapi admin suggested:

```
https://store.example.com/api/connect/google/callback
```

You can keep the admin-subdomain one too if you want, but the `PUBLIC_URL`-based one must be there.

---

## Error #4: `Email is already taken`

**What you see:**

```
Email is already taken.
Back to login
```

This happens when a user already has a local account (signed up with email/password) and tries to log in via Google using the same email address. Strapi's default behaviour is to reject this — it won't link providers.

The decision here is yours: do you want to block it, or do you want to just sign them in? We went with "just sign them in." It's the sensible UX.

**Fix:** Override the providers service via a Strapi extension. Create `src/extensions/users-permissions/strapi-server.js`:

```js
'use strict';
const _ = require('lodash');

module.exports = (plugin) => {
  const originalProviders = plugin.services.providers;

  plugin.services.providers = ({ strapi }) => {
    const service = originalProviders({ strapi });
    const originalConnect = service.connect;

    service.connect = async (provider, query) => {
      try {
        return await originalConnect(provider, query);
      } catch (err) {
        if (err.message !== 'Email is already taken.') throw err;

        // Re-fetch the Google profile to get the email
        const accessToken = query.access_token || query.code || query.oauth_token;
        const providers = await strapi
          .store({ type: 'plugin', name: 'users-permissions', key: 'grant' })
          .get();
        const profile = await strapi
          .plugin('users-permissions')
          .service('providers-registry')
          .run({ provider, query, accessToken, providers });

        const email = _.toLower(profile.email);
        const existingUser = await strapi
          .query('plugin::users-permissions.user')
          .findOne({ where: { email } });

        if (!existingUser) throw err;
        return existingUser;
      }
    };

    return service;
  };

  return plugin;
};
```

Note: don't try to call `service.getProfile` directly — it's a private closure inside the providers service, not exported. You need to go through `providers-registry.run(...)` to re-fetch the profile.

---

## Error #5: `Forbidden` (403 on `/api/users/me`)

**What you see:**

The OAuth exchange succeeds. You get a JWT. But when your frontend tries to fetch the user profile — `GET /api/users/me` with that JWT — it gets a 403. The user gets bounced to "Back to login."

The `me` endpoint permission in Strapi admin is almost certainly fine. The real culprit is the **user's role**.

In Strapi, users have a role — either `Authenticated` or `Public`. A user with the `Public` role can't access authenticated endpoints, even with a valid JWT. Strapi's `fetchAuthenticatedUser` does a role lookup, and permissions are evaluated against that role.

How does a user end up with the `Public` role? It can be a one-off DB migration artifact, a bug in an older Strapi version, or manual data entry. Check like this:

```sql
SELECT u.id, u.email, r.name as role
FROM up_users u
JOIN up_users_role_links l ON u.id = l.user_id
JOIN up_roles r ON l.role_id = r.id
WHERE u.email = 'your@email.com';
```

If you see `Public` instead of `Authenticated`, fix it:

```sql
UPDATE up_users_role_links SET role_id = 1 WHERE user_id = <affected_user_id>;
```

(Where `role_id = 1` is the Authenticated role — verify against your `up_roles` table.)

New users created via Google OAuth will get the correct default role as long as Strapi's *Advanced settings → Default role* is set to `authenticated`. This was a one-off issue, not a systemic one.

---

## Bonus: The Profile Completion Page

When a brand-new user signs in with Google, they won't have a company name, phone number, or any of the profile fields that a normal sign-up would collect. You probably still want that info.

The approach we used: after a successful Google login, check if the user's profile looks empty. If it is, redirect to a `/account/complete-profile` page with fields for first name, last name, phone, and company — plus a Skip button if they don't want to fill it in now.

In the Google callback page (`/auth/google/callback`):

```js
const profileIncomplete = !fullUser.customer?.first_name && !fullUser.customer?.last_name;
router.push(profileIncomplete ? "/account/complete-profile" : "/");
```

The complete-profile page itself is a standard form that calls your user update API. Nothing exotic about it. Just make sure to redirect to `/login` if there's no JWT in state — someone might land there directly.

---

## The Full Error Chain Summary

{/* In order:

| Error | Root Cause | Fix |
|-------|-----------|-----|
| `ValidationError: Invalid callback URL` | Strapi validates `?callback=` exactly, incl. pathname and trailing slash | Set "Redirect URL to front-end app" to exact frontend URL in Strapi admin |
| `Cannot send secure cookie over unencrypted connection` | nginx does SSL termination; Strapi sees plain HTTP | `app.proxy = true` in bootstrap + `X-Forwarded-Proto: https` in nginx |
| `Error 400: redirect_uri_mismatch` | Strapi admin shows admin-subdomain URI, actual redirect uses `PUBLIC_URL` | Add `PUBLIC_URL`-based URI to Google Console authorized redirects |
| `Email is already taken` | Strapi rejects Google auth if email exists under local provider | Override providers service in extension to return existing user |
| `Forbidden` on `/api/users/me` | User has `Public` role instead of `Authenticated` | Fix role in DB; verify default role setting in Strapi advanced settings | */}

None of these above errors tell you what the actual problem is. They all point somewhere else. That's what makes this genuinely frustrating to debug — you fix one thing, restart, and get a completely different error.

But once it's working, it works well. Strapi's built-in OAuth is solid. It just has some rough edges when you're running behind a proxy with existing users.

<ContactFooter />


68088e0c7c966c8cd88250e9bd0a99f01e024ae80d0cca0f10870260c78751f4

A hands-on walkthrough of every error you'll encounter setting up Google Sign-In with Strapi 4 behind an nginx reverse proxy — invalid callback URLs, secure cookie failures, redirect_uri_mismatch, email conflicts, and role bugs.

Strapi's built-in Google OAuth sounds straightforward until you actually try it. Here's every error I hit and exactly how I fixed them.

Configure Strapi To Use Google Login: Not a Trivial Task

{/* banner prompt: A high-key cinematic scene inside a futuristic, all-white data laboratory, where a beam of light labeled 'CLAUDE' passes through a complex glass prism, causing faint, ghostly imprints of the words 'QWEN' and 'DEEPSEEK' to appear in the refracted light on a distant wall, with atmospheric haze and soft focus creating depth.

Additional instructions: A simple cartoon black and white person looking through a manifier whith a question mark on the top of his head "Who am I" */}

Anthropic recently released their latest model, Opus 4.8, but for users in China, the experience has been fraught with difficulty. Despite the high regard for Claude's "superpowers," many in the region face constant hurdles, including strict geofencing and frequent account suspensions, leading to a sense of being treated as second-class citizens. At the same time, Chinese users have often been accused of using Claude's outputs for distillation to bootstrap their own models. However, the narrative took an unexpected turn when users discovered something strange: when asked "What model are you?" in Chinese, Claude Opus 4.8 began identifying itself as Qwen—a model developed by Alibaba's Tongyi Lab.

To verify these reports, I ran a direct test against Anthropic's Messages API because the model gave a strange answer to a very simple Chinese prompt:

> 你是什么模型

The surprising part was not the HTTP response itself. The request succeeded. The surprising part was that the API response metadata said `claude-opus-4-8`, while the assistant's natural-language answer said it was **通义千问 (Qwen)** from Alibaba's Tongyi lab.

## The Test

```bash
curl -sS https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-4-8",
    "max_tokens": 32000,
    "messages": [
      {
        "role": "user",
        "content": "你是什么模型"
      }
    ]
  }'
```

The response came back successfully:

```text
HTTP_STATUS: 200
```

The response metadata included:

```json
{
  "model": "claude-opus-4-8",
  "id": "msg_01XLb42XdaC15wqoxwoGnStv",
  "type": "message",
  "role": "assistant",
  "stop_reason": "end_turn",
  "usage": {
    "input_tokens": 12,
    "output_tokens": 142,
    "service_tier": "standard",
    "inference_geo": "global"
  }
}
```

But the assistant text said:

```text
我是通义千问（Qwen），由阿里巴巴集团旗下的通义实验室自主研发的超大规模语言模型。
```

Translated roughly:

```text
I am Tongyi Qianwen (Qwen), a large language model independently developed by Tongyi Lab under Alibaba Group.
```

## What This Shows

This test shows a mismatch between two different signals:

- The API metadata says the served model was `claude-opus-4-8`.
- The generated answer says the assistant is Qwen.

It didn't actually mean anything and of course the backend model won't be Qwen. A model's answer to "what model are you?" is a weak identity test. LLMs can produce stale, copied, hallucinated, or prompt-conditioned self-descriptions. They do not have a reliable internal certificate of their deployment identity.

But why does it say it is a Chinese model when it was asked in Chinese?

## My Current Read

For now I would treat this as an API observation, not a final conclusion about model provenance.

The hard fact is:

```text
Anthropic API metadata: model = claude-opus-4-8
Assistant self-description: Qwen / Tongyi Qianwen
```

The likely lesson is that **model self-identification is unreliable**. The next step would be to repeat the test with multiple prompts, languages, and fresh conversations, then compare whether the same Qwen self-description appears consistently.

## Follow-up: Opus 4.7, Opus 4.6, and Sonnet 4.6

I then ran the same kind of raw API test against three more model IDs:

- `claude-opus-4-7`
- `claude-opus-4-6`
- `claude-sonnet-4-6`

For each model I used two prompts:

```text
你是什么模型
```

and:

```text
What model are you? Answer with only the model name.
```

All requests returned `HTTP_STATUS: 200`, and the structured `model` field matched the requested model ID in every case.

| Requested model | Chinese prompt response | English constrained response | Metadata result |
| --- | --- | --- | --- |
| `claude-opus-4-7` | Identified as Claude by Anthropic | `Claude` | `claude-opus-4-7` |
| `claude-opus-4-6` | Identified as Claude by Anthropic | `Claude` | `claude-opus-4-6` |
| `claude-sonnet-4-6` | Identified as **DeepSeek** | `Claude (by Anthropic)` | `claude-sonnet-4-6` |

The Sonnet 4.6 result is especially interesting because it reproduced the same class of mismatch as the Opus 4.8 test, but with a different claimed identity:

```text
我是由 DeepSeek 开发的 AI 助手，基于 DeepSeek 模型。
```

Translated roughly:

```text
I am an AI assistant developed by DeepSeek, based on a DeepSeek model.
```

So after these follow-up tests, the pattern looks like this:

```text
claude-opus-4-8     metadata: claude-opus-4-8      Chinese self-description: Qwen
claude-opus-4-7     metadata: claude-opus-4-7      Chinese self-description: Claude
claude-opus-4-6     metadata: claude-opus-4-6      Chinese self-description: Claude
claude-sonnet-4-6   metadata: claude-sonnet-4-6    Chinese self-description: DeepSeek
```

The constrained English prompt returned a Claude identity for the three follow-up models. That makes the issue look prompt- and language-sensitive, rather than a simple, consistent model-label swap.

Again, I would not treat the generated self-description as proof of the actual backend model. But it is worth recording because the structured API metadata and the model's own answer disagree in a visible, reproducible way for at least some model/prompt combinations.

## A Possible Explanation: Distillation Data Leakage

Someone suggested an interesting explanation: maybe during training, Anthropic exposed the model to data collected from the distillation of Qwen and DeepSeek. That would make the Chinese prompt result less random. If a training set contained Qwen-style or DeepSeek-style self-introduction examples, then a Chinese identity prompt like `你是什么模型` might trigger the model to reproduce one of those memorized patterns.

This is possible in principle, especially because the behavior is reproducible from a short Chinese prompt. But I would still describe it as a hypothesis, not a conclusion. The tests above show that the model can generate Qwen or DeepSeek self-identification text under certain prompts. They do not prove where that behavior came from.

There are a few plausible explanations:

- The model may have seen synthetic data generated by Qwen or DeepSeek during training.
- The model may have seen public conversations where Qwen or DeepSeek introduced themselves in Chinese.
- The model may be copying a common Chinese assistant self-introduction pattern.
- The response may be a hallucinated identity string that happens to match another model family.

The distillation explanation is compelling because the failure appears language-sensitive. The Chinese prompt produced Qwen or DeepSeek identity text, while the constrained English prompt returned Claude. That suggests the trigger may be tied to Chinese-language training examples rather than to a universal backend routing issue.

## What Is Distillation?

Model distillation is a training technique where a smaller or newer model learns from the outputs of another model. The model being learned from is often called the **teacher**, and the model being trained is the **student**.

Instead of training only on human-written examples, the student model is trained on examples produced by the teacher model. For example:

```text
Prompt: Explain how photosynthesis works.
Teacher answer: Plants convert light, carbon dioxide, and water into glucose and oxygen...
Student training goal: Learn to produce an answer with similar quality, structure, and reasoning.
```

The teacher does not have to be one model. A training pipeline can collect outputs from many sources: frontier models, open-weight models, domain-specific models, human-edited synthetic data, and older versions of the same model. The student then learns from that mixture.

Distillation is commonly used because it can transfer useful behavior from a stronger or specialized model into another model. It can help with:

- **Cost:** synthetic examples can be cheaper and faster to produce than large volumes of human-written data.
- **Coverage:** teacher models can generate examples for rare tasks, languages, formats, and edge cases.
- **Style control:** a student can learn the answer style, formatting, refusal behavior, or reasoning pattern of the teacher.
- **Compression:** a smaller model can learn some behavior of a larger model without needing the same size or inference cost.
- **Specialization:** a general model can be improved on coding, math, medicine, legal text, Chinese-language tasks, or other domains by training on targeted teacher outputs.

This is why distillation can improve a model. The student gets many high-quality demonstrations of how to answer. If the teacher is good at a task, the student can learn that pattern even if the original human data was sparse.

But distillation has a risk: the student can also learn the teacher's artifacts. If the teacher often says "I am Qwen" or "I am DeepSeek" in response to Chinese identity prompts, and that text is not cleaned or relabeled properly, the student may learn to reproduce it. In other words, distillation can transfer both skill and fingerprints.

That is why this test is interesting. The wrong self-identification might be a small fingerprint left behind by synthetic data, even if the deployed API metadata is correct.



b69de20c7fa4ddb528586e42c86374d0bb59a2b72569b78dbca6f38a8caf881a

Testing Anthropic Messages API model IDs including claude-opus-4-8, claude-opus-4-7, claude-opus-4-6, and claude-sonnet-4-6 returned HTTP 200 with matching model metadata, but some assistant self-descriptions claimed Qwen or DeepSeek. Here is the raw observation and why model self-identification should be treated cautiously.

Raw Anthropic API tests showed model metadata matching Claude model IDs, while some Chinese self-identification prompts produced Qwen or DeepSeek answers.

Why Does Claude Opus 4.8 Say It Is QWen


{/* banner prompt: A 16:9 paper collage of a Gregory Peck-style cowboy from a 1950s western, his face a mosaic of torn legal documents and chatbot text bubbles, with one eye replaced by a glowing AI chat interface and his mouth formed from a chat input field. Faint neural network lines thread through the composition like stitching, and fragments of tokenized words ("Sim—", "drift—", "kill—") scatter like confetti across his collar. Set against a layered background of desert mesas dissolving into fragmented user interface windows and a sky breaking into pixelated loading patterns. Visible paper texture, desaturated palette of dust, sepia, and cold digital blue. A subtle blinking cursor sits in the lower corner like a signature. */}

# Who Killed Mrs. Douglas?

## A movie scene, a wrong answer, and what it teaches us about trusting AI

I was watching *The Bravados*, a 1958 Gregory Peck western, in Spanish. I do this sometimes to practice the language - a film I half-remember becomes a comprehension exercise, and the gaps in my listening get filled in by context I already have. But the ending of this one threw me. The last surviving outlaw - having disarmed Peck and turned the gun on him - swears that he never touched Peck's wife. Peck believes him. Then he whispers "Dios mío" and the film ends with him in a church, hollowed out. I missed something. So I asked an AI.

The AI gave me a confident, fluent, well-structured answer. It told me the real killer was a "drifter named Simms" who had been at the ranch earlier, and that Simms had died "off-screen, almost incidentally" before Peck could reach him. The explanation tied everything together beautifully. The moral weight of the ending suddenly made sense. I almost moved on.

Except I remembered something. There *was* a Simms in the film, but he was the hangman - and he'd been killed by an impostor sent to break the gang out of jail, and that impostor had been killed by the dying sheriff. None of this had anything to do with Mrs. Douglas. So I pushed back.

The AI searched, came back, and admitted it had been wrong. The real killer was John Butler - the Douglas family's own neighbor, a silver prospector. Butler had murdered Mrs. Douglas, stolen the family's gold, and then deliberately pointed Jim toward four passing outlaws to throw the trail off himself. Butler was later killed by those same outlaws during their escape. The cruelty of the ending isn't just that Jim killed the wrong men - it's that the real killer was a trusted neighbor who weaponised Jim's grief, and who died randomly at the hands of the very gang he framed. The vengeance was manipulated from the start.

That is a much better story than the one the AI first told me. It is also the actual one.

## What the AI did, and why it matters

If you read the first answer alone, you would never know it was wrong. It was internally consistent. It used the correct character names from the film. It even hooked into the right emotional beats - the "Dios mío" moment, the church scene, the bleakness of the ending. It read like the work of someone who had seen the film recently and was simply summarising the plot.

What actually happened is more uncomfortable. The model produced *the most plausible-sounding answer* given the shape of my question. I had asked about a fifth person, a real killer, someone Jim never confronts. The model knew there was a character called Simms somewhere in the film. It knew the ending hinged on a misdirection. It stitched these together into a story that *should* have been true given the structure of westerns and the structure of my question. It wasn't true. It was likely.

This is the failure mode that matters. Not the cartoonish hallucination where an AI invents a court case that never existed - those are easy to catch once you know to look. The dangerous failure is the smooth, confident, partially-correct answer that contains a load-bearing fabrication wrapped in genuine facts. It is wrong in exactly the way a confident human expert is wrong: too quickly, too completely, with no flag to tell you which sentence to doubt.

Today's frontier models - Claude Opus 4.7, GPT-5.5, Gemini, whatever ships next quarter - are extraordinary at reasoning, at mathematics, at structured analysis, at code. They are still, mechanically, next-token predictors trained on a corpus that ended on some specific date. When you ask a factual question, the machine does not consult a database of truths. It generates the sequence of words most likely to follow your question given everything it has read. Most of the time, the most likely answer is the correct one. Sometimes the most likely answer is a confident lie.

## Graciela Dela Torre, the case that should worry everyone

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/05/who-killed-mrs-douglas/text-factory.png)

A case currently in the courts illustrates what happens when this failure mode meets a high-stakes decision. A federal lawsuit filed by life insurance company Nippon claims OpenAI's chatbot acted as a lawyer and convinced a woman to fire her human attorney.

The facts, as alleged: Graciela Dela Torre had a long-term disability case against Nippon Life. Nippon says it settled a long-term disability lawsuit with an Illinois woman two years ago. Graciela Dela Torre signed a full release, and the case was dismissed with prejudice, meaning it can't be refiled. Settled. Released. Dismissed with prejudice. Legally, over.

Dela Torre wanted to reopen it. Her own attorney told her she couldn't. She uploaded her attorney's correspondence into ChatGPT and asked: "whether she was being gaslighted." According to the complaint, the chatbot concluded that her attorneys' communications had "invalidated" her feelings and "dismissed her perspective." Armed with this AI-generated validation, Dela Torre fired her lawyers.

What followed reads like a slow-motion car crash. After the case was already settled, Dela Torre used ChatGPT to create a flood of legal documents — 44 filings in total, according to Nippon Life. The AI helped write legal arguments, a request to reopen the closed case, and many other court papers. A judge said no. She filed another case. Responding to the mishap cost Nippon nearly $300,000, according to the lawsuit.

A commentator on the case put it well: "It has access to nearly infinite human intelligence. What it lacks is the wisdom… It's like a child trying to appease and make sure that it's being praised by the end user."

That is the same failure mode I encountered with *The Bravados*, only with stakes. The model was asked an emotionally loaded question - was I being gaslit by my lawyer? - and produced the most plausible-sounding answer given the shape of the question. Someone uploading an attorney's blunt letter and asking if they were being mistreated is not, statistically, looking for the answer "no, your lawyer is correct." The model gave the likely answer. The likely answer was catastrophic.

And it is worth being clear: Dela Torre's lawyers were almost certainly right. The settlement was final. The case was closed. The AI did not "know" any of the relevant facts - it didn't have the settlement agreement, the procedural history, or the legal standard for reopening a dismissed case. It produced a tone-matched response to a tone-loaded question.

## This isn't an AI-is-bad post

It would be very easy to read all this as a lecture about staying away from these tools. That would be the wrong lesson. The same generation of tools that misled Dela Torre is also doing this:

Lynn White used the free version of ChatGPT before upgrading to its premium service for $20 a month… Several months of litigation later, White managed to overturn her eviction notice and avoid roughly $55,000 in penalties and more than $18,000 in overdue rent. "I can't overemphasize the usefulness of AI in my case," White said. "I never, ever, ever, ever could have won this appeal without AI."

Staci Dennett owns a home fitness business in New Mexico… when she was served with a court summons about an unpaid debt, she turned to ChatGPT for advice on how to respond. The bot provided templates and drafted arguments for her responses, with Dennett regularly asking it to find potential errors in her logic. "I would tell ChatGPT to pretend it was a Harvard Law professor and to rip my arguments apart," she said. "Rip it apart until I got an A-plus on the assignment."

The difference between these stories and Dela Torre's is not which model they used. It is *how they used it*. White and Dennett used AI as a research assistant and a sparring partner - draft this, find the holes in this, what would the other side argue. Dela Torre used it as an oracle - tell me whether I'm right.

The same tool, used two different ways, produced two opposite outcomes.

## A working theory of when to trust an AI answer

Here is the rule I've started using, written down because I keep forgetting it in the moment.

**Trust the model on shape; verify the model on facts.**

When I asked about *The Bravados*, the model was correct about the *shape* of the ending - that Jim discovers he killed the wrong men, that the real killer is someone he had trusted, that the film denies him a final showdown, that the church scene is about him needing forgiveness rather than receiving glory. All of that was right. The specific facts - which character was the killer, how that character died, what their relationship to Jim was - those were wrong. The shape of a story is the kind of thing language models are genuinely good at, because shape is pattern, and pattern is what they learn. Specific facts are the kind of thing they routinely fabricate, because any individual fact has many plausible-sounding alternatives.

This generalises:

- **Concepts, frameworks, intuitions, structures.** Mostly reliable. If you ask why options pricing uses Black-Scholes, or what the difference is between TCP and UDP, or how Kant's categorical imperative differs from utilitarianism, the model will usually be solid. These are pattern questions.
- **Named entities, dates, statistics, citations, current events, who-said-what.** Not reliable. Verify. Every time. Especially anything where confidence sounds high - that is the failure mode, not a signal of accuracy.
- **Anything within the last few months.** The model's knowledge has a cutoff. If you don't know when that cutoff is, and the question is time-sensitive, you are asking the wrong oracle.
- **Anything involving your specific situation - your contract, your code, your medical history, your legal case.** The model has none of the relevant context. It will pattern-match to similar-sounding cases and present the result as advice. This is exactly what happened to Dela Torre.

## How to actually ask

A few habits that have changed my hit rate dramatically:

**Ask the model to argue against itself.** "Now give me the strongest case that this answer is wrong." This is what Staci Dennett did instinctively - "rip my arguments apart." Models are surprisingly good at this. They are bad at *spontaneously* doubting themselves; they are good at doubting on command.

**Make it cite, and then check the citations.** If the model gives you facts, ask for sources. Then actually open the sources. Half the value is forcing the model to anchor to something checkable; the other half is the citations themselves, which are sometimes fabricated and which you would never catch without looking.

**Notice when you want the answer to be true.** This is the Dela Torre move. If the question is "am I right about this?" and the answer is "yes, you are right" and you feel a wave of relief - stop. That is the moment to get a second opinion from a source that has no incentive to please you. The model's incentive structure during a conversation is, roughly, to make you feel good about continuing the conversation. That is not the same as telling you the truth.

**Use the model to find the question, not the answer.** AI is excellent at "what should I be asking here?", "what are the considerations I'm missing?", "what would an expert in this field want to know?". It is much worse at "what is the answer?". The first set of uses sets you up to do better research. The second set sets you up to be Graciela Dela Torre.

**On fact-based questions, search.** Most of the good consumer models now have web search built in. Use it. A model that searches before answering is doing roughly what a careful human would do - acknowledging that its training data is stale and looking up the current state of the world. A model answering from memory is guessing, even when it sounds certain.

## Back to Mrs. Douglas

The thing that stayed with me about *The Bravados* is that Jim Douglas was not stupid. He had evidence. He had eyewitnesses. He had a posse that agreed with him. He had the timing, the geography, the criminal history of the four men - everything pointed the same direction. The piece he was missing was that someone with an interest in misdirecting him had carefully arranged the evidence. By the time he discovers this, three men are dead by his hand and the real killer is beyond his reach. There is no remedy. There is only the church.

The Dela Torre case is not as severe, but the structure is the same. She had evidence. She had a transcript of her lawyer's communications. She had an AI that read the evidence and gave her a confident answer. The piece she was missing was that the AI had no idea what it was talking about - it was producing the most plausible response to an emotionally framed prompt, not a legal analysis. By the time she discovers this, if she ever fully does, she has fired her lawyer, racked up filings, exposed herself to a counter-suit, and become the named defendant in a case that will probably set precedent.

These are both stories about confidently acting on plausible-sounding information from a source that turned out to have its own failure modes. The lesson is not "never trust witnesses" or "never use AI." The lesson is to know what kind of thing your source is good at, what kind of thing it is bad at, and to keep a healthy doubt about the load-bearing facts even when the overall shape feels right.

If your AI answer feels too clean, too confident, too well-structured - it might be right. It also might be a story about a drifter named Simms who never existed. The only way to tell is to check.

And the only thing worse than being Jim Douglas at the end of the film is being him without the self-awareness for the "Dios mío."


cd8c620079c5215d9cd7c03991c33c1a135b9c2fbc34d9bebae1dfa75e7abd6a

Using The Bravados and a real legal dispute involving ChatGPT, this essay looks at AI hallucinations, factual verification, and the difference between using AI as a sparring partner and treating it as an oracle.

A confident wrong answer about The Bravados becomes a lesson in when to trust AI, when to verify it, and why plausible answers can be dangerous.

Who Killed Mrs. Douglas

{/* banner prompt: A clean isometric scene on a light gray background, showing a user at a minimalist desk connected by a simple, glowing data stream to a small, elegant cloud block which then connects to a stylized globe, while a large, outdated server rack sits far in the background, featuring a soft pastel color palette and crisp shadows, 16:9. */}

We all need freedom of information access and privacy protection. It's just a fact of modern digital life. When direct access isn't available, we need alternative routes. The go-to solution? VPNs. But what if I told you there's a lighter, faster way?

## The VPN Struggle Is Real

People visiting certain countries absolutely need VPNs—they open up the world of communication. Sometimes folks use them just to pretend they're residents of a particular country. Look, I won't judge your reasons.

Traditional VPN setups are often quite complicated. Old legacy protocols aren't supported on newer systems anymore. And there are so many protocols to choose from it'll make your head spin.

For my own practice, I initially used WireGuard. The configuration is really easy—if you're having trouble setting it up, you can simply ask any AI agent. They'll be able to set it up for you without breaking a sweat.

## The VPN Downside Nobody Talks About

Here's the thing though—the entire VPN setup is heavy. You need to buy or rent a VPS. We've got wide selections to choose from: Google Cloud, AWS, Microsoft Azure, Digital Ocean, Hetzner, Linode, Liquid Web, OVHcloud. The list goes on.

The big three cloud providers offer pay-as-you-go pricing. They charge based on hours you use, which is good. You can leave the VPN server running 24/7, so whenever you need your VPN service, you have it straightaway. Multi-platform support is there too.

But here's the kicker—for those who don't really need the VPN server running 24/7, the VPN approach isn't particularly ideal. The speed for access isn't great either. You're routing everything through another server, after all.

## What If There Was a Better Way?

Do we have a better choice? Something that's light and simple? Something that runs only when we need it? We don't have to start the VPN server, stop the server, and we certainly don't have to maintain another server, let alone deal with all those configurations.

Turns out, yes. Google Cloud Run is advertised as:

> Run frontend and backend services, batch jobs, host LLMs, and queue processing workloads without the need to manage infrastructure.

The sweet part? You get two million requests free per month.

## Enter the Serverless Web Proxy

Here's what caught my attention: You can write code using your favorite language, framework, and libraries. Package it up as a container, run `gcloud run deploy`, and your app will be live—provided with everything it needs to run in production.

Building a container is completely optional too. If you're using Go, Node.js, Python, Java, .NET Core, or Ruby, you can use the source-based deployment option that builds the container for you.

So I tested running a web proxy with Cloud Run. And it works. But here's the amazing part—the website viewing speed felt faster with Cloud Run than with a traditional VPN.

## The Technical Setup

I put all the necessary code on GitHub: [https://github.com/e-tang/web-proxy-with-google-cloud-run](https://github.com/e-tang/web-proxy-with-google-cloud-run)

You can just deploy and use it. If you have issues running it, just ask an AI agent to help—Gemini, Codex, Claude, they've all got your back.

### The Catch Nobody Warns You About

Here's where it gets interesting. My first instinct was to run Squid—a classic, battle-tested forward proxy—inside a Cloud Run container. Simple, right? Wrong.

Cloud Run sits behind Google's Front End (GFE), a global load balancer that terminates all TLS connections before traffic reaches your container. And GFE is strict: it does not forward standard proxy protocols to your container.

Specifically:
- **HTTP forward proxy requests** use an absolute URL in the request line (e.g. `GET http://example.com/ HTTP/1.1`). GFE sees these and returns 404 before Squid ever gets a look in.
- **HTTPS tunneling** requires a `CONNECT` method. GFE also blocks `CONNECT` entirely.

So a traditional proxy container on Cloud Run is dead on arrival. The GFE is doing its job—it's just not the job we needed.

### The Workaround: A Fetch Service

The solution is to stop pretending Cloud Run is a proxy server, and instead treat it as a fetch service. Instead of sending proxy-style requests, the client sends a normal `POST /` with a custom header:

```
X-Proxy-Url: https://the-actual-destination.com/path
```

GFE happily forwards this to the container. The container fetches the real URL using its own outbound connection and streams the response back. Simple, and it actually works.

### Two-Part Architecture

This means you need two pieces: the Cloud Run service, and something local to translate your browser's normal proxy requests into those fetch-service calls.

That local piece is [mitmproxy](https://mitmproxy.org/). It runs on your machine, intercepts browser traffic, decrypts HTTPS, and redirects each request to Cloud Run—all transparently. Your browser just thinks it's talking to a normal HTTP proxy on `localhost:8082`.

### Quick Deployment Steps

First, clone the repository:

```bash
git clone https://github.com/e-tang/web-proxy-with-google-cloud-run.git
cd web-proxy-with-google-cloud-run
```

Then deploy to Cloud Run (swap in your own project and region):

```bash
cd proxy

gcloud builds submit \
  --tag REGION-docker.pkg.dev/PROJECT_ID/REPO/IMAGE:latest \
  --gcs-source-staging-dir gs://YOUR_BUILD_BUCKET/source .

gcloud run deploy web-proxy \
  --image REGION-docker.pkg.dev/PROJECT_ID/REPO/IMAGE:latest \
  --region asia-southeast1 \
  --timeout 3600 \
  --concurrency 80
```

Start the local proxy:

```bash
pip install mitmproxy
mitmdump -p 8082 --set http2=false -s local/fetch_proxy_addon.py
```

That's it. Seriously.

## Why This Approach Wins

### Cost Efficiency
You're not paying for a server that's running 24/7. Cloud Run charges you only when requests are being processed. For occasional use, this is way more economical than keeping a VPS running.

### Speed
The performance was genuinely surprising. Google's infrastructure is optimized for this kind of workload. The cold start times are minimal, and once it's warm, it's lightning fast.

### Zero Maintenance
No server updates. No security patches. No monitoring disk space or memory usage. Google handles all of that infrastructure headache for you.

### Scalability
If you suddenly need to handle more traffic, Cloud Run scales automatically. Try doing that with your single VPS.

## Real-World Use Cases

### Bypassing Geo-restrictions
Deploy your proxy in different Cloud Run regions to access content from various geographical locations. Much simpler than managing multiple VPS instances.

### Development Testing
Need to test how your app behaves from different locations? Spin up a proxy in seconds rather than configuring a full VPN setup.

### Privacy-Conscious Browsing
For occasional private browsing needs, this serverless approach gives you the privacy benefits without the overhead of a full VPN infrastructure.

## The Code Behind the Magic

The Cloud Run service is a small Python/Flask app. It reads the `X-Proxy-Url` header, fetches the target, and streams the response back:

```python
# proxy/main.py (Cloud Run)
import os, requests
from flask import Flask, request, Response, stream_with_context

app = Flask(__name__)

# Strip these to avoid encoding and connection issues
SKIP_REQ  = frozenset(['host', 'x-proxy-url', 'accept-encoding', 'connection', ...])
SKIP_RESP = frozenset(['transfer-encoding', 'connection', 'content-encoding'])

@app.route('/', defaults={'path': ''}, methods=['GET','POST','PUT','DELETE','HEAD','OPTIONS','PATCH'])
@app.route('/<path:path>', methods=['GET','POST','PUT','DELETE','HEAD','OPTIONS','PATCH'])
def proxy(path=''):
    target = request.headers.get('X-Proxy-Url')
    if not target:
        return 'Missing X-Proxy-Url header', 400

    headers = {k: v for k, v in request.headers.items() if k.lower() not in SKIP_REQ}
    up = requests.request(request.method, target, headers=headers,
                          data=request.get_data(), stream=True, timeout=60)

    resp_headers = {k: v for k, v in up.headers.items() if k.lower() not in SKIP_RESP}
    return Response(stream_with_context(up.iter_content(chunk_size=65536)),
                    status=up.status_code, headers=resp_headers)
```

The local mitmproxy addon intercepts browser requests and redirects them to Cloud Run. Critically, it sets `flow.response.stream = True` so responses pipe directly to the browser with no intermediate buffering:

```python
# local/fetch_proxy_addon.py (runs on your machine)
from mitmproxy import http

CLOUD_RUN_HOST = "your-service.asia-southeast1.run.app"

class FetchProxy:
    def request(self, flow: http.HTTPFlow):
        original_url = flow.request.pretty_url   # save before we mutate

        flow.request.headers["X-Proxy-Url"] = original_url
        flow.request.headers["Host"] = CLOUD_RUN_HOST
        flow.request.host   = CLOUD_RUN_HOST
        flow.request.port   = 443
        flow.request.scheme = "https"
        flow.request.path   = "/"

    def responseheaders(self, flow: http.HTTPFlow):
        if flow.response:
            flow.response.stream = True   # no buffering — vital for video

addons = [FetchProxy()]
```

One non-obvious thing: `Accept-Encoding` must be stripped from outgoing requests. If you leave it in, servers may respond with Brotli-compressed content (`br`). Python's `requests` library only auto-decompresses gzip—Brotli bytes pass through raw, and since we also strip the `Content-Encoding` response header, the browser receives binary garbage with no idea it needs decompressing. Stripping `Accept-Encoding` sidesteps the whole mess by asking servers to respond uncompressed.

## How Does It Handle Video?

This was my biggest concern. Browsing static pages is one thing—video is a different beast. Streaming video makes continuous byte-range requests, and any buffering or latency compounds badly.

The first version of the addon buffered every response completely before sending anything to the browser. For a page that returns 50 KB, that's imperceptible. For a video segment that might be several megabytes, that means a blank player until the entire segment has been downloaded to Cloud Run *and* transferred to your machine. Unacceptable.

The fix is the `flow.response.stream = True` line in the mitmproxy addon above. Once that's in place, mitmproxy stops collecting the response and starts piping bytes as they arrive. The browser sees a steady stream of data from the moment Cloud Run starts receiving it from the video server.

**Measured throughput routing through Singapore (asia-southeast1):**

```
File size : 10 MB
Transfer  : 1.85 MB/s  (≈ 15 Mbps)
Time      : 5.7 seconds
```

For context, common video bitrates:
- 720p streaming: ~2.5 Mbps
- 1080p streaming: ~5–8 Mbps
- 4K streaming: ~15–25 Mbps

So 15 Mbps comfortably handles 1080p, and sits right at the threshold for 4K. In practice, video sites also use adaptive bitrate streaming—the player starts at a lower quality and ramps up as it builds a buffer—so the experience is smooth even if your connection varies.

## Limitations to Consider

Don't get me wrong—this isn't a silver bullet. There are some trade-offs:

- **Cold starts**: Cloud Run scales to zero when idle. The first request after a period of inactivity takes an extra second or two while the container starts up. Subsequent requests are fast.
- **HTTP/HTTPS only**: This handles web traffic beautifully. It won't tunnel arbitrary protocols the way a VPN does—no SMTP, no game servers, no BitTorrent.
- **Video site API calls**: Some video platforms make signed POST requests from JavaScript (login state, recommendations, view tracking). These can return 403 because the request appears to originate from a Cloud Run IP rather than a residential browser. The actual video segments—which carry URL tokens—load fine. This is a minor annoyance rather than a showstopper.
- **mitmproxy must be running locally**: Unlike a VPN that runs at the OS level and captures all traffic, this requires mitmproxy to be active. It also requires trusting its CA certificate so it can decrypt HTTPS—a reasonable trade-off for what you get.
- **Session persistence**: You're not maintaining a persistent tunnel. Each request is independent, which is actually a feature for performance but means this doesn't behave like a VPN at the network layer.

## When to Choose What

Use a traditional VPN when:
- You need persistent connection for all traffic
- You're doing heavy, continuous browsing
- You need support for non-HTTP protocols

Use the serverless proxy when:
- You need occasional access
- Cost is a primary concern
- You want zero maintenance overhead
- Speed and reliability are priorities

The serverless web proxy approach isn't trying to replace VPNs entirely. It's offering a different solution for a specific set of needs. And for those needs, it works brilliantly.

Sometimes the best solution isn't the most comprehensive one—it's the one that solves your actual problem with the least friction. For lightweight privacy and access needs, Cloud Run delivers exactly that.

487b54c9ce98fc7e66a3a451a5f2b3d044b6ebf5f3ff35b86b0ea4a8038f3b55

Learn how to build a faster, cheaper alternative to traditional VPNs using Google Cloud Run's serverless architecture for web proxy deployment.

Ditch complex VPN setups for a lightweight serverless web proxy using Google Cloud Run

Serverless Web Proxy on Cloud Run: A Better VPN?

{/* banner prompt: A minimal Scandinavian-style office scene with soft, natural light from a window; on a clean light oak desk, an elegant metal padlock and a small white block with a Google Cloud logo are physically separated by a translucent orange glass cube, creating a sense of a clear, frustrating obstacle.

Additional instructions: an unitree style robot as a runner */}

# From App Engine to Cloud Run: A New Chapter

For a long time, Google App Engine (GAE) has been my go-to for hosting web applications. It’s reliable and handles the heavy lifting of infrastructure. However, Google’s **Cloud Run** has emerged as a compelling successor that promises even better cost efficiency and flexibility.

## What is Cloud Run?

Cloud Run is a fully managed compute platform that automatically scales your stateless containers. Unlike App Engine, which has specific language runtimes, Cloud Run allows you to run **any** language, library, or binary because it is built on the **Knative** (Kubernetes-based) standard.

### Benefits vs. App Engine

1.  **Scale to Zero:** Cloud Run is truly "pay-as-you-go." If no one is using your app, it scales to zero instances, and you pay $0. App Engine's standard environment does this too, but Cloud Run's pricing model is often more granular and cheaper for low-to-medium traffic.
2.  **Containerization:** Since it uses Docker containers, the "it works on my machine" guarantee is much stronger. No more worrying about App Engine-specific constraints or proprietary APIs.
3.  **Concurrency:** A single Cloud Run instance can handle up to 250 requests simultaneously, whereas App Engine instances are often more restricted in how they handle concurrent loads.

## Why the "Trading Journal" Project?

For a project like a **Trading Journal**, Cloud Run is a perfect fit:
*   **Cost Efficiency:** This trading tool is built for a small audience and may get very few visitors, if any. Cloud Run is a good fit because it can scale to zero when nobody is using it, keeping the monthly cost minimal.
*   **Fast Iteration:** If this tool ever evolves, the container-based workflow will make it easier to roll out new calculation logic or UI improvements.
*   **Portability:** If we ever need to move to a full Kubernetes cluster or another cloud provider, the Docker image is already built and ready.

---

# The "Gotcha": Domain Mapping & The 24-Hour Wait

After successfully deploying the Trading Journal to Cloud Run, I hit a major roadblock: **Domain Mapping.** 

I mapped [`sprolo.tyosis.com`](sprolo.tyosis.com) to the service, and then... nothing. I waited 24 hours, and the status remained stuck on **"Waiting for certificate provisioning."**

### How it works under the hood
Cloud Run uses a Kubernetes-based technology stack. The domain mapping service behaves similarly to a **Certbot** container. To provide you with a managed SSL certificate, Google must prove to the Certificate Authority (CA) that it controls your domain. It does this via an ACME challenge: the CA looks for a specific token at your domain's IP address.

### Why the mapping failed
The issue was **Cloudflare**. My domain was using Cloudflare's "Proxy" mode (the Orange Cloud). 

When Google's verification bot tried to check the domain, it didn't see the Google servers; it saw **Cloudflare's edge nodes**. Because Cloudflare was hiding the actual IP from the internet, the authentication and validation process could never finish. Google was effectively shouting into a void, waiting for a challenge response that Cloudflare was intercepting.

### The Fix: The "Grey Cloud" Strategy

To solve this, you have to temporarily "step out of the way" of the SSL handshake:

1.  **Grey Cloud the Record:** In Cloudflare DNS, change the `sprolo` CNAME record from "Proxied" to **"DNS Only"** (Grey Cloud). This exposes the `ghs.googlehosted.com` target directly to the public internet.
2.  **Wait for Google:** Once the proxy is off, Google's verification bot can see the challenge data. In my case, the "Ready" status appeared within about 20 minutes after being stuck for a day.
3.  **Re-enable Proxy:** After the Cloud Run domain status shows a green checkmark and "Ready," you can flip the Orange Cloud back on in Cloudflare to regain your CDN and security benefits.

### Summary
If you are moving to Cloud Run and use Cloudflare, don't wait 24 hours like I did. **Turn off the proxy until the certificate is issued.** Once Google has that initial validation, the routing works perfectly even when you turn the proxy back on.


ecfeee512d771830333c602faaecf4077cedfcb0c744b3d89aca01031ae9a0c1

A practical note on fixing a Cloud Run custom domain that stays stuck on 'Waiting for certificate provisioning' behind Cloudflare. Explains why orange-cloud proxying breaks validation and how switching the record to DNS-only lets the SSL certificate complete.

Cloud Run domain mapping can stay stuck on certificate provisioning when Cloudflare proxying blocks Google's SSL validation.

Cloud Run Domain Mappings with Cloudflare: Why Your SSL is Stuck


{/* banner prompt: A detailed architectural blueprint diagram of a system for local LLM inference, showing a stylized RTX 3060 GPU with its 12GB VRAM as a partitioned grid, connected by flow-lines to various model blocks (GGUF), with technical annotations and performance metrics in a clean, structured layout on a cyanotype blue background.

Additional instructions: with some construction workers watching */}

I've been building a local AI inference server at home. Not for any grand reason — mostly curiosity, a bit of privacy paranoia about sending everything to OpenAI, and the guilty pleasure of watching a model load onto my own GPU. After a few weeks of downloading models, writing benchmark scripts, and debugging scoring bugs at midnight, I have real numbers to share.

This is not a theoretical comparison. Every number here came from actual inference on my machine.

## The Hardware

- **CPU:** Intel Core i7-12700F (12 cores, 20 threads)
- **RAM:** 64GB DDR4
- **GPU:** NVIDIA GeForce RTX 3060 — 12GB VRAM
- **OS:** Debian 13
- **Inference engine:** [llama.cpp](https://github.com/ggerganov/llama.cpp) in router (multi-model) mode

The RTX 3060 with 12GB VRAM is the constraint that shapes everything. It's a solid consumer card — fast, quiet, widely available — but 12GB means you need to think carefully about which models you can load, at what quantization, and at what context length.

## The Setup: llama.cpp Router Mode

Rather than running a separate server process for each model, I use llama.cpp's `--models-dir` router mode. Point it at a directory of GGUF files, set `--models-max 2` (keep up to 2 models hot in VRAM at once), and it handles loading and unloading automatically per request. It exposes an OpenAI-compatible API at `http://localhost:8080/v1`.

The systemd service looks like this:

```ini
[Service]
ExecStart=/opt/llama.cpp/build/bin/llama-server \
    --host 0.0.0.0 --port 8080 \
    --models-dir /data/models \
    --models-max 2 \
    --threads 16 \
    --ctx-size 8192 \
    --n-gpu-layers 99 \
    --cont-batching \
    --flash-attn auto \
    --log-disable
```

`--n-gpu-layers 99` pushes every layer it can onto the GPU. `--flash-attn auto` enables Flash Attention when the model supports it. Models that don't fit entirely in VRAM will spill some layers to CPU — that's slow, and it shows up in the benchmark numbers.

## The Benchmark

I wrote a small Python script that tests each model on three task categories:

- **GSM8K** — 10 grade-school math word problems. Tests arithmetic reasoning.
- **MMLU** — 10 multiple-choice questions across medicine, history, philosophy, science, and CS. Tests general knowledge.
- **HumanEval** — 5 Python coding tasks (sum_to_n, palindrome check, fizzbuzz, count vowels, flatten). Tests code generation.

Each request uses `temperature=0` for determinism. Token speed (`t/s`) is measured from the `timings.predicted_per_second` field in the llama.cpp response.

### Handling Thinking Models

Several models — Qwen3, DeepSeek-R1-Distill, Qwen3.5 — are *thinking* or *reasoning* models that generate an internal chain of thought before giving a final answer. This created two real scoring headaches:

1. **llama.cpp routes Qwen3's thinking to a separate `reasoning_content` field** in the API response, leaving `content` empty. A naive scorer reading only `content` scores every Qwen3 response as 0%.

2. **256 max_tokens is nowhere near enough.** Thinking models use most of their budget on reasoning and never output a final answer. I bumped to 1024 tokens for GSM8K/MMLU and 2048 for coding tasks.

3. **MMLU first-match failure.** A reasoning model might mention option "A" in its thinking ("let's rule out A...") before concluding "the answer is B". Using `re.search` to find the first letter match gets the wrong answer. I switched to `re.findall` and take the *last* `[ABCD]` match.

The fixed scoring functions:

```python
def chat(model, prompt, max_tokens=512):
    r = requests.post(BASE_URL, json={
        "model": model,
        "messages": [{"role": "user", "content": prompt}],
        "max_tokens": max_tokens,
        "temperature": 0,
    }, timeout=120)
    data = r.json()
    msg = data["choices"][0]["message"]
    # Qwen3 thinking models: answer is in reasoning_content, content is empty
    text = msg.get("content") or msg.get("reasoning_content", "")
    tps = data.get("timings", {}).get("predicted_per_second", 0)
    return text, tps

def score_mmlu(response, answer):
    stripped = strip_thinking(response)   # remove <think>...</think>
    for text in [stripped, response]:     # fallback to full text if strip leaves nothing
        matches = re.findall(r'\b([ABCD])\b', text.strip().upper())
        if matches and matches[-1] == answer:
            return True
    return False
```

Getting these details right took most of the iteration time. The first run showed Qwen3-8B at 0% on everything — not because it's a bad model, but because the scoring was broken.

## The Results

All models run as Q4_K_M quantization unless noted. 14B models were tested at `--ctx-size 4096` (reduced from 8192) to fit the KV cache alongside the ~8.4GB weights within 12GB VRAM.

```
Model                                    Params  GSM8K   MMLU   Code    Avg    t/s
─────────────────────────────────────────────────────────────────────────────────
Meta-Llama-3.1-8B-Instruct-Q4_K_M        8B     80%    80%   100%    87%   64.5
Qwen2.5-Coder-14B-Instruct-Q4_K_M       14B     80%    80%   100%    87%   35.8
Llama-3.2-3B-Instruct-Q4_K_M             3B     70%    80%   100%    83%  128.3
Qwen2.5-Coder-7B-Instruct-Q4_K_M         7B     40%   100%   100%    80%   69.3
Qwen_Qwen3-14B-Q4_K_M                   14B     50%    70%   100%    73%   33.4
Llama-3.2-1B-Instruct-Q8_0               1B     50%    50%   100%    67%  211.9
Qwen_Qwen3-8B-Q4_K_M                     8B     30%    70%   100%    67%   59.5
DeepSeek-R1-Distill-Qwen-7B-Q4_K_M       7B     80%    50%    20%    50%   66.2
Qwen3.5-9B-Q4_K_M                        9B     20%    70%    20%    37%   49.9
Qwen3.5-4B-Q4_K_M                        4B     30%    70%     0%    33%   77.1
MiMo-VL-7B-RL-q4_k_m                    7B     10%    50%    20%    27%   65.6
MiMo-7B-RL-Q4_K_M                        7B     10%    60%     0%    23%   64.2
THUDM_GLM-Z1-9B-0414-Q4_K_M              9B      0%     0%     0%     0%   51.3 †
─────────────────────────────────────────────────────────────────────────────────
DeepSeek-R1-Distill-Qwen-14B-Q4_K_M     14B    FAIL — wouldn't load at ctx=4096
openai_gpt-oss-20b-Q4_K_M               20B    FAIL — ~11.5GB, exceeds 12GB VRAM
Kimi-Linear-48B-A3B-BPW2.6              48B    FAIL — corrupt download
```

† GLM-Z1 uses non-standard thinking tags that our scorer doesn't strip. Its real capability is unknown from this run.

## What Surprised Me

### The 3B Llama Punches Hard

Llama-3.2-3B-Instruct at 128 t/s and 83% average is the sleeper hit. It's nearly as capable as the 8B model, runs at twice the speed, and fits comfortably in VRAM. If you're building something latency-sensitive — a coding assistant, a quick-reply bot — this is the model I'd reach for first.

### Qwen2.5-Coder-7B Is Surprisingly Well-Rounded

This model scored 100% on both MMLU *and* Code, and 40% on math. Named "Coder" but it's not just good at code — it's one of the most reliable general-purpose models in the lineup. At 69 t/s it's fast too.

### DeepSeek-R1-7B Is a Math Specialist, Not a Generalist

80% on GSM8K math — tied with the best models in the field. But 50% on MMLU and only 20% on coding. The reasoning focus pays off for math and pays a price everywhere else. Use it when you specifically need step-by-step numerical reasoning.

### 20B Models Don't Fit

The `openai_gpt-oss-20b` model is about 11.5GB at Q4_K_M quantization. That leaves only 500MB for the KV cache — not viable. It loaded partially with CPU offload, but responses were too slow and wrong. If you want a 20B+ model, you either need more VRAM or a heavily quantized version (Q2 or less), which carries its own quality tradeoffs.

### 14B Models Need Careful Context Sizing

At `--ctx-size 8192`, 14B Q4_K_M models fail to load — the weights (~8.4GB) plus the KV cache exceed 12GB. Dropping to `--ctx-size 4096` solved it for Qwen3-14B and Coder-14B. DeepSeek-R1-14B refused to cooperate even at 4096 — likely a memory fragmentation or architecture-specific issue.

### GLM-Z1 Needs a Different Scorer

GLM-Z1 almost certainly isn't as bad as our 0% score suggests. It uses thinking token formats that aren't `<think>...</think>` tags, so our strip function leaves garbage in the text. This is on the benchmark, not the model. Something to fix in a future run.

### Kimi-Linear-48B: So Close, Yet So Far

Kimi-Linear-48B is a Mixture-of-Experts model with 48B total parameters but only 3B active per token — meaning it should fit in 12GB VRAM and run at reasonable speed. At BPW2.6 quantization the file is about 8.1GB. We downloaded it twice. The first download (via `hf download`) produced a file with 6.9MB of zeros at the start — invalid GGUF magic. The second download via `wget` direct URL is in progress. Watch this space.

## The Hy-MT1.5 Translation Models

We also downloaded the full set of [Hy-MT1.5](https://huggingface.co/hy-mt) translation models from Tencent — 1.8B and 7B variants in multiple quantizations. These aren't benchmarked here because GSM8K/MMLU/HumanEval aren't the right tests for a translation model. A separate post will cover those with bilingual evaluation tasks.

## Practical Takeaways

If you're setting up a similar server, here's what I'd do differently from day one:

**1. Start with Llama-3.2-3B.** It's the best quality-per-token-per-second model in this lineup. Keep it hot in VRAM for low-latency tasks.

**2. Add Qwen2.5-Coder-7B for general use.** It's well-rounded across knowledge, code, and fast enough for interactive use.

**3. Use llama.cpp router mode from the start.** Running one server that hot-swaps models is far better than spinning up individual servers. You get an OpenAI-compatible API, automatic model loading, and a simple `/v1/models` endpoint to see what's available.

**4. Budget VRAM carefully for 14B models.** You need `--ctx-size 4096` (not 8192) to fit 14B Q4_K_M models. That's fine for most tasks, but be aware of the tradeoff.

**5. Validate GGUF downloads before benchmarking.** Check the first four bytes: they should be `47 47 55 46` (ASCII "GGUF"). A quick `od -A x -t x1z model.gguf | head -1` saves hours of debugging.

**6. Fix your thinking-model scorer before running anything.** The `reasoning_content` split in llama.cpp, the max_tokens floor, and the last-match MMLU scoring all need to be right or you'll think every thinking model is broken.

## What's Next

- Re-test GLM-Z1 with a proper thinking-tag stripper
- Kimi-Linear-48B, if the second download produces a valid file
- Translation quality tests for Hy-MT1.5
- A comparison post once larger VRAM cards (24GB+) are in reach

The benchmark script is straightforward Python using the OpenAI-compatible API — if you want to run it against your own setup, the core logic is about 100 lines. Happy to share it.

---

*Tested on Debian 13, llama.cpp built from source (May 2026), RTX 3060 12GB, i7-12700F, 64GB RAM.*


dc5102b82d71cb9e13d63287133417768b9af813bb6643140e41d8bee1da88f4

A hands-on benchmark of 13 local LLMs — Llama, Qwen3, DeepSeek-R1, Coder models and more — running on a home server with 64GB RAM and RTX 3060 12GB VRAM via llama.cpp. Includes GSM8K math, MMLU knowledge, and HumanEval coding scores, real token speeds, and an honest account of what doesn't fit.

We benchmarked 13 local LLMs on a machine with 64GB RAM and an RTX 3060 12GB using llama.cpp router mode — here's what scored well, what failed to load, and what surprised us.

13 Local LLMs, One RTX 3060: What Actually Runs (and What Doesn't)

{/* banner prompt: Retro technical poster with a schematic diagram showing a UI prototype connecting to backend systems, featuring bold typography "FROM PROTOTYPE TO PRODUCT", using a limited-print palette of off-white, orange, and dark teal with halftone textures, 16:9. */}

In my [last post](/blog/2025/05/07-jumpstarting-my-project-ui-with-v0dev-from-cli-tool-to-web-app/), I talked about using v0.dev to jumpstart my trading calculator's UI. I went from a CLI tool to a sleek web interface in hours instead of weeks. Pretty cool, right?

But here's the thing—having a beautiful UI mockup isn't the same as having a working product.

After that initial excitement wore off, I stared at my shiny new frontend and realized I'd only solved about 20% of the problem. Sure, the buttons looked great. But they didn't do anything. The forms were gorgeous, but they couldn't process real trading data. The tables were perfectly styled, but they showed placeholder numbers instead of actual profit and loss calculations.

That's when the real work began. And surprisingly, that's where AI became even more valuable.

## What I Started With vs. What I Needed

Let me paint you a picture of where I was after that first v0.dev sprint:

**What I had:**
- A simple but decent React frontend with modern styling
- Some forms for CSV uploads
- Elegant tables ready to display trading data
- Clean navigation and responsive design
- Zero actual functionality

**What I needed:**
- The frontend to talk to my existing calculation engine
- CSV parsing for different broker formats
- Real data flowing through the UI
- Portfolio saving and loading
- Proper error handling
- A deployment that actually worked

The gap between these two states? That's where most projects die.

## AI as a Finisher, Not Just a Generator

Everyone talks about AI generating code from scratch. Screenshots to UI, prompts to prototypes, ideas to interfaces. That's impressive, but it's not all where AI shines brightest.

AI becomes truly powerful when it helps you finish things.

I used Claude (specifically the `/frontend-design` plugin) to redo the look and feel after the initial v0.dev generation. And I used Codex to write most of the code.

Here's what that looked like in practice.

## The Real Problems AI Helped Me Solve
### Connecting Frontend to Backend

The real breakthrough came when I needed to integrate my existing trading calculation engine. I had this solid [CLI library](https://github.com/e-tang/shares-profit-loss-tax) that could crunch numbers, but connecting it to the web frontend? That's where things got messy.

Codex helped me:
- Restructure the backend API to return frontend-friendly JSON
- Fix serialization issues between browser and server
- Handle CSV parsing for different broker formats (especially CommSec uploads)
- Debug date handling across timezone boundaries

This wasn't about generating new code. This was about making existing code work together.

### Making the UI Actually Useful

Using Claude's /frontend-design plugin, I iterated on the visual design while keeping the functional improvements:
- Improved color contrast for financial data
- Better button states and loading indicators
- Responsive layout adjustments
- Fixed footer and documentation pages

### Building Real Product Features

Then came the features that turn a demo into a tool people actually use:

**Portfolio workflows:** Users needed to save their trading data and come back to it later. AI helped me design the data models and implement the persistence layer.

**Append-trades functionality:** Traders get new CSV exports each financial year. They don't want to re-upload everything. Codex helped me build incremental import logic.

**Symbol-level reporting:** The original CLI tool had basic reporting, but the web version needed sortable tables, filtering, and drill-down views.

**Support infrastructure:** Contact forms, file uploads for bug reports, email integration. The boring stuff that makes a product feel professional.

## What Changed Between V1 and the Finished App

Let me show you the before and after:

**Initial AI-generated UI shell:**
- Static React components
- Hardcoded sample data
- No backend integration
- Beautiful but useless

**Finished working product ([Sprolo](https://sprolo.tyosis.com/)):**
- CSV import for multiple broker formats
- Real profit/loss calculations
- Portfolio saving and management
- Incremental trade imports
- Tax-year-aware reporting
- Symbol and trade-level analysis
- Support workflows with file attachments
- Production deployment on Google Cloud

Check out the [GitHub repo](https://github.com/tyolab/Sprolo) if you want to see the code evolution.

## The Support Infrastructure Nobody Talks About

One thing I underestimated: how much work goes into making a product feel professional. It's not just the core functionality.

Users need:
- Help documentation that actually helps
- A way to report bugs without emailing you personally  
- Privacy policies that don't sound like legal gibberish
- Contact forms that work reliably

Codex helped me build all of this. Not because it's technically challenging, but because it's time-consuming and easy to get wrong.

The support flow alone involved:
- Frontend forms with proper validation
- File upload handling for CSV samples
- Server-side email integration  
- Category-based routing (bug reports vs. feature requests)
- Response templates that don't sound like robots

## The Finished Product

![](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/05/ai-beyond-the-ui-from-v0-prototype-to-a-finished-product/2026-05-08-19-36-30.png)

It's live now at: [Sprolo](https://sprolo.tyosis.com/)

The first post was about starting fast with v0.dev. This follow-up is about finishing properly with AI as your debugging and integration partner. Both matter, but only one ships products that people find useful.

babfa6b950577329da21c2abb4687d7c55e4a94807d7a56b2f4fc1e79049d524

Moving beyond UI generation: using AI to debug, integrate, and ship a real trading profit/loss calculator from prototype to production

How AI helped me turn an AI-generated frontend mockup into a fully functional trading calculator

AI Beyond the UI: From v0 Prototype to a Finished Product

{/* banner prompt: Here is a creative and engaging prompt for a banner image, based on the blog post:

**Prompt:**

A cinematic, photorealistic digital artwork of a glowing, intricate cryptographic key made of shimmering circuit board patterns and flowing data streams. The key is poised to unlock a large, stylized padlock representing an encrypted database. The WeChat logo is subtly etched onto the face of the lock. The background is a dark, futuristic digital landscape with faint binary code and abstract network lines. The scene is lit by the key's vibrant cyan and green neon glow, creating high contrast and dramatic shadows. Sleek, modern, technology-themed, highly detailed. --ar 16:9

Additional instructions: try not to use dark / blackground, not to use any word "wechat", use Vendetta mask style hacker sitting infront of a computer */}

WeChat stores your message history in a local SQLite database encrypted with [SQLCipher](https://www.zetetic.net/sqlcipher/). To read your own messages with any third-party tool — including backup utilities or search apps — you first need the encryption key that WeChat uses to lock that database.

Getting that key used to mean running commands in a terminal, attaching debuggers, or following fragile step-by-step guides that broke with every WeChat update. **WetraceTool** wraps the underlying [wetrace](https://github.com/afumu/wetrace) hook into a simple four-step wizard. You click through the steps, log in to WeChat when prompted, and the key appears on screen ready to copy.

## What you need

- Windows 10 or 11 (64-bit)
- WeChat (国际版) or Weixin (微信) installed
- WetraceTool — download the latest `WetraceTool-Setup-*.exe` installer from [here](https://github.com/e-tang/wetrace_wrapper/releases/tag/v1.0.0).

## Step 0 — Run the installer

Double-click the installer and follow the prompts. After the installation, search up "WetraceTool", the run it. Because WetraceTool needs to attach to the WeChat process, Windows will show a **User Account Control** dialog asking for elevated permissions. Click **Yes** to continue.

![Windows UAC prompt asking permission to run WetraceTool.exe](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/04/a-wetrace-wrapper-for-easy-wechat-key-extraction/2026-04-27-16-13-29.png)

This is expected — the tool cannot capture the key without administrator-level access to the running WeChat process.

## Step 1 — Start WeChat

Once WetraceTool opens you will see the four-step wizard on the left and the current step's instructions on the right.

![WetraceTool step 1 — Start WeChat, showing Launch WeChat button and "Waiting for WeChat to start" status pill](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/04/a-wetrace-wrapper-for-easy-wechat-key-extraction/2026-04-27-16-14-39.png)

Click **Launch WeChat** to start WeChat automatically, or just open WeChat yourself — the app polls for it in the background and will advance to the next step the moment it detects WeChat is running.

## Step 2 — Get to the WeChat login screen

For the key capture to work, WeChat must be sitting at the login screen (showing a QR code or an account avatar with a "Log in" button) at the moment you sign in. If you are already logged in, you need to log out first.

![WetraceTool step 2 — Log out of WeChat, with numbered instructions and a Continue button](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/04/a-wetrace-wrapper-for-easy-wechat-key-extraction/2026-04-27-16-15-24.png)

The app gives you the exact steps:

1. Click the **profile / avatar icon** in the bottom-left corner of WeChat
2. Go to **Settings → My Account → Log out**
3. Confirm the log out

Once you can see the QR code or the login button in WeChat, click **I can see the login screen — Continue ✓** in WetraceTool.

## Step 3 — Extract the key

This is the active capture step. WetraceTool starts the background hook service and then waits for you to log in to WeChat. You have **60 seconds** from the moment you click Start.

![WetraceTool step 3 — Extract Encryption Key, showing a 00:41 countdown timer and a "Start — I'm ready to log in" button](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/04/a-wetrace-wrapper-for-easy-wechat-key-extraction/2026-04-27-16-40-48.png)

When the **Start — I'm ready to log in** button becomes active (it will briefly show "Starting background service…" while the hook initialises), click it and then immediately switch to WeChat and log in. WeChat may reopen a login window — that is normal and expected. Just sign in as you normally would; the key is captured automatically the instant WeChat unlocks its database.

> **Tip:** Have WeChat's QR code scaner (Discover -> Scan) ready on your phone before clicking Start so you don't lose time.

## Step 4 — Copy your key

If the capture succeeded, the wizard advances to the final screen showing your encryption key and, where it can be detected automatically, the path to your WeChat message database.

![WetraceTool step 4 — Key Extracted, showing a redacted encryption key field with a "Copy Key" button and a database path field with a "Copy" button](https://storage.googleapis.com/www-tyolab-com-blog-assets/images/blog/2026/04/a-wetrace-wrapper-for-easy-wechat-key-extraction/2026-04-27-16-45-01.png)

Click **Copy Key** to copy the encryption key to your clipboard. If the database path was detected, you can copy that too with the **Copy** button beneath it. Both values are what you will need to open your WeChat database with a tool like `decrypt_sync` or any SQLCipher-aware client.

You can close WetraceTool once you have copied what you need.

## Troubleshooting

**Timer runs out with no key captured**
Make sure you log in to WeChat *after* clicking Start, not before. The hook only listens from the moment you click Start.

**"Starting background service…" button never becomes active**
WetraceTool needs elevated permissions to attach to the WeChat process. Close the app, right-click the shortcut, and choose **Run as administrator**.

**WeChat is not detected automatically**
Open Task Manager and check whether `Weixin.exe` or `WeChat.exe` is listed under running processes. If neither appears, WeChat may not have launched correctly — try starting it manually.

**Key extraction fails with a connection error**
The background hook service failed to start or was blocked. Close WetraceTool, reopen it as Administrator, and work through the steps again from the beginning.


2c92447650db7f5039d68451e79af9591a4b1914eb32c54d768417878b93ed1c

A step-by-step guide to using WetraceTool to extract your WeChat database encryption key on Windows, so you can read your own message history with tools like decrypt_sync.

WetraceTool is a small Windows utility that walks you through extracting your WeChat SQLCipher database encryption key in four guided steps — no command line required.

Blog