---
id: isopod
type: project
title: Isopod
url: https://auejin.com/en/projects/isopod/
lang: en
alternate:
  ko: https://auejin.com/ko/projects/isopod/
updated: '2026-08-11'
---

# Isopod

*Not an agent that writes code, but a harness that runs an organization's development process*

Aug 2026

- **What** — A harness that makes multiple agents carry a natural-language development spec through to the end.
- **Why** — A spec written by a person leaves too much room for interpretation for a machine to judge how far it has got.
- **How** — It pins the spec into a checkable state machine, splits it into work cards on a dependency graph, and runs agents with no human intervention until the graph is drained, producing verified commits.
- **New** — Progress is measured by what is left in the graph rather than by what an agent reports.

## Summary

> **Conditions it was validated under**: Roughly two months on the conversational AI assistant of Cupix, a B2B 3D digital-twin SaaS company. **Three teams (two internal, one external) collaborating through a tracker**, **20.5% of cards cut from scope**, and **17 architecture decisions superseding one another** while it ran. These were not laboratory conditions.

## How This Differs From "Just Telling an Agent to Build It"

A coding agent optimizes for **one unit of work**. An organization has to run **a process that keeps going**. The five things real work demands below are axes most agent harnesses do not address at all — and nothing in the survey did all of them **on a single dependency graph at once**.

| What real work demands | A typical coding agent / SDD tool | isopod |
|---|---|---|
| **Pin the spec rigorously, then prove the implementation conforms** | The spec is a prose prompt; conformance is stood in for by "the tests are green" | The spec schema **requires a state machine** (states, events, transitions — **including illegal transitions, enumerated**) → every acceptance criterion ships falsifying examples → tests target transitions → and all four gate layers are **adjudicated by deterministic programs** (TS compiler / AST mutation / deep-diff / process spawn) |
| **Report progress at any moment** | A chat scrollback is the only state | 200 cards as version-controlled SSOT, a 7-state machine, tracker mirroring. "What is where" **exists continuously in both the files and the tracker** |
| **Collaborate with other teams** | Absent (built on the premise of working alone) | External-team cards are a **read-only mirror** — ownership is a first-class concept in the data model, write scope is restricted field by field, and **the code raises an exception** to enforce it. What we expect back is pinned in a `consumer_contract` field |
| **Spans that must wait on a human policy decision** | Either absent, or so many approval gates that the pipeline runs at human speed | Gates are separated **by layer** — humans only in the requirements layer, **zero** in the execution layer (385 passes measured unattended). Undecided policy lives as a **gate card that is a graph node**, so exactly the cards depending on it wait |
| **Deterministic tracking of specs that change mid-flight** | Rewrite the prompt — the previous state is gone | Per-state editability rules plus **migration cards** (a shipped card is never edited; a new path forward is created instead). Reverse-dependency cleanup on retirement, forward-dependency injection on new decisions, and **edge review when a decision is superseded** — all three axes codified as audit functions |
| **All of the above in one system** | Of six adjacent tools surveyed: **one** has built-in tracker integration; **none** has a multi-team ownership model | The five above operate **on one dependency graph**, and agents drain that graph on top of it |

## Headline Numbers (All Measured; Reproduction Commands Included)

| Metric | Value | Note |
|---|---|---|
| **Conditions it ran under** | **3 teams · 200 cards · 438 dependency edges** | 41 cards (20.5%) cut from scope, 17 decisions superseding one another, 1,018 subtask seeds |
| **Stories the agent pipeline completed** | **70 of 73 verified (95.9%)** | 8 stories a human implemented directly are **excluded** |
| **Unattended agent passes** | **385** (5 personas) | **Zero** human approvals in the execution layer |
| **Measured agent cost / compute** | **$2,754 / 144.4 hours** | ≈ $39 per verified story |
| **Peak concurrency** | **13 passes** (1.81× average) | Measured in one-minute event buckets |
| **Harness code** | **142K LOC**, test:source ratio **0.94** | 284 test files / 63.9K LOC |
| **Incident → rule → audit code** | **20 of 29 rules trace to real incidents**; **22 graph audits** | Some fire automatically in `npm run check` |

## Technical Highlights

The design goal fits in one sentence: **policy judgment goes to the LLM, everything else goes to a deterministic machine.** The four items below are about where that seam was drawn and how it is held.

**1. Not a toy — a harness run in a real multi-team environment.** It managed 200 cards and 438 dependency edges across three teams (two internal, one external) collaborating through an issue tracker, under conditions where the spec kept moving: **20.5% of cards (41) were cut from scope**, and 17 architecture decisions accumulated and superseded one another. So the harness is not built around "how to create cards" but around **"how to move an already-shipped card to a new destination without falsifying history"** — migration cards, per-state editability rules, and mandatory dependency-edge review when a decision is superseded. External-team cards are a **read-only mirror**, and that ownership boundary is not a documentation convention: **the code raises an exception** (verified by execution). Of six adjacent tools surveyed, only one has built-in tracker integration, and **none has a multi-team ownership model.**

**2. Humans only for policy decisions — deliberate HITL placement.** The industry has no established principle for which layer approval gates belong in, so practice tends toward either all-in or all-out. isopod puts humans **only** in the requirements layer (authoring specs, reviewing and publishing cards, opening observation gates) and **zero** gates in the code-execution layer. **385 passes ran unattended**, and the human touches exactly **8 commands**. This placement was not a preference — it came out of an incident. After the pipeline stalled waiting for approvals, drift signals were **demoted from a gate to an input annotation for the spec agent.** Decisions that genuinely require waiting were not removed but **lifted onto the graph as gate cards**, so that exactly their downstream stops until a human opens them.

**3. Determinism at the anchor graph — two-model cross-check with a hard-zero gate.** When natural-language specs are converted into an anchor graph, **two different models** extract independently, and the run passes only if the **symmetric difference of anchors is zero AND the symmetric difference of edges is zero.** The real design work is in the join key: instead of an ID the model invents, the key is **a character interval (a span) over the source text.** The quote (`claim_text`) may not differ by a character, but it is not itself the identity — it is a **pointer** used to resolve the span. Two extractors segment a spec independently, so the same claim comes back quoted at different lengths; under a string comparison those are total mismatches, but **overlapping spans mean the same claim**, and overlap admits no threshold. After the gate passes, deterministic checks run (single-concern, grouping, retirement, per-kind). Across the whole pipeline there is **exactly one LLM judgment point** — the card-drafting step — and even there all four judgments are forced into **enum / bool / id fields.** Anchor adoption (copy and renumber) and card publishing and cleanup are **pure determinism with zero model calls.**

**4. Tests defined by a state machine — TDD in action.** The spec document is not prose; it is a **state machine.** `states`, `events`, and `transitions` are **required** by schema, and every transition carries a `legal` flag so that **illegal transitions are enumerated too** (one real card: 15 of 25 transitions were illegal, i.e. that many negative tests). Acceptance criteria must ship with `falsifying_examples` — examples that would refute the criterion — and test-plan items link back to transitions via `targets`. On top of that sits **red-baseline**: before implementation, a test must be proven to fail, and the proof is **the runner's own subprocess exit code**, not the agent's word. The implementation agent cannot edit test files. At the spec stage, a rule rejecting specs where the happy-path and edge-case expectations are identical **structurally forecloses passing with a constant-returning stub.**

**+ Even the places where a gate was waived are recorded deterministically.** Exemption surfaces exist in the verification stack (n/a verdicts at L3/L4, demoting L1 to warnings, lowering the L2 threshold). The choice was **to count them rather than block them** — every exemption passes through a single seam into a ledger (NDJSON), per-surface counts surface in the run report, and a static audit detects any new exemption point that bypasses the seam. Blocking would stall the pipeline on human response time (see incident 2 below); not counting would leave nobody knowing which defense line is nominal and which is real.

## My Role

**Sole designer and implementer** of the harness. On the target project I operated it — owning the requirements layer (authoring specs, reviewing and publishing cards, opening observation gates) plus PR review and merge. The spec → plan → test → implement → review execution span ran unattended.

## Honest Limitations (Details in the full portfolio)

- **`done` still is not a complete proof that it works.** A thin vertical-slice smoke test now runs automatically right after merge, bringing runtime truth inside the pipeline — but **a failure does not block the state transition.** Surfacing an evidence-less `done` in the report is as far as the current design goes, because blocking would directly reproduce the incident where the pipeline stalled waiting on a human.
- **Exemptions are counted, not blocked.** Agents can still enable gate exemptions. What changed is that they went **from invisible to being a metric** — and **the 70 stories in this document predate that instrumentation**, so their exemption distribution cannot be known retroactively.
- **Portability was demonstrated; its economics were not.** On a second project the full chain ran from **nothing but a single natural-language spec** — 19 anchors → 3 cards + 1 gate → all 3 code stories verified → PR, with the service actually booted and **all 13 spec claims confirmed over HTTP.** But **"zero harness edits" does not hold at this scale**: it took 3 roll attempts and 4 harness defect fixes (three of them latent in the original project, firing for the first time under the new layout). And this is a **practice example** (3 cards), not a second real project. **The cost of building the harness was still not measured**, and likely exceeds the estimated savings on this one project.
- **Two agent CLIs supported** — clearly behind adjacent tools (30+ / 40+ / 11 platforms), and deliberately so: proving the adapter is genuinely swappable with two was chosen over supporting N.
- **The real-world sample is one project** (14 runs / 73 stories). The second project is a 3-card practice example and does not widen it. Never placed on a public benchmark.
- **No ablation separating harness contribution from model contribution.** The defensible claim stops at "the spans that could not exist without the harness" — dependency ordering, automatic restart, gates, and merge verification.

---

*Reproduction commands for every number are in the the full portfolio appendix. The company is named; product and personal names are anonymized.*

Full portfolio: https://auejin.com/en/projects/isopod/portfolio/

## Related

- [Software Engineer @ Cupix](https://auejin.com/en/experience/cupix/)