---
id: issue-fixer
type: project
title: Issue Fixer
url: https://auejin.com/en/projects/issue-fixer/
lang: en
alternate:
  ko: https://auejin.com/ko/projects/issue-fixer/
updated: '2026-08-11'
---

# Issue Fixer

*A harness system that absorbs a never-ending production bug queue*

Feb 2026

- **What** — An agent system that owns a production bug queue end to end.
- **Why** — Most of the time a bug costs goes not into the fix but into working out what broke under which conditions.
- **How** — It polls Jira and picks its own issues, compresses the attached recording into keyframes for a VLM to recover the capture conditions, then reproduces in a real browser, fixes, refutes its own fix with a before/after check, and opens the pull request.
- **New** — The point is that an LLM saying it is done never counts as the completion signal — verdicts are demoted to typed enums — and a biweekly self-audit files improvement pull requests against its own code.

## Summary

> **Level of validation**: Ran **continuously in production for 5 months**, unattended through nights and weekends, against the frontend monorepo of a B2B SaaS company in the 3D spatial-data domain. Not a benchmark or a demo: an operating record against a real bug queue taking in more than 6 issues per calendar day. Only the final merge and incident response stay behind a human gate.

## How This Differs From "an AI That Fixes Code"

| A typical bug-fixing agent | Issue-Fixer |
|---|---|
| A human picks the issue and spoon-feeds it as a prompt | **It operates the tracker itself** — polls Jira every 5 minutes, selects its own targets, and owns the issue lifecycle: state transitions, clarifying comments, rollback on failure |
| Reads the repo and goes straight to editing code | **It establishes product context before touching code** — parses the QA link to identify the target app and environment, compresses the attached recording into keyframes for VLM analysis, recovers the capture conditions (date, floor, view mode), and injects codebase analysis plus prior-PR history |
| One issue at a time (a single checkout) | **It runs them concurrently** — a git-worktree slot pool isolates each issue, 1.1 s reset between jobs, and the "hold" state that caused contention was removed so thoroughly that the schema cannot express it |
| An LLM saying "fixed it" is the completion signal | **The LLM's utterances are never the completion signal** — real-browser before/after refutation, verdicts as typed enums, failures as a 5-way taxonomy, completion via a marker protocol |
| Improving it means a human edits the prompts | **It ships its own improvement PRs** — biweekly self-audit of KPIs (every number computed by deterministic code) → automated roadmap implementation → draft PR → the next cycle statistically tests the effect |
| Validated on benchmarks and demos | **5 months of continuous production operation** — 924 incoming issues absorbed, and three major incidents converted into structures that cannot recur |

## Key Numbers (5 months of production operation, from the system's own analytics DB)

| Metric | Value |
|---|---|
| Intake pressure | **924 issues** over 5 months (~150 days) — a sustained 6+ per calendar day, with a peak throughput of 35 issues/day |
| Automated triage | All 924 classified automatically — **176** non-bugs / duplicates / out-of-scope filtered without human hands |
| Fix PRs generated | **571** — median **35 minutes** from issue detection to PR, p90 59 minutes |
| Merge rate | **35.4%** of fully reviewed PRs (159/449, excluding 122 still-open PRs — 27.8% of all generated PRs) · **82.4% of merges required zero human edits** |
| Root-cause-analysis (RCA) accuracy | **82.9%** (file-set comparison against humans' follow-up commits; over 237 adjudicable cases) |
| Merge-rate trend | 19.5% at the first audit → 35.4% cumulative (+15.9pp; approximate comparison due to differing formulas — 3 improvements shipped by the self-improvement loop during this period) |
| Operating mode | 5 months unattended and always-on (launchd services + auto-deploy on main polling with graceful-drain restarts) |

## Four Technical Highlights

**1. Grounding: establishing product context before touching code** — Before opening a file, the system pins down *what broke under which conditions*, in the real environment: it parses the QA link to identify the target app and environment, compresses the attached reproduction recording into an evidence frame set (scene detection merged with user-input timestamps) for VLM analysis, and — after observing that the dominant reproduction-failure cause is condition mismatch rather than frame misreading — promoted capture-context extraction (date, floor, view mode) to a first-class artifact. Validation serves the **working tree containing the fix** locally rather than a staging URL, because judging FIXED against staging means judging pre-fix code: false success at the source. The apps are WebGL-canvas-centric, so where DOM assertions are impossible a VLM adjudicates before/after, and reproduction and validation artifacts share one conditions–variable–response experiment schema.

**2. An execution layer that never takes the LLM at its word (agent harness)** — Every LLM utterance — completion claims, verdicts, requests for user input — is consumed only after being converted into a verifiable signal: sessions are split into 22 phases (context isolation), inter-phase interfaces are enforced as file schemas, and verdicts are demoted to typed enums that feed a state machine. Checkpoint-based resume, a 5-way failure taxonomy (retryable / issue-permanent / needs-user-input / harness-fatal / shutdown), and model-version pinning managed as a set with per-phase time budgets.

**3. Reliability and concurrency: open the throughput, close the incidents structurally** — A runaway-loop incident (5,534 rollbacks in two days) was redesigned around the principle "termination conditions must live where the loop cannot erase them" (a durable failure ledger plus a circuit breaker with no TTL). A shared-resource cascade (one held lock condemning 97 queued issues) was eliminated structurally with a git-worktree slot pool — and concurrency was opened on top of it. The migration was measurement-first: baseline metrics → isolation → a 12-hour soak → opening concurrency, with a quantitative gate at every step.

**4. A closed loop that writes its own report card and ships its own improvement PRs** — A biweekly report audits the KPIs, but every number is computed by deterministic code (the LLM writes narrative only), and KPI gaming is blocked by regex gates. The system implements the report's improvement roadmap itself and submits a draft PR; the next cycle's report regression-tests the effect with statistical checks. A complete lap — proposal → automated implementation → human merge → measured effect — is preserved in the repository history.

## Role

Solo project — architecture, agent orchestration, reliability design, observability/evaluation stack, and 5 months of production operations including incident response.

Full portfolio: https://auejin.com/en/projects/issue-fixer/portfolio/

## Related

- [Software Engineer @ Cupix](https://auejin.com/en/experience/cupix/)