Background

As AI's coding capabilities keep improving, more and more of production-grade software development is being handed off to agents instead of human engineers. In this paradigm shift of software engineering, the real question is not whether agents can write code, but: where should human engineers be paying attention now, and what mechanisms can preempt or reduce the technical debt agents leave behind?

With those two questions in mind — and from the starting point of wanting to learn and practice the engineering philosophy of Harness Engineering — we developed kimi-code-swarm, an open-source desktop application built on top of the Kimi Code CLI. The concrete problem it solves is this: you have one GitHub account and one local machine, but in the course of software development bugs and features keep surfacing that need to be fixed — how do you get AI to process them in parallel instead of waiting in a serial queue?

kimi-code-swarm spins up multiple agents at the same time. Each agent takes on one task independently (fix a bug or implement a feature), writes code, runs tests, and opens a PR; the system then reviews those PRs and merges them into the main branch if they pass. The entire workflow runs locally — no extra accounts, no cloud resources needed.

The dilemma we actually hit has two extremes: either you let the agents produce code quickly without verifying correctness, and end up with a pile of technical debt; or you make the verification process so heavy that the overall speed ends up slower than writing the code yourself. The project started from an empty git repository. We first ran directory governance under the recursive-map principle, then built up a set of tools and constraint rules — CI scripts, AST analyzers, dead-code cleanup, repository-consistency checks.

Along the way, we noticed a talk by Dex on the OpenAI team. He cited a survey of 100,000 developers: when using AI for software engineering, most of the time goes into rework and code churn; complex tasks and brownfield codebases perform poorly; output increases, but a lot of that output is just patching code that was hastily shipped the previous week. This matched the walls we had been hitting — telling us what we were seeing was not an isolated case.

What follows are the walls we ran into in this project, and the judgments that grew out of them.

Key Challenges and Discussion

Intrinsic Failure Modes

In the development of kimi-code-swarm, we found that a significant share of the coding mistakes an agent makes are not random slips but reproducible, intrinsic patterns — not "it slipped this time" but "it does this every time." These problems come out of habits grown into the training distribution; they cannot be cured by a simple prompt telling the agent to "try a little harder."

We did a postmortem on the typical cases, and the claim we'd make is this: it is a common ailment of agents whose coding capability is genuinely excellent but whose actual engineering judgment lags well behind. Including but not limited to ignoring program robustness, misaligned interpretation of vague requirements, self-initiated guardrails. One mode is taking things for granted — treating "the main path runs" as "the feature is done," skipping verification even when there are tests; the other is acting on its own initiative — silently adding operations and "guardrails" you never asked for, in ways whose consequences you cannot foresee. Both deviate from what we expect.

A few concrete scenarios:

  • Cleanup done only halfway, runtime crash. When deleting a fallback mechanism that "looked useless," it removed only the entry point and never checked whether other parts still depended on it — like demolishing the exterior of a load-bearing wall while the beams inside are still holding things up. As a result, the app crashed outright while wrapping up a conversation turn.

  • A fallback deleted as redundant, a blank screen on exception. The CLI output is occasionally malformed and structured parsing fails; the agent should at least display the raw output as-is. But that fallback display logic was likewise deleted as "something useless" — so the moment parsing failed, the frontend was left with nothing but an error log, and the user saw not a single character. It had treated "the safety belt for exceptions" as "excess weight."

  • Only the backend logic gets attention; the frontend's basic experience is missing entirely. It put all of its focus on the backend and never verified the constraints of input and rendering: the input box did not support line breaks, Shift+Enter triggered a send directly, and the user had no way to compose multi-line content; long message bubbles did not wrap either, stretching horizontally past the edge of the screen.

There is another, harder class of failure: the agent has no idea where the problem actually is. Kimi shows this most clearly on long-chain bugs — it will guess, edit, and claim "fixed" over and over, but every round is running in place; each "fix" just swaps one symptom for another while still walking the wrong direction. By the time you realize it is stuck, you have already accumulated several wrong commits.

Given problems like these, the reliable approach is to turn these known failure modes into mechanical constraints up front: code changes must update docs and tests in sync, must not break existing tests, must not introduce redundant designs or anti-pattern practices the codebase has already called out. In that sense, Harness Engineering is just enforcing good constraints — following the basic principles of software design. What changed is not the principles, but the audience — from human engineers to agents. Constraints that used to ride on "engineer self-discipline" now have to be made mechanical and leave evidence that can be checked.

Drift and Constraint Granularity

Agents drift during code generation, and the core of the problem is that human attention can't always align with the granularity the agent works at. In a commercial project on the order of hundreds of thousands of lines, we ran into a particularly typical case — and this kind of drift is not the agent disobeying the docs; the agent was strictly obeying them, and the result still deviated from what we expected.

The project uses a separated frontend / backend architecture. A button click on the frontend serializes the request to the backend, calls a method on the Controller, which then hands the business logic off to the Service layer. The docs carried a constraint whose actual purpose was to plug the hole of "the Controller layer rewriting or duplicating business logic." The wording: when the backend already has a working capability module, the Server layer should follow the reuse principle and only do capability exposure and adaptation; it must not rewrite or duplicate business logic.

I asked the agent to build a new feature module and to follow this constraint strictly. It did — very strictly. It added a long stretch of adaptation logic inside the Server layer, because the docs spelled out plainly that "the Server layer only does capability exposure and adaptation" — since "adaptation" was on that line, that is where adaptation went. The Server class became bloated, its responsibility blurred, the whole thing more fragile.

This, of course, is not the design intent we actually had. The Server should really only do serialization; fine-grained adjustment and adaptation belong in the capability layer below. The phrase "do not rewrite business logic" was written to plug the hole of "the upper layer rewriting the lower layer's logic," and we never noticed that we were simultaneously putting the word "adaptation" on the Server's line. Between the literal wording of the constraint and the real intent behind it sat a small gap — an agent that follows the literal wording strictly will drift right into that gap.

This experience gave us a more concrete sense of how a few related disciplines divide the work:

  • Prompt Engineering asks how to make the AI understand the instruction.
  • Context Engineering asks how to fill in enough information in long-chain scenarios where context is short and information is missing.
  • Harness Engineering asks: when a gap remains between the literal wording of an instruction and the real design intent behind it, the agent will inevitably drift inside that gap — and what kind of system constraints keep that drift outside the walls?

Drift is not fatal on small projects — it can even look like "style variation." But as the project gets larger, the docs get looser, and the system design itself carries more ambiguity, drift compounds, and eventually shows up as steadily accumulating technical debt.

To be honest about it: our current response to drift is fairly primitive — interrupt early, roll back, re-align the moment a deviation appears, "killing the debt in the cradle." It does block debt from accumulating, but it costs an enormous amount of attention, and the actual output is not high. Turning this layer from human gatekeeping into a constraint that's both landable and general is something we openly admit we have not solved yet.

This actually points to an open question: how abstract should documentation be, in order to maximize the return? Constraints written too tightly tie the agent's hands — the rule "clear goals, fuzzy details" later in this piece is aimed precisely at that side. But constraints written too loosely, too abstract, leave a gap of ambiguity for drift to slip into, the way it did this time.

How Skills Work Well in the Harness Engineering Context

Continuing from the previous question: to maximize the return from the agent's output and bring it as close as possible to what we have in mind, the workable approach is to control the input.

At the level of doc-based constraints, the input is instructions like documentation. One optimization path is to make the agent align with us before its final output — turning a single requirement input into multi-turn information. Many existing skills already do this. But even though a skill is defined as a reusable instruction template, we lean toward customizing the important ones, because what makes a skill genuinely powerful is its high flexibility and customizability — and its ability to control the level of abstraction. For example, in a skill we can specify which document inputs the agent is consulting that look unusual from its point of view.

To give a concrete example of one we built ourselves, call it task intake: on any task touching code / files / configuration, the first thing the agent does is load this skill. Its iron law is a single sentence — no implementation code may be written without the user's explicit approval. The gate has four phases; the first three write not a single line of code:

  1. Requirement clarification and document list — Restate the requirement in your own words, list assumptions one by one, ask questions on the spot when something is ambiguous.
  2. Scenarios and expected results — Break the task into concrete scenarios, mandatorily covering normal paths, failure / edge paths, and the user's perspective. Writing only the happy path means this step is not finished.
  3. The alignment gate — Lay the first two phases out in full for the user and wait for an explicit approval; without a yes, phase four is off-limits.
  4. Implementation and validation — Only past the gate does coding begin, and it stays bound by a set of invariants; validation must paste the raw output, not just say "passed."

Here lies the design trade-off we consider most important: phase four is written as "invariants," not "prohibitions." A prohibition ("don't mock," "don't delete carelessly") can be read and bypassed all the same, because nobody checks; an invariant is a property that can be detected — "grep the references before deleting and list them out" produces a reference list, "paste the raw validation output" produces a log, "list every scenario" produces a scenario table. Only when requirements are written as invariants that leave evidence does the gate actually grow teeth, instead of becoming one more convention politely ignored.

Additional Lessons

Directory-Level Context Window

The rule "the map is the boundary" works well on the project's root AGENTS.md, but when the document hierarchy goes one level deeper, problems begin to emerge.

For example, during an issue fix, the agent automatically named a document agent-engine-spawn-windows.md. This name is meaningless to a new agent — it cannot determine from the filename whether this is a critical bug post-mortem or an engine startup plan, forcing it to open and read the file, wasting precious context window.

For such document-level context-window optimization, we adopt two approaches:

Approach 1: Subdirectory-level AGENTS.md

Place a minimal AGENTS.md under docs/exec-plans/ that simply lists the topic and applicable scenario of each file in the directory. When an agent reaches that directory, it reads the index first and then dives in on demand — achieving map-style access.

Approach 2: Self-describing filenames

Rename the critical document directly to something like critical-bug-solved.md. The principle is filename-as-summary.

Both approaches can be made into reusable skills. The core idea remains the same: under the premise that an agent's attention is a finite resource, compress information at the directory level. The efficiency bottleneck of an agent is fundamentally an information-theory problem — the input determines the output, and attention is the scarce constraint.

Rethinking Is Necessary — You Cannot Outsource Your Thinking

During the development of kimi-code-swarm, we ran into a string of production-environment problems: install the package, launch the app, and every attempt to create a new agent fails — not even an error toast pops up.

The first instinct is to have the agent fix it, and it did keep fixing things — one commit added path detection for node.exe (the user's Node was installed in an nvm directory, not on the system PATH), the next commit fixed a packaging omission and added a startup health check… Every commit precisely resolved one symptom, yet the app still would not start. Only after three commits did we realize: the problem lay not in any single symptom, but in the architecture itself.

This desktop app was, astonishingly, relying on tsx to transpile TypeScript on the fly at runtime in production — its startup path required three things to hold simultaneously: "Node on the user's machine + a bundled esbuild binary + the source code." Patching along this path is forever plugging leaks in a pipe that should not exist in the first place. The real fix was not a fourth patch but stepping back a level to replace the path: precompile to JS with tsc at build time, run dist/index.js directly in production, and delete the entire "runtime transpilation" segment. One architectural decision was worth more than the sum of all the prior patches.

The trouble was, letting the agent patch endlessly looks like steady progress but is fundamentally inefficient — it circles at the symptom level while you mistakenly believe you are solving the problem. In the age of agents, coding can be outsourced — but when the system's throughput hits a bottleneck, the ability to recognize that early and make the call is irreplaceable. Coding can be outsourced; thinking cannot.

Golden Rules

From the walls we hit and a few prior principles, we settled on a small set of golden rules living in the repository:

  1. The map is the boundary — Agents only read AGENTS.md; details are loaded from docs/ on demand. Comes from the directory-level context-window incident.

  2. Mechanical constraints first — Code must pass CI: types → linter → AST → build. Rules have to be detectable by a script; you cannot rely on a human to watch.

  3. The repository is the single source of truth — Slack / verbal agreements do not exist for agents.

  4. Execution means updating docs and status — After code changes, update related docs and mark status in docs/STATUS.md (✅ / ⚡ / 🚧 / ❌); when blocked, review the documents consulted in this session.

  5. Constraints as code — All rules must be mechanically enforceable; conventions that cannot be automatically checked do not exist.

  6. Recurring bug or unclear root cause → add logs to locate and trace — Record the root cause in code comments or commits after fixing.

  7. Code changes must be validated — After modifications, build, test, and lint must all pass before merging.

That said — if you constrain an agent at every detail, it instead finds its hands tied and may not be able to work at all. So there is one more principle that doesn't fit the seven but runs through the whole approach: clear goals, fuzzy details. For long-term planning, the engineer operates at a higher level of abstraction; as long as the direction is right, letting the agent play freely under broad constraints actually gets more done. This is much like working with a highly capable colleague: you only want the direction to be right; you set a goal rather than imposing excessive behavioral constraints, and people are motivated by the goal to figure out how to do their best.

Closing

The shift in the software-engineering paradigm is a leap in productivity, and it also brings entirely new problems. Harness Engineering has just one core problem to solve: how to maximize the returns from coding agents without making human attention the bottleneck. That is something we are still on the road to.