LLM Harness Writing Study

2026-05-26#llm-engineering#agent-runtime#harness#writing-method

Source

Note: this is a study note from reading — it records only the conceptual framework and the writing method. Any concrete numbers, company cases or future dates from the original need to be re-verified item by item before they enter a formal article.

Conceptual framework

The original tells the evolution of LLM Engineering as a three-ring spiral:

  1. Prompt Engineering: the task is relatively static; a human optimizes a single input through wording, roles, examples and output format. It solves "how to ask", but gets stuck on multi-step tasks.
  2. Context Engineering: the task unfolds dynamically; a program injects the previous step's output, tool feedback, retrieval results and compressed history into the next step. It solves "what to show the model", but the rules are mostly preset by humans.
  3. Harness Engineering: once the model is strong enough, the focus shifts to giving the agent tools and feedback channels so it judges for itself what to look at and what to verify — while sandboxes, permissions, tests, CI round limits and structural constraints keep it from running away.

The most valuable distinctions:

  • Context Engineering is a program orchestrating information on the agent's behalf.
  • Harness Engineering gives the agent the ability to acquire information and feedback, then constrains behavior with boundaries.
  • Runtime capability is not a harness. State persistence, compaction and session resumption are more like agent-runtime infrastructure; a harness cares about whether the agent can actively obtain feedback at runtime, and whether it is constrained by executable boundaries.

Reusable judgments

The article's through-line is not "new term replaces old term" but "bottleneck migration":

  • in the prompt era, the bottleneck was expression;
  • in the context era, the bottleneck was information flow;
  • in the harness era, the bottleneck is environment, feedback loops and control systems.

The framework suits agent-engineering articles because it puts abstract concepts back into engineering problems: why the previous stage stopped being enough, which concrete blocker the next stage actually solved, and what new reliability problems it introduced.

Sentence skeletons transferable to my own writing:

  • "The bottleneck moved from A to B."
  • "X is not a paradigm designed out of thin air, but experience reality taught us after Y started running."
  • "Let go first, then derive boundaries backward from the failure modes."
  • "The previous ring did not disappear — it became the current ring's infrastructure."

These skeletons work, but formal writing must swap in my own cases and language rather than copying them outright.

Writing method

1. Give the reader a map before unrolling the history

The opening uses one paragraph to lay out the three-stage relationship and a TL;DR, so the reader knows what follows is not concept-stacking but an answer to three questions:

  • why the old approach stopped being enough;
  • what the new approach solves;
  • where the new approach gets stuck in turn.

This opening suits long-form pieces: the reader holds the navigation map before entering the details.

2. Every section advances on the same template

Each stage in the article walks roughly the same order:

  1. What the typical practice was at the time.
  2. Why it genuinely worked.
  3. What it could not solve.
  4. Where the bottleneck therefore migrated.

The repeated structure makes each conceptual upgrade feel natural instead of like jargon being forced in.

3. Abstract definitions must grow out of cases

Harness Engineering is not defined up front; the article first talks about "letting go" and "adding insurance", then validates with three cases — Stripe, Anthropic, OpenAI:

  • Stripe: a fast-feedback toolchain and a restricted execution environment.
  • Anthropic: deriving feature granularity, JSON state and end-to-end verification constraints backward from long-task failure modes.
  • OpenAI Codex: shifting the engineer's center of work from writing product code to environments, feedback loops and control systems.

This matches thought-forge's rule against concept-first writing: let the reader see the practice first, then compress the practice with the term.

4. Concept boundaries need counterexamples

The article explicitly says "what does not count as a harness", assigning compaction, session resumption and memory management to the runtime, not the harness.

This move matters: a new concept's worst enemy is boundary inflation. Giving counterexamples tells the reader the author is not using a new word to hold everything.

5. Close by pointing at the next ring's bottleneck, not by summarizing

The ending does not stop at a "Prompt -> Context -> Harness" recap; it points the next ring's problems at eval and governance.

This close suits technology-trend pieces: it lets the article step naturally from "defining the current stage" to "what the next article could be".

Article angles worth developing

  • A harness is not an agent runtime: stop selling compaction as a product feature
  • Let go first, then add insurance: where boundaries in agent engineering come from
  • From context rot to eval rot: why LLM Engineering's bottleneck keeps migrating
  • Agent engineering is not writing longer prompts — it's designing shorter feedback loops

Verification checklist

To grow this X Article's content into a formal piece, at minimum verify:

  • Karpathy's original formulation of Context Engineering and its publication date.
  • Stripe Minions' toolchain, CI round counts and PR numbers.
  • The concrete setup of Anthropic's long-task harness experiments.
  • The original context in the OpenAI Codex team's article about harnesses, environment design and code volume.
  • The experimental conditions behind the quantitative claims from Epsilla, LangChain Terminal Bench 2.0, Microsoft/Salesforce context rot.
  • Whether the 2026 industry events around eval, governance and agent standards trace to official sources.