Harness from Scratch: building the system that checks its own work

01 / THE QUESTION

Why build a harness when good coding agents already exist?

I had just learned to see a coding agent as more than a chat window. The model is only one component. A harness supplies the loop, context, tools, permissions, workspace, event history, budgets, and delivery rules that turn model output into controlled work.

Pi, OpenCode, Goose, and OpenHands each made a different boundary visible. My goal was not to clone all four. I wanted to build a small vertical slice so I could understand where a harness earns trust: at the contracts between intention, execution, and evidence.

The learning method came from Linux From Scratch. Instead of beginning with a finished platform, I would assemble one layer at a time, keep every boundary inspectable, and make the project operate on itself as early as possible. The working name became Harness from Scratch.

02 / THE MENTAL MODEL

Four projects helped me separate the layers.

I used the projects below as design references, not dependencies. The repository produced on this day does not import their code or claim compatibility with them.

That first five-level sketch was an exploration map. The repository’s final architecture contract simplifies it to contracts, execution, services, and interfaces. That is the more useful boundary because it describes code that can be tested.

03 / THE FIRST CUT

I reduced the platform to one M0 exit gate.

The initial idea included a web app, TUI, control plane, agent server, sandbox runner, SQLite, Postgres, object storage, observability, MCP, ACP, Docker, and eventually Kubernetes. Building all of that first would have produced a wide scaffold with no proof that the central contracts worked.

So I imposed three constraints:

1. 01

   One language until measurement says otherwise

   TypeScript, Node 22 or newer, and pnpm workspaces across every layer.

2. 02

   Offline determinism before a live model adapter

   A scripted FakeModel would make the kernel testable without network or credentials.

3. 03

   A manifest in and a report out

   The first milestone would accept a task contract, evaluate an existing working tree, run verification, and emit machine-readable evidence.

04 / BOOTSTRAPPING WITH PI

Pi was the harness; Qwen was the model.

I launched Pi through Ollama and selected the model label shown in the session. Pi supplied the interactive harness—context, read/write/edit/bash tools, and the turn loop. Qwen supplied model output. The files and commits were their result, but the new repository does not embed either Pi or Qwen.

# Start Pi with Ollama as its model provider
ollama launch pi

# The interactive picker showed this model for the session
qwen3.8:27b

The long bootstrap prompt described the monorepo, task schema, events, fake model, policy engine, tests, documentation, and infrastructure direction. I also gave it an explicit finish line: do not stop at a file tree; demonstrate the task manifest and report pipeline.

The session itself ran on Node 24.18.0 and pnpm 11.15.1. The repository deliberately targets Node 22 or newer. Ollama now documents the same ollama launch pi integration and Pi’s core coding tools.

05 / REPOSITORY SHAPE

The monorepo separates contracts from products and processes.

I wanted the kernel to remain a local library rather than become a microservice. Apps can present the work; services can schedule and isolate it; packages own the portable contracts. That keeps the part most likely to be evaluated—goal, model, tools, budget, events—small enough to reason about.

harness-platform/
├── apps/
│   ├── cli/                 # working validate/run exit gate
│   ├── tui/                 # typed placeholder
│   └── web/                 # typed placeholder
├── services/
│   ├── agent-server/        # roadmap placeholder
│   ├── control-plane/       # roadmap placeholder
│   └── sandbox-runner/      # roadmap placeholder
├── packages/
│   ├── events/              # schemas + wire serialization
│   ├── kernel/              # local model/tool loop
│   ├── models/              # interface + FakeModel
│   ├── tools/               # validated tool registry
│   ├── policy/              # pure permission decisions
│   ├── sdk/                 # manifest in + report out
│   ├── sessions/            # in-memory append-only log
│   ├── workspace/           # lexical path scoping
│   ├── mcp/                 # contract types
│   └── acp/                 # contract types
├── tasks/                   # YAML contracts + ignored reports
├── evals/                   # future scenarios and golden repos
├── skills/                  # platform-builder operating guide
├── infra/                   # Docker + MinIO development files
└── {ARCHITECTURE,EVENTS,ROADMAP,SECURITY}.md

At the verified public commit, the repository contains 73 tracked files and 3,847 lines excluding the lockfile. The breadth is real, but maturity is uneven by design: the CLI and core packages work; the TUI, web app, and three services are placeholders that declare future boundaries.

pnpm install
pnpm test
pnpm typecheck
pnpm harness validate tasks/kernel-0001.yaml
pnpm harness run tasks/kernel-0001.yaml

06 / TASK CONTRACT

The manifest became the spine of the design.

A natural-language request is useful to a model, but it is a weak contract for a runner. I wanted the same input to express scope, proof, permissions, resource ceilings, and the expected delivery form. That became tasks/kernel-0001.yaml.

id: kernel-0001
title: Add agent event serialization

goal: >
  Implement JSON serialization and deserialization for all kernel events.

acceptance:
  - All event variants round-trip without data loss
  - Unknown event versions return a typed error
  - Unit and integration tests pass

allowed_paths:
  - packages/events/**
  - packages/kernel/**
  - evals/**

permissions:
  fs.read: allow
  fs.write: ask
  process.exec:
    "pnpm test*": allow
    "pnpm lint*": allow
    "*": deny
  network: deny
  git.push: deny

budget:
  max_model_tokens: 100000
  max_tool_calls: 100

delivery:
  type: pull_request

Goal + acceptance

Human intent and observable completion criteria.

Allowed paths

The portion of the working tree this task is permitted to change.

Permissions

Declared allow, ask, and deny decisions for files, commands, network, and Git.

Budget

Maximum model tokens and tool calls for a future integrated run.

Delivery

The intended artifact—in this case, a pull request.

07 / EVENTS

Events are the common language between replaceable parts.

If the kernel can run locally, in a sandbox, or inside an evaluation, its observable output cannot depend on a particular UI or database. I used a fixed envelope and twelve discriminated payload schemas.

{
  "v": 1,
  "type": "agent.started",
  "eventId": "evt-…",
  "at": "2026-08-30T12:00:00.000Z",
  "actor": "kernel",
  "data": {
    "agentId": "agent-…",
    "sessionId": "sess-…",
    "model": "fake-model/v1"
  }
}

• session.created
• agent.started
• agent.stopped
• model.request
• model.response
• tool.call
• tool.result
• task.updated
• budget.warning
• policy.decision
• run.recorded
• error

The decoder is intentionally staged. Invalid JSON, an unsupported envelope version, an unknown event type, and an invalid payload are different operational failures. A caller might retry one, migrate another, quarantine a third, and treat the last as a producer bug.

// The decoder fails at a precise boundary.
JSON.parse(raw)                         // EventParseError
supportedVersions.includes(event.v)    // EventVersionError
isEventType(event.type)                 // UnknownEventTypeError
eventSchemas[event.type].safeParse(...) // EventSchemaError

08 / DETERMINISTIC CORE

A fake model made the real loop testable.

The first model implementation is deliberately not Ollama. FakeModel replays a queue of scripted turns, records requests, estimates tokens deterministically, and returns a fixed acknowledgement when the queue is empty. That makes tool calls and budget edges reproducible offline.

const turn =
  this.queue.shift() ??
  { content: `[fake-model] ack #${seq}` }

const promptTokens = estimateTokens(prompt)
const completionTokens = estimateTokens(scriptedCompletion)

return {
  content: turn.content ?? "",
  toolCalls: turn.toolCalls ?? [],
  usage: {
    promptTokens,
    completionTokens,
    totalTokens: promptTokens + completionTokens,
  },
}

The kernel contract then stays narrow:

goal + model + tools + budget
                │
                ▼
        runAgent(options)
                │
       ┌────────┴────────┐
       ▼                 ▼
typed event stream   final text

On each turn, the kernel emits a model request, calls the model, records usage, and emits a response. With no tool calls it stops successfully. With tool calls it validates arguments against the tool’s Zod schema, executes the registered tool, emits a typed result, adds that result to context, and continues. Token and tool-call ceilings can warn and stop the run.

09 / POLICY + EVIDENCE

The harness’s best moment was refusing its own work.

The policy package is pure: given a permission map, action, and optional subject, it returnsallow, ask, or deny. More-specific patterns win. For paths, * stays within one segment while ** can cross directories.

The CLI combines that decision logic with Git status, tests, and report generation. Its pipeline is schema → branch → dirty paths → command policy → tests → evidence.

const changed = changedPaths(cwd)
const violations = changed.filter(
  (path) =>
    path !== relManifest &&
    !pathAllowed(manifest.allowed_paths, path),
)

const policyCheck = {
  ok: violations.length === 0,
  violations,
}

The first real run was blocked. The new scaffold touched root, app, service, documentation, and infrastructure files while the task allowed only packages/events/**, packages/kernel/**, and evals/**. That was not a nuisance. It was the first evidence that the contract could contradict the builder and win.

After the base scaffold was committed, the task branch was cleaned up, type and event bugs were fixed, and the report parser was corrected, a later run passed with no dirty paths and no policy violations.

{
  "schema": "run-report/v1",
  "status": "passed",
  "branch": "tasks/kernel-0001",
  "policy": {
    "changedPathsOk": true,
    "changedPaths": [],
    "violations": []
  },
  "tests": {
    "command": "pnpm test",
    "exitCode": 0,
    "ok": true,
    "total": 40,
    "passed": 40,
    "failed": 0
  }
}

10 / WHAT BROKE

The failures improved the design more than the initial scaffold did.

11 / VERIFIED RESULT

What I can substantiate at the public commit.

Test Files  6 passed (6)
Tests       40 passed (40)

$ pnpm typecheck
$ tsc -p tsconfig.json
# exit 0

$ pnpm harness validate tasks/kernel-0001.yaml
valid task manifest: kernel-0001 "Add agent event serialization"

I re-cloned the public repository at 88ef2f4, installed its locked dependencies, and reran the checks. The test fixtures inherit a developer’s global Git signing setting, so I disabled signing for their temporary repositories; with that environment isolation, all 40 tests passed. Type checking and manifest validation passed directly.

1. 1c17c71M0 foundation: events, kernel, policy, SDK, CLI, packages, docs, and infrastructure scaffold.
2. d2783c4Exit-gate fixes: path policy, event types, FakeModel accounting, kernel stop behavior, and pnpm build approval.
3. c831d57Report accuracy: parse Vitest’s Tests line instead of mistaking six test files for six tests.
4. 88ef2f4Public README and quick-start documentation.

12 / CURRENT TRUTH

M0 is a foundation, not the finished platform.

It would be easy to turn the directory names into a claim that the platform already has a distributed control plane, secure sandboxes, durable sessions, live protocols, and a web console. It does not. The table keeps working code and roadmap work separate.

The public roadmap names the sequence honestly: CI and SQLite first, then evaluation credibility and a web task board, followed by a real agent server, sandbox runner, provider adapter, control plane, Postgres, object storage, and only then a Kubernetes decision.

13 / FILE GUIDE

Where to read the implementation.

14 / NEXT DEVELOPMENT LOG

The next useful milestone is not more scaffolding.

The project now needs to make the two working slices meet. The operator loop should run in CI against a real base-branch diff, treat headless ask as blocked, persist events in SQLite, and execute the first evaluation scenario. After that, a real provider adapter can connect the manifest, kernel, policy decisions, and report into one auditable run.

1. 01Make the exit gate a required CI check.
2. 02Compare the task branch with its base, not only Git status.
3. 03Persist the typed event stream in SQLite.
4. 04Run one golden-repository evaluation end to end.
5. 05Wire a real model only after those controls are observable.

SOURCES + PROVENANCE

Primary material used for this note.

• Harness Platform at verified commit 88ef2f4
• Ollama’s Pi integration documentation
• Pi Agent Harness repository
• OpenCode repository
• Goose repository
• OpenHands repository

Build-session chronology comes from my attached local Pi transcript. Repository claims were checked against the public commit above and rerun locally on 30 August 2026.