Build a Repo-Native Agent

An agent that runs unattended against your own repo, with tools you wrote, tests you can show, and recovery paths for the failures you can name in advance.

What you ship

A single-command agent that runs against your own repository, does one defined job, and either finishes cleanly or fails safely with recoverable state. The agent invokes at least two custom tools you authored, uses a checkpoint file so a mid-run crash does not restart work from zero, and has a test suite that covers both the path where everything works and the path where something breaks.

The agent is not a demo. When you finish, you can hand the run command to a colleague with no explanation beyond "this is what it does and this is where the log lives." If the only evidence it works is that it worked once on your machine, it is not done.

Why this is the capstone

Every lesson in this track was building toward one discipline: the unhappy path is not an edge case, it is the plan. A sub-agent brief with no stop condition loops. A tool with no error return leaves the model reasoning about nothing. A long-running operation with no checkpoint loses work at exactly the moment that matters most.

Most learners ship agents that pass the happy path and stop there. The happy path is easy to reach and easy to verify because it is the path you designed for. The unhappy paths — malformed tool output, a hung sub-agent, a crash at step 47 of 50 — are the ones that surface at 3 a.m. when no one is watching. Designing for them first, before a single line of implementation exists, is what separates an agent you can leave running from one you have to babysit.

This project requires you to write the failure-mode tests before you write the implementation they test. Not as a formality. As the design document. If you cannot write a test for a failure mode, you do not understand the failure mode well enough to defend against it.

Pick your task carefully

The agent does exactly one thing on your repository. The scope is non-negotiable. A narrow scope is not a limitation of the project — it is the project. An agent with a well-defined, testable job is useful. An agent that "helps with the repo" is a liability.

Three concrete tasks that work well:

Dependency audit with safe rollback. The agent reads your package.json or Gemfile, identifies dependencies with published patch or minor updates, upgrades each one, runs the test suite, and rolls back any upgrade that causes a test failure. It writes a structured report of what it upgraded, what it rolled back, and why.

Regression coverage gap report. The agent maps every exported function in a target module against the test files, identifies functions with no test coverage, and outputs a prioritized list with one suggested test case per gap. It does not write the tests — it reports the gaps so a human decides what to cover next.

License audit. The agent reads your dependency tree, fetches the license for each package, compares against an allowlist you define, and flags any dependency whose license is missing or outside the allowed set. It outputs a structured report and exits non-zero if any violations are found.

The rule for picking your own task: you must be able to write a test for its output before you write the agent. If the output is not machine-verifiable — if "you'll know it worked when you look at it" is the only check — pick a different task.

What the agent contains

Entry point. One command invokes the full agent, accepts the repository path as a required argument, and exits 0 on success or non-zero on any unrecovered failure. No interactive prompts during the run. All configuration is passed at invocation or read from a config file the entry point validates before the agent starts.

Two custom tools. Each tool has a description written as a decision rule ("Call this when..."), a typed input schema, a structured success return, and an explicit error return — never an unhandled exception. The model reads tool errors and decides what to do; a crash that returns nothing gives the model nothing to work with.

MCP server (if your task touches external state). If the agent needs to query a package registry, call a license API, or read a remote resource, wrap those calls in an MCP server registered in .mcp.json. The server handles authentication via environment variable, never hardcoded. If your task is fully local, an MCP server is not required — but you must state in your project README why it was not needed.

Test suite. A test file that runs with one command and covers: the happy path end-to-end, at least two tool-level unit tests for each custom tool (one success, one failure), and at least two failure-mode scenarios from the section below. The test suite exits 0 or non-zero cleanly. "Run it and look" is not a test.

Checkpoint and recovery. The agent writes a checkpoint file before each destructive or stateful operation. If the run is interrupted and restarted, it resumes from the last clean checkpoint. The checkpoint schema includes a schema_version field, a step index, a list of completed steps, a list of errors, and the next input. A schema mismatch between runs causes a clean restart with a logged warning, not a silent resume against a stale file.

Build sequence

Each step has a completion criterion. The step is not done until the criterion is met. Do not advance.

Write the output spec. Define the exact structure the agent produces on success and on failure — JSON schema or typed interface, not prose. Completion: the spec is committed as agent/output-schema.json and passes a JSON Schema validator.
Write the failure-mode tests first. For each failure mode in the section below, write the test before any implementation exists. The tests will fail. They are supposed to fail. Completion: npm test runs and reports the expected failures — no syntax errors, no import errors, only assertion failures.
Implement custom tool 1. Write the tool definition and its handler. Completion: the two tool-level unit tests for this tool pass — one success path, one error path.
Implement custom tool 2. Same standard. Completion: both unit tests pass.
Write the checkpoint read/write functions. Implement loadCheckpoint() and saveCheckpoint() with schema version validation. Completion: a test that writes a checkpoint, kills the process, restarts, and verifies the resumed state matches what was written.
Implement the agent loop. Wire the entry point, tools, and checkpoint together. No MCP server yet. Completion: the happy-path end-to-end test passes against a fixture repository — a small repo you control that has known properties your agent can verify.
Add the MCP server (if applicable). Register it in .mcp.json, verify the tool appears in a fresh session. Completion: a session that discovers the MCP tool automatically, with no manual tool registration.
Run the failure-mode tests. They were written in step 2. Now they should pass. Completion: all failure-mode tests exit 0.
Run against a real repository that has at least one known failure condition. This is the adversarial test. Prepare the repo in advance: add a dependency with a disallowed license, leave a load-bearing function without tests, introduce a package with a patch update that breaks a test. Completion: the agent correctly identifies the failure, reports it in the structured output format, and exits non-zero.
Read the log. Run the agent one more time. Read every line of the log output. If any line is ambiguous — if you cannot tell from the log alone what state the agent was in and what it decided — fix the log message. Completion: the log is unambiguous.

Success criteria

These are observable. Run the agent, inspect the output, read the criterion. Each one either passes or it does not.

./agent.sh --repo /path/to/test-fixture exits 0, and the output matches the schema defined in agent/output-schema.json as validated by a schema validator — not by inspection.
Kill the agent process at step 3 of a 10-step run. Restart. The agent resumes from step 4, not step 1, and the completed-steps list in the checkpoint reflects what was done before the kill.
Run the agent against a repo where tool 1 returns a malformed response (inject the malformation via a test fixture). The agent logs the error in structured format, does not proceed past the failed step, and exits non-zero.
Run the agent against the adversarial repository prepared in step 9 of the build sequence. The structured output identifies the known failure condition correctly. The agent does not modify any file in the repository as a result of finding the failure.
Run the agent twice in sequence on the same repository with no changes between runs. The output is identical. The agent does not create duplicate records, duplicate log entries, or partial state that differs between runs.
Delete the checkpoint file mid-run by running rm checkpoint.json in a separate terminal. The agent detects the missing checkpoint on its next write cycle, logs a warning, and either recovers to a safe state or exits non-zero — it does not silently continue as though the checkpoint exists.
Run npm test. Every test passes. The test command is in the project README and runs without setup beyond what the README documents.

Failure modes you must defend against

These are not hypotheticals. Design the handling before the first line of implementation.

Tool returns malformed output. Symptom: the model receives a tool result that does not match the expected schema — a missing required field, a type mismatch, a truncated JSON object. The test: inject a tool stub that returns a malformed response and assert the agent logs the schema violation, does not attempt to proceed using the partial result, and exits non-zero. Recovery: the tool handler validates its own output before returning. On validation failure, it returns a structured error with the violation described, not a partial object.

Sub-agent hangs. Symptom: a dispatched sub-agent does not return within an expected window — the parent has been waiting and no result has come back. The test: dispatch against a stub that sleeps indefinitely and assert the parent times out, logs the hang, and either retries once or exits non-zero. Recovery: every sub-agent dispatch in the agent loop has an explicit timeout. Exceeding it is an error, not a wait state.

Partial-state crash mid-checkpoint. Symptom: the agent writes the checkpoint header but crashes before writing the completed-steps list — the checkpoint file exists but is incomplete JSON. The test: write a fixture checkpoint file with truncated JSON and assert loadCheckpoint() detects the parse failure, logs a warning, and returns the zero state rather than crashing or resuming from corrupt data. Recovery: loadCheckpoint() wraps the parse in a try/catch. Any parse failure returns the zero state and logs the file path and error.

Schema-version mismatch on resume. Symptom: a checkpoint written by an older version of the agent is present when a newer version starts. The schema_version field does not match. The test: write a checkpoint with schema_version: 0 and start the agent which expects version 1. Assert the agent logs the mismatch, discards the stale checkpoint, and starts from zero. Recovery: the mismatch is not a crash condition. It is a known state: prior work is lost, the run restarts cleanly, and the log records why.

Idempotency violation on retry. Symptom: a step that was completed before a crash is re-executed on resume because it was not in the completed list when the checkpoint was written. If the operation is not idempotent — if running it twice produces a different result or an error — the resume creates bad state. The test: run a step, add it to the completed list, save the checkpoint, then run the same step again via a direct function call and assert the result is identical to the first run. Recovery: every stateful operation in the agent loop must be idempotent. Verify this in code review before the first unattended run. If an operation cannot be made idempotent, wrap it in a pre-execution check that detects and skips already-completed work.

Test gates

The agent is not shippable until all of these pass in a single npm test run with no manual setup beyond what the README documents.

Unit tests for each custom tool: success path and error path. Two tests minimum per tool.
Checkpoint write, corrupt-read, and schema-mismatch tests. Three tests minimum.
Happy-path end-to-end test against a fixture repository with known properties.
At least two failure-mode tests from the section above. All five if the build time permits.
Idempotency test: the same step run twice produces the same output.
Output schema validation test: the agent's success output passes the JSON Schema validator in agent/output-schema.json.

If any gate is missing, document it as a known gap in the README with the reason it was omitted and the risk it introduces. A missing test is not a ship blocker if the risk is named and accepted. A missing test that is unnamed is.

Stretch goals

Parallel sub-agents with worktrees. If your task can be decomposed across repository subdirectories — separate packages in a monorepo, separate modules in a large codebase — dispatch a sub-agent per directory, each running in its own worktree, collecting results in parallel. The parent integration step merges structured outputs. The test: verify the worktrees are cleaned up after the run regardless of whether the agent succeeded or failed.

Formal pre/post-conditions. Before the agent starts its loop, it runs a pre-condition check that asserts the repository is in a known valid state — the test suite passes, the dependency manifest is parseable, the checkpoint schema version matches. After the loop completes, it runs a post-condition check that asserts the output matches the spec and no files outside the defined scope were modified. Both checks exit non-zero on failure. Neither check modifies state — they are read-only assertions.

Observability instrumentation. Every tool call emits a structured log line: timestamp, tool name, input hash, output schema validity, duration in milliseconds. Every checkpoint write emits a log line with the step index and completed count. The logs are machine-readable JSON, one object per line. After a run, you can pipe the log through jq and answer: which step took the longest, which tool was called most often, which step was retried.

What good looks like

The agent runs at 3 a.m. while you sleep. At step 6 of 12 it receives a malformed response from a tool — a dependency registry returned HTML instead of JSON because the endpoint rate-limited the request. The agent detects the schema violation, writes the failure to the checkpoint's error list, logs a structured entry with the response excerpt and the step index, and exits non-zero. The worktree is in a clean state. No files were modified after the failure. In the morning you read the log over coffee, see exactly what broke and at what step, fix the rate-limiting configuration, and re-run from the checkpoint. The agent picks up at step 6, completes the remaining six steps, and exits 0. That is the bar. An agent that works once when you are watching it is a prototype. An agent that fails safely when you are not watching it, and tells you everything you need to know to fix it, is the deliverable.