Verify the docs before the agent writes a line

Hook

You dispatched an agent against a real ticket. It came back with three thousand lines of code, a clean diff, no syntax errors, and tests that imported nothing fictional. One hundred and twenty-one calls to throw new Response(). Fourteen references to rc.user. process.env in a runtime that does not expose it. The reviewer reading the diff would not have caught it. The framework caught it — at runtime, when the sandbox refused to load. This is not a broken agent. It is a known failure mode with a known fix.

Model memory is not a query

When the agent writes code against a framework, it is not looking up that framework's API. It is generating from its training-data impression of what frameworks like this look like. The output is plausible because plausibility is what the model is optimized for. The output is wrong because the model has never seen this framework — only frameworks like it.

Think of a sketch artist who has drawn ten thousand faces and is now drawing yours from a description. The result will look like a face. It will not look like your face. The pencil knows faces. The pencil does not know yours.

The agent reaches for throw new Response() because that surface exists in Cloudflare Workers. It reaches for process.env because that surface exists in Node. It reaches for rc.user because some auth library in its training data named the request context rc. None of those exist in Emdash. Authentication does not catch it. Code review does not catch it. Only the framework, at runtime, knows.

The fix is structural, not careful prompting

Four mechanics, each from the same principle: replace model memory with ground truth, and put the verification in the pipeline. The mechanics below are the worked example for one framework. The principle is what travels.

Mechanic 1 — the verified guide. Write docs/<framework>-GUIDE.md by reading the framework's source code, not its public docs. Public docs lag the source and omit exactly the surfaces the agent reaches for. Extract the response API, the request API, env access, the user/session API. One file. Real exports only.

Mechanic 2 — doc-reading as pipeline input. Every brief that touches the framework starts with one line:

Read `docs/EMDASH-GUIDE.md` before writing any code. Use only the
surfaces it documents. If you need a surface not in the guide, stop
and ask.

"See also docs/EMDASH-GUIDE.md" produces zero behavior change. The mandatory form above produces near-total compliance. The doc is now a tool, not a suggestion.

Mechanic 3 — the banned-patterns list. BANNED-PATTERNS.md enumerates the wrong surfaces the model reaches for in this framework. Three parts per entry: the wrong surface, the right surface, and one sentence on why the wrong one is appealing.

# BANNED-PATTERNS.md

## throw new Response()
Wrong: throw new Response("not found", { status: 404 })
Right: ctx.json({ error: "not found" }, 404)
Why appealing: works in Cloudflare Workers; Emdash uses ctx.json.

Grow the file from observed failures, not from imagination. When an agent ships a hallucination, add the pattern before the next dispatch.

Mechanic 4 — the QA grep. A pre-review script scans the staged diff for every banned pattern:

# scripts/qa-grep.sh
PATTERNS=("throw new Response" "rc\.user" "process\.env")
for pattern in "${PATTERNS[@]}"; do
  if git diff --cached | grep -qE "$pattern"; then
    echo "QA: banned pattern '$pattern' in staged diff"
    exit 1
  fi
done

This is not a linter. A linter checks for general code quality. This is a narrow pattern-match for one class of failure — hallucinated API surfaces — and it runs before any human reads the diff. Do not fold it into ESLint. The two tools answer different questions.

If you get stuck

Wrote the guide from public docs. Public docs omit the surfaces the model needs. Read the source. Check exports, not documentation pages.

Doc-reading mentioned but not mandatory. "See also" produces no behavior change. The brief must say Read this before writing any code. That phrasing moves compliance from near-zero to near-total.

BANNED-PATTERNS.md curated by speculation. Grow it from agent failures, not from imagination. Agent hallucinates, you add the entry, you re-dispatch. The file gets stronger every cycle.

Cross-reference

l7 establishes the brief as the unit of work; this lesson extends the same discipline to framework verification — the brief is where doc-reading becomes mandatory. l5 is the same problem at the tool-definition layer: descriptions that lie produce tools that fabricate.

Deliverable

Pick one framework or internal API you use with agents — yours, not Emdash. Write docs/<framework>-GUIDE.md by reading the source. Add the one-line doc-reading directive to your next brief. Run the agent. If the diff is clean, the discipline worked. If the diff contains a hallucinated surface, add it to BANNED-PATTERNS.md and re-dispatch. The file gets stronger with each failure. That is the loop.

Further reading

• "Code Review Is Not QA" — why catching hallucinated API surfaces is a structural problem, not a reviewer's vigilance problem.