Banned patterns: how skills shut down hallucination

An agent wrote 3,600 lines of code against a plugin API that does not exist. Every throw new Response() call, every rc.user reference, every process.env lookup — all plausible, all wrong. The model generated from training data, not from the actual sandbox. The problem was not the model's intelligence. It was the absence of a constraint file.

What this actually is

A banned patterns file is a skill file with a single section: a list of patterns that are known to be wrong in a specific environment, each paired with what to use instead. It is loaded into the agent's context before any code is written, and it is enforced by a grep gate that runs before human review.

The combination — context-window constraint plus automated gate — closes the hallucination loop. The agent cannot use patterns it has been explicitly told are wrong, and even if it does, the gate catches them before they become a human problem.

Walk-through

Here is the banned patterns file that would have caught the 3,600-line failure:

# BANNED-PATTERNS.md

# Banned Patterns — Emdash Plugin API

These patterns do not exist in the Emdash sandbox. Any occurrence fails QA automatically.

| Pattern | Why banned | Use instead |
|---------|-----------|-------------|
| `throw new Response(` | Not available in sandbox | `rc.sendError()` |
| `rc.user` | Property does not exist | `rc.member` |
| `process.env` | Environment variables not available | `rc.config.get('KEY')` |
| `fetch(` | Global not available | `rc.http()` |
| `console.log(` | Output not captured in sandbox | `rc.log()` |

And the grep gate:

# scripts/qa-banned.sh
#!/bin/bash
set -e

PATTERNS=(
  "throw new Response("
  "rc\.user"
  "process\.env"
  "fetch("
  "console\.log("
)

FOUND=0
for pattern in "${PATTERNS[@]}"; do
  if grep -rn --include="*.js" -e "$pattern" src/; then
    echo "BANNED PATTERN FOUND: $pattern"
    FOUND=1
  fi
done

if [ $FOUND -eq 1 ]; then
  echo "Fix banned patterns before submitting for review."
  exit 1
fi

echo "No banned patterns found."

The skill file that invokes the code-writing agent includes Read BANNED-PATTERNS.md before writing any code as step one of the procedure. Not as a suggestion — as step one.

Why sourcing matters

Banned patterns work only when they are sourced from actual failures, not from guesses about what might go wrong. The five patterns above came from five specific bugs found in production code. A banned patterns file written speculatively — "I imagine this API probably doesn't have X" — will miss the actual failure modes and catch things that are fine.

The sourcing discipline is: add a pattern to the file only after it has caused a bug that reached code review or beyond. The file should grow over time as the API's actual edges reveal themselves.

Extending to other environments

The same pattern applies to any constrained environment: a serverless runtime that does not have fs, a WordPress plugin sandbox that does not have $_SERVER, a browser extension context that does not have Node globals.

For each environment you regularly write agents for, keep a BANNED-PATTERNS.md in the project root. Add patterns as you discover them. Add the grep gate to your CI pipeline and to the skill file that invokes agents in that environment.

If you get stuck

If the grep gate produces false positives (catches patterns that are actually fine): tighten the pattern. fetch( is too broad if rc.fetch( is allowed — use ^[^.]fetch( with a word-boundary anchor, or exclude the allowed form explicitly.

If the banned patterns file is getting long: organize by category (network, DOM, environment, API). A long file is a good sign — it means the environment has real edges and you are documenting them.

If you are not sure whether a pattern is actually banned: test it in the sandbox. Do not add it to the file based on a guess.

Cross-reference

L6 (Encoding order and gotchas) covers the broader gotchas discipline that banned patterns extends into an automated gate.

Deliverable

A BANNED-PATTERNS.md with at least four entries, each sourced from real API behavior — a bug you found, a limitation in a framework you use, or an environment constraint documented in official docs. A scripts/qa-banned.sh that exits non-zero when any banned pattern appears. The success criterion: run the script against code you know is wrong and confirm it exits 1. Run it against clean code and confirm it exits 0.