Failure modes in chains

A single prompt either works or fails. You can see which. A chain can fail invisibly: step 1 produces something plausible-looking but wrong, step 2 ingests it without complaint, and the final output is confident garbage. The chain itself reports no error because no individual step threw one.

This is not a hypothetical. It is the dominant failure mode in production chains once you move past toy examples.

The three failure modes, named precisely

Silent garbage propagation. Step 1 misclassifies an input or hallucinates a field. Step 2 receives the result and operates on it as if it were correct. The error compounds at each step. The final output has no visible relationship to where the error entered. The symptom: a support ticket classified as "access" gets routed to the billing team, every time, for a specific class of ticket that contains both keywords. No error is logged anywhere.

Schema drift. You update step 1's prompt to rename a field — say, ticket_type becomes category. Step 2's prompt still references ticket_type. Both calls succeed. The field is null in step 2's context. Depending on the prompt, step 2 may invent a value rather than report the absence. The symptom: a field that is mysteriously null in the final output even though step 1 clearly produced something. You look at step 2's logs and see it received the data — but under the old field name.

Context loss. Step 2 needs information that was available in the original input but that step 1 did not pass forward. Rather than failing, it constructs a plausible substitute — which is a hallucination with a structural cause. The symptom: step 2 returns a customer ID that looks correct but is from a different customer. The original customer ID was in the raw ticket but was not included in step 1's output schema.

The gate for each failure mode

For silent garbage propagation — add an assertion step between steps 1 and 2. This is a lightweight prompt that reads step 1's output and verifies it against known constraints before step 2 runs.

# prompts/assert-claims-v1.txt
You are a validator. Check the following JSON against these rules:
1. "claims" is a non-empty array of strings
2. Each string is a verbatim customer statement, not a paraphrase or category label
3. "ticket_id" matches the format "t-NNNN"

If all rules pass, return: {"valid": true}
If any rule fails, return: {"valid": false, "reason": "..."}

Input: {{step1_output}}

# → example failure output from assertion step
{
  "valid": false,
  "reason": "claims[1] appears to be a category label ('billing issue') rather than a verbatim customer statement"
}

When valid is false, stop the chain and log the reason. Do not pass forward. The silence is broken — you now have a specific, actionable failure message instead of garbage at the end.

For schema drift — version your schemas alongside your prompts. When a prompt update changes the output shape, bump the schema version and review every downstream step that consumes it.

# schemas/claims-v2.json  ← bumped when "ticket_type" became "category"

The version number is not documentation. It is a dependency declaration. Any CI check that runs against the prompt also validates that the schema version matches what downstream steps declare as their expected input. l7 of this course shows how to wire this into a CI gate.

For context loss — include a _passthrough field in step 1's output schema, populated before step 1 runs by the application layer.

"_passthrough": {
  "raw_ticket": "I was charged twice on March 4th...",
  "customer_id": "cust-4421",
  "account_age_days": 847
}

Step 2 reads _passthrough.customer_id if it needs it. If that field is null, it reports null — it does not invent a customer ID. The _passthrough pattern is described in l2; this lesson shows why it exists as a structural defense against a specific failure mode, not just a convenience.

Cross-reference

prompt-engineering-101 m3/l12 covers hallucination prevention in a single prompt. This lesson is about the structural causes of hallucination that emerge specifically from chains — causes that single-prompt techniques cannot address. The three failure modes here map directly to the three gates in your assertion layer.

If you get stuck

Schema drift is the failure mode that blindsides experienced engineers most often. The symptom is a field that is mysteriously null in the final output even though you know step 1 should have produced it. The diagnostic: log step 1's raw output and compare field names character-for-character against what step 2's prompt references. The mismatch is usually obvious once you look at it side by side.

If you cannot reproduce a failure in isolation, replay the exact step 1 output against step 2 directly, bypassing the application layer. If the failure disappears, the application layer is mangling the artifact between steps — look at how the JSON is serialized and deserialized. Unicode escaping and float precision are common culprits.

Deliverable

For a 2-step chain you have built or are building, write the assertion check between steps 1 and 2. It can be a prompt (like the validator above) or a code check (like the Python assertions in l2). The check must be specific enough to catch each of the three failure modes described here: a content check for garbage propagation, a field-name check for schema drift, and a null check on any field that downstream steps depend on but did not originate in step 1.

Further reading

• "Failure, Visible Three Ways" — three layers at which a chain can fail silently and the instrumentation that surfaces each one.
• "The Wrong Probe" — what happens when one bad diagnostic ("use case not supported") gets canonised into release notes, and the discipline that catches that class of failure earlier.