Your Tested Prompt Library

What you ship

A directory — prompts/ in your own repo or wherever you keep working files — containing exactly five prompt files. Each file holds the current prompt, the version that came before it, a one-paragraph rationale explaining why this prompt exists and not a simpler one, at least one eval test case with a named expected output, and a one-line description for the README.

The README is part of the deliverable. It should be short enough that a colleague — someone who did not write these prompts — can clone the directory, read the README, and run an eval in under ten minutes. If they can't, the library is not done.

Why five

Five prompts fit in your head at once. You can name all five without looking at the file. You know which one to reach for when a task arrives. Ten prompts means you have to search; a library you search is a library you stop maintaining.

The constraint does something else. When you limit yourself to five, the question is no longer "should I add this?" The question becomes "does this replace one of the five, or does the one it would replace deserve to stay?" That discipline — the forced trade — is what the library is actually training. The sixth-best prompt reveals which five are real.

Pick the workflow first

Choose one workflow: weekly investor update drafts, customer-support triage, PR review comments, content editing passes, meeting notes synthesis into action items. Not two. One.

Three candidates worth considering: support triage (you do it under time pressure, consistency matters, bad outputs have visible costs), PR review (the task is well-defined, the expected output shape is stable, you have a ready corpus of real PRs to use as eval inputs), content editing (the feedback you give is probably more idiosyncratic than you think, which makes a rationale valuable). Pick the one where a bad prompt costs you something. Pain is a good eval signal.

What each prompt contains

For every entry:

The prompt itself (versioned). The actual text, in a file named <slug>/v2.md or similar. Not paraphrased. Not summarized. The prompt you paste or inject.

A one-paragraph rationale. Why this prompt and not a one-sentence instruction? What did you try before this? What failure mode did you patch? The rationale is the diff annotation for a change that happened in your head instead of in code.

At least one eval test case. A named input and an expected output behavior — not the exact output word-for-word, but a behavioral description: "response does not exceed three bullet points," "the draft opens with a data point, not a greeting," "no recommendation is offered without a cited source." Run the eval by hand and record whether the prompt passed. Date the run.

A rollback prompt. The previous version, in <slug>/v1.md. If you are writing v1 — if there is no previous version — write a minimal baseline prompt first, run it, then write the prompt you actually want to use as v2. You need a rollback because the next version you write might break the eval that v2 passed.

A one-line description. Goes in the README. Twenty words or fewer. Reads like a function signature, not a feature announcement.

Build sequence

1. Name the workflow. Write it on a single line: "Workflow: PR review comments for backend changes." If you can't write it in one line, it isn't a workflow yet.
2. List every prompt you currently use — or could imagine using — for that workflow. Write the list fast; don't edit it yet.
3. Cut the list to five candidates. Keep only prompts you have actually used or immediately plan to use. Strike anything you wrote "just in case."
4. For each of the five, write the rationale before you write the eval. The rationale forces you to name what the prompt is supposed to do. Without it, an eval passing tells you nothing.
5. Write the eval test case for the first prompt. Name the input. Name the expected behavior. Run the prompt against the input. Record pass or fail and the date.
6. Capture the rollback. If there is no prior version, write a minimal baseline first and call it v1. The prompt you're actually building is v2.
7. Commit the first prompt — rationale, eval result, v1, v2, one-line description — before moving to the second. Each prompt is a discrete unit. Don't batch.
8. Repeat steps 4–7 for prompts two through five, in order.
9. Write the README. Colleague installs and runs an eval in ten minutes. Test this by actually reading the README cold, not by imagining a colleague.
10. Run every eval one more time from a fresh context. No cached outputs. Record results.
11. Confirm the rollback works for at least one prompt. Change v2 back to v1, run the eval, see what breaks. Restore v2. Now you know the rollback is real.
12. Tag the commit: v1.0 — five prompts, all evals run, rollback verified. The tag is the deliverable marker.

Success criteria

• Every prompt has an eval that has been run at least once, with a recorded result and a date.
• The library has a README a colleague could use without asking you for help.
• The rollback path has been tested — not just documented — for at least one prompt.
• No prompt appears in the library without a rationale.
• You can name all five prompts and their workflows without opening the directory.
• The commit history shows each prompt was committed individually, not in a single batch.

What to cut

Prompts you have used once. A prompt you ran once to solve a one-time problem is not a library entry. It is a note. Put it in a notes file if you want to keep it. It has no eval corpus, no version history, and no rollback value because it has no maintenance obligation. Including it in the library adds noise and nothing else.

Prompts that work but you can't explain why. If you can't write the rationale — if you don't know what failure mode the prompt is solving — then you don't own the prompt yet. When it breaks, you will have no diagnosis. When you want to improve it, you will have nothing to improve against. A prompt without a rationale is a dependency you can't audit.

Prompts someone else wrote that you adopted without testing. Borrowed prompts that you haven't run against your own eval cases are guesses. They may have been designed for a different model, a different output shape, a different failure mode. Until you run your evals against them, you don't know what you have. Adopt them by testing them, not by copying them.

Common ways this goes wrong

You added a prompt because it might be useful. The library is now six entries, the sixth has no eval and no rationale, and three months from now you don't remember whether you tested it. Recovery: enforce the five-prompt constraint before you commit anything. If you want to add a sixth, decide which of the five it replaces first.

Your eval passes but the prompt drifted from the rationale. You edited the prompt to fix a new failure mode, the eval still passes, but the rationale describes the old version. The prompt and the rationale are now out of sync. Someone reading the file — including future you — will not understand why the prompt is the way it is. Recovery: update the rationale every time you update the prompt. They should describe the same artifact.

You skipped the rollback step on the third prompt. You wrote v1, wrote v2, committed v2, moved on. v1 doesn't exist as a file. When v2 breaks, you don't have a working fallback. Recovery: before you edit any prompt, copy the current version to the v-minus-one file first. Commit the backup before the change. This takes thirty seconds and it's the only reason a rollback works when you need it.

Stretch goals

Add a CI gate that runs evals on every pull request. The eval script reads each prompt file, runs the test case, and fails the build if the behavioral expectation is not met. This turns the library from a thing you maintain manually into a thing that tells you when it's broken.

Expose the prompt library as a config file consumable by other tools — a script that reads prompts/<slug>/v2.md and injects it at runtime. Now the library is not just documentation; it is the actual prompt your systems use. Changing the file changes the behavior in production. The evals become integration tests.

Run a quarterly audit. Set a calendar event: once every three months, open the library and ask two questions about each prompt. Has this prompt run in the last month? Has the rationale changed? If a prompt hasn't been used in ninety days, remove it. If the rationale has changed but the file hasn't, update the file. The audit keeps the library alive rather than archived.

What good looks like

The library is small. The author can name what each prompt does in one breath, without reading the file. The evals have been run, recently, and the dates are recorded. The README opens with what the library covers and ends with how to run an eval — in that order, with no preamble. Three months from now the library has either grown by one entry, because a new workflow earned a slot, or shrunk by one, because a prompt stopped being used and the quarterly audit caught it. It is not a record of prompts the author once thought were good. It is the prompts they are actually using right now, and the evidence that those prompts work.