Public Surfaces for AI Agents

How to design the public-facing surfaces — well-known files, MCP endpoints, robot-readable docs — so agents can use your product without a human in the loop.

Principle

For an AI agent to discover and query your site with no credentials, three things have to be in place: a public read-only endpoint, a discovery file, and a formatter that strips admin data. Skip any one and the loop is open. The architectural choices that make this safe are not the same as the choices that make a normal API safe — agents have less access than human visitors, not more, and the controls that matter live in different places.

Techniques

1. Recognize the asymmetry

A human visitor has cookies, a session, the ability to scroll a site and figure out what's offered. An agent has none of that. Without a deliberate public surface, your site is a 401 wall to every agent that didn't come pre-credentialed — which is most of them, including every AI client a prospective customer might use.

The asymmetry runs the other direction from the usual API security stance. You're not protecting against agents probing too deep; you're deciding whether agents can see anything at all.

2. Build the discovery loop in three steps

The full loop: visit URL → fetch /.well-known/mcp.json → query the public endpoint. Each step is required.

Without the URL the agent has no entry point.
Without the discovery file the agent has the entry but doesn't know what's behind it.
Without the public endpoint the discovery file points to a 401.

Most "agent commerce" implementations stop at step three (authenticated MCP, which works for the site owner). The first two steps are what closes the loop for prospects.

3. Make formatters the security boundary, not permission checks

A permission_callback controls who reaches the endpoint. A formatter controls what the endpoint is capable of returning. They're different controls with different guarantees.

Write a separate public_format() method, not an is_public flag on your existing formatter. The flag is a bug magnet — it can be flipped, bypassed, or default-wrong on a future refactor. The separate method is structural: there is no code path by which it returns admin data, regardless of who reaches it.

The audit question is: "if every permission check failed open today, what would leak?" If the answer involves a formatter that conditionally hides fields, the formatter is too clever. The formatter should not conditionally hide; it should structurally lack the ability to expose.

4. Default OFF for public surfaces

Public surfaces should never be implicit. Make the admin opt in via a deliberate toggle, with help text that explains what data the toggle exposes. The default is the safe state; the active state is the considered choice.

Pair the global toggle with a per-resource opt-out so the admin can hide specific items even when the surface is on. Some catalog items are B2B-only, internal pricing, or in transition — the per-resource control gives the admin a scalpel where the global toggle is a switch.

When the toggle is OFF, the discovery file should also 404 — its existence is itself opt-in. Otherwise the file leaks the existence of an endpoint that's supposed to be hidden.

5. Whitelist tools, never blacklist

The public tool registry should list specific tools by name. Do not start with the full registry and remove sensitive ones — that pattern leaks the moment a new tool is added without a corresponding removal.

A separate PublicTools class with 2–4 explicitly named tools is the safe shape. The authenticated registry might have 47 tools; the public one has 4. The discrepancy is the security control.

6. Rate limit by IP, cache aggressively

Public endpoints have no token to key rate limits on, so key on IP. 60 req/min is a reasonable default; tune based on traffic.

Catalog data doesn't change often. Add Cache-Control: public, max-age=300 headers so a CDN can absorb most traffic. Without cache headers, every agent query hits your database — and agents query a lot.

When to Use

Any commerce-adjacent surface where prospective customers might ask AI clients about your offerings (memberships, products, pricing, services)
Documentation surfaces where agents could fetch reference material on behalf of users
Public APIs where you want AI clients as first-class consumers, not afterthoughts
Any surface where "AI-discoverability" is becoming a competitive feature

When Not to Use

Surfaces with any PII, financial data, or per-user state — those belong behind authentication
Internal tools, B2B portals, or admin interfaces — these have audiences that already have credentials
Surfaces under regulatory constraints (HIPAA, PCI, GDPR-sensitive) where exposure of even structured catalog data could be problematic — review with legal before shipping

The Audit Question

Before shipping a public surface, ask: "If every permission check failed open today, what data would leak?"

If the answer is "nothing, because the formatter doesn't have admin fields" — you're done. If the answer involves any conditional logic — the formatter is too clever, and you have work to do.

The right answer comes from the formatter being structurally limited, not from a chain of checks that all have to hold.