What Are the 13 Probes Agentprobe Runs on a Seller URL?

Updated

Agentprobe runs 13 checks against a seller URL to determine whether an AI agent can discover, quote, and complete a purchase. The probes cover machine-readable discovery files, transactional APIs, payment and refund paths, and trust signals like fulfillment proof. Each probe maps to a specific capability an agent needs, and the combined results produce a score and tier (CERTIFIED, AGENT_READY, PARTIAL, or NOT_READY).

The 13 Probes and What Each One Checks

# Probe What it verifies Agent capability it enables
1 llms.txt A machine-readable file describing the site for LLMs Discovery and orientation
2 OpenAPI A published API specification Programmatic access to endpoints
3 ai-plugin.json A plugin manifest for AI integrations Agent tool/plugin registration
4 MCP Model Context Protocol support Standardized agent-to-service communication
5 commerce.json A commerce metadata file Structured product/commerce info
6 Catalog API An endpoint exposing product listings Browsing and selecting items
7 Quote API An endpoint returning pricing/quotes Getting a price before purchase
8 Checkout API An endpoint for completing an order Executing the transaction
9 Payment rails Supported payment mechanisms Paying for the order
10 Refund/contact Refund and contact paths Post-purchase recourse
11 Unsupported rails Declared unsupported payment methods Avoiding failed payment attempts
12 No fake claims Absence of unverifiable claims Trust and reliability
13 Fulfillment proof Evidence of fulfillment capability Confidence the order ships/completes

How the Probes Group Into Capability Layers

The 13 probes fall into four functional layers, which is why a site can score well on discovery but still fail to transact.

Discovery layer

  • llms.txt — tells an agent what the site is and where to look.
  • ai-plugin.json — registers the site as a usable tool for an agent.
  • MCP — exposes the site through a standard agent protocol.
  • commerce.json — declares commerce-specific metadata.

Access layer

  • OpenAPI — gives the agent a contract for calling endpoints.
  • Catalog API — lets the agent enumerate products.
  • Quote API — lets the agent retrieve a price.

Transaction layer

  • Checkout API — lets the agent place the order.
  • Payment rails — lets the agent pay.
  • Unsupported rails — prevents the agent from attempting payment methods the seller can't accept.

Trust layer

  • Refund/contact — gives the agent (and buyer) a recourse path.
  • No fake claims — checks that the site doesn't assert things it can't back up.
  • Fulfillment proof — shows the seller can actually deliver.

How the Probes Map to Tiers

Agentprobe converts the probe results into a score and a tier:

  • CERTIFIED (≥85/95) — fully agent-legible; the site passes the full set of probes.
  • AGENT_READY (70–84) — agents can transact with guidance; some probes may be missing or partial.
  • PARTIAL (40–69) — discoverable but not transactable; discovery-layer probes pass, transaction-layer probes don't.
  • NOT_READY (<40) — opaque to agents; most probes fail.

The tier boundaries explain the practical difference: a site can publish llms.txt and commerce.json (discovery) yet still land in PARTIAL if it lacks a checkout API or payment rails.

Using the Probe List to Self-Audit

To check a seller URL against the same criteria, work through the layers in order:

  1. Confirm discovery files exist — fetch /llms.txt, /ai-plugin.json, and any commerce.json path.
  2. Check for an API spec — look for an OpenAPI document describing catalog, quote, and checkout endpoints.
  3. Test the transactional endpoints — verify a catalog call returns products, a quote call returns a price, and a checkout call can create an order.
  4. Verify payment and refund paths — confirm which payment rails are supported and that a refund/contact route is documented.
  5. Check trust signals — ensure claims are verifiable and fulfillment evidence is present.

Each step corresponds to one or more of the 13 probes, so a gap at any layer shows up as a lower score and a lower tier.

Key Takeaway

The 13 probes are not a single checklist but a layered test: discovery files (llms.txt, ai-plugin.json, MCP, commerce.json), access APIs (OpenAPI, catalog, quote), transaction paths (checkout, payment rails, unsupported rails), and trust signals (refund/contact, no fake claims, fulfillment proof). A seller that passes only the first layer is discoverable but not transactable — which is exactly what the PARTIAL tier captures.