haystack.sh

Haystack is a pre-production verification platform that gives every code change its own copy of production and replays real customer journeys against it to expose what breaks. It's aimed at teams whose failures slip past conventional tests — the quiet kind where nothing throws, every request returns 200, and the test suite still passes. According to the site, Haystack is early and taking on a small number of teams, so access is via early signup or a demo rather than open self-serve.

How it works

The workflow has four moving parts, and the ordering matters: cases are derived from the change, environments are forked per case, then each case runs twice so the change is the only variable.

Blast radius

Haystack reads the diff and works out every case the change can affect. The cases are your customers' real journeys, learned from production traffic, narrowed to the ones the change can reach — not scenarios someone invented for a test.

In the worked example on the site, a permissions refactor touching two files reaches 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. You do not write the cases.

Fork your infrastructure

Haystack snapshots your services and the data behind them — Postgres and Redis through to queues and object storage — then forks that snapshot once per case. The example: 14 copies of prod, ready in 26 seconds. These are real environments, not mocks, and nothing inside one reaches prod.

Run every case at once

Each environment is seeded into a different state: a workspace over its seat limit, an invite nobody accepted, a member whose role just changed. Every environment runs twice — with and without your change — so any difference is caused by the change.

See what actually broke

Failures come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it and look around.

Your agent runs the loop

The failing case goes back to your coding agent in a form it can act on. It fixes, re-runs, and keeps going until nothing fails.

Why it catches what tests miss

The failures that reach production are usually quiet. In the site's worked example, a frontend stopped sending a member's role and the access API treated a missing role as editor — so anyone with a pending invite could edit and delete anything in the workspace. Nothing threw, every request returned 200, and the test suite passed.

That example is the clearest statement of what Haystack is for: it targets silent behavioral regressions that don't surface as errors, which is exactly the class of bug a passing test suite can't rule out.

Using it

The CLI is haystack verify. The site points to three references for the details:

Getting started

Haystack is early and taking on a small number of teams. You can sign up for early access at https://haystack.sh/ or book a demo.

Who this fits

Haystack is worth evaluating if your changes touch shared services and data — permissions, roles, invite flows, anything where a missing field silently changes behavior — and your existing tests keep passing while production breaks. It's a weaker fit if your changes are isolated enough that a conventional suite already covers the reachable cases, or if you need open self-serve access today, since the site describes early access for a limited number of teams rather than general availability.

Haystack verifies a code change by reading your diff, forking a copy of production for every customer journey that change can reach, and running each case with and without the change so any difference is attributable to it. The main entry point is the CLI command haystack verify. It fits teams whose changes touch shared services, permissions, or data flows where a passing test suite has not been enough. Haystack is early and onboarding a small number of teams, so access starts with early access signup or a demo booking rather than open self-serve use.

What Haystack does with your change

The workflow has four moving parts, and you only supply the change itself.

• Blast radius. Haystack reads the diff and works out every case the change can affect. The cases are real customer journeys learned from production traffic, narrowed to the ones the change can reach — not scenarios someone invented for a test. In one worked example, a permissions refactor touching two files reached 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. You do not write the cases.
• Fork your infrastructure. Haystack snapshots your services and the data behind them — Postgres, Redis, queues, object storage — then forks that snapshot once per case. In the same example, 14 copies of prod were ready in 26 seconds. These are real environments, not mocks, and nothing inside one reaches prod.
• Run every case at once. Each environment is seeded into a different state: a workspace over its seat limit, an invite nobody accepted, a member whose role just changed. Every environment runs twice, with and without your change, so any difference is caused by the change.
• See what actually broke. Failures come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it.

Running the verification

The command to start with is:

haystack verify

The site points to haystack verify for the command and its flags, so check that reference for the exact options your version accepts rather than assuming flag names. The expected result is a set of runs, one per affected case, each executed in its own forked environment with and without your change.

To read what came back, the site directs you to Working with runs. That is where you interpret results: which cases failed, and the evidence attached to each failure — the before/after screen, the changed network calls, and the events that stopped firing.

Closing the loop with your coding agent

The failing case goes back to your coding agent in a form it can act on. The agent fixes, re-runs, and keeps going until nothing fails. The site covers this under Agents, which documents the MCP server and the agent loop.

So the full cycle is: read the diff and compute blast radius → fork production snapshots → run all cases in parallel → inspect failure evidence → hand the failing case to your agent → fix and re-run until clean.

Why this catches what tests miss

The failures that reach production are usually quiet. In the worked example, a frontend stopped sending a member's role and the access API treated a missing role as editor, so anyone with a pending invite could edit and delete anything in the workspace. Nothing threw, every request returned 200, and the test suite passed.

That is the gap Haystack targets: a change that is technically valid, passes assertions, and still breaks a real journey. Because each case runs against a real forked environment seeded into a specific state, and runs twice with and without the change, the difference shows up even when no error is raised.

Getting started

Haystack is early and taking on a small number of teams. Sign up for early access at https://haystack.sh/ or book a demo. Until you are onboarded, the CLI, runs, and agent loop described above are the pieces to plan around — the site's own references for haystack verify, Working with runs, and Agents are the places to confirm current command details and flags.

Haystack catches failures that tests miss because it runs each change against a real copy of your production environment and replays real customer journeys, then compares runs with and without the change. The failures that reach production are usually quiet: nothing throws, requests return 200, and the test suite passes, but behavior has already gone wrong. Haystack is built to surface exactly that class of silent regression, and it is currently early and taking on a small number of teams.

The failures that matter are usually silent

The clearest illustration comes from Haystack's own worked example. A permissions refactor touched two files. In the resulting failure, a frontend stopped sending a member's role, and the access API treated a missing role as editor. The practical effect: anyone with a pending invite could edit and delete anything in the workspace.

Nothing in that chain raised an error. Every request returned 200. The test suite passed. That is the shape of the problem Haystack targets — not crashes, but behavior that quietly diverges from what you intended.

How it finds them

It derives cases from real traffic, not invented scenarios

Haystack reads the diff and works out every case the change can affect. Those cases are your customers' real journeys, learned from production traffic, narrowed to the ones the change can actually reach — not scenarios somebody wrote by hand for a test.

In the worked example, a permissions refactor touching two files reached 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. You do not write the cases.

It forks real infrastructure instead of mocking

Haystack snapshots your services and the data behind them — Postgres and Redis, queues, object storage — then forks that snapshot once per case. In the example, that produced 14 copies of prod, ready in 26 seconds. These are real environments, not mocks, and nothing that happens inside one reaches prod.

It compares with and without the change

Each environment is seeded into a different state: a workspace over its seat limit, an invite nobody accepted, a member whose role just changed. Every environment runs twice — with and without your change — so any difference is caused by the change. That controlled comparison is what turns a vague "something looks off" into a specific, attributable regression.

It gives you evidence you can inspect

Failures come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it. For the permissions example, that means seeing the missing role on the wire and the access API's fallback behavior, rather than inferring it from a stack trace that never existed.

What this means for your workflow

The failing case goes back to your coding agent in a form it can act on. The agent fixes, re-runs, and keeps going until nothing fails. The CLI entry point is haystack verify; results are read through runs, and the agent loop runs over an MCP server.

If you are evaluating whether Haystack is worth attention, the relevant question is whether your risk lives in silent behavior changes across services and data states. If your failures tend to be loud — exceptions, failed assertions, obvious 500s — a conventional test suite may already cover you. If your failures tend to look like the permissions example, where everything returns 200 and the suite is green, that is the gap Haystack is designed to close.

Haystack is early and taking on a small number of teams. You can sign up for early access at https://haystack.sh/ or book a demo.

Haystack is in early access and is onboarding a small number of teams, so there is no self-serve signup flow described on the site. To get in, you sign up for early access at https://haystack.sh/ or book a demo. Both routes are listed under the site's "Getting started" section, and neither is described as instant — expect to be evaluated as part of a limited cohort rather than getting immediate access.

What you are actually requesting access to

Haystack is a pre-production verification tool. It gives every code change its own copy of production and replays real customer journeys against it to expose what breaks, before customers see it. That framing matters for access: this is not a product you can meaningfully trial on a toy repo, since its value comes from reading your diff, forking your real infrastructure, and replaying your production traffic.

The two access routes

The site does not state pricing, seat limits, or contract terms for either route. If cost or procurement terms are a deciding factor, treat the demo or signup conversation as the place to get them — nothing in the public material answers it.

What happens once you have access

Access leads into a CLI-driven workflow, so it helps to know what you are signing up for:

1. Point Haystack at a change. It reads the diff and works out every case the change can affect. The cases are your customers' real journeys, learned from production traffic and narrowed to what the change can reach — you do not write them.
2. Fork your infrastructure. Haystack snapshots your services and their data (Postgres, Redis, queues, object storage) and forks that snapshot once per case. In the site's worked example, 14 copies of prod were ready in 26 seconds.
3. Run every case at once. Each environment is seeded into a different state — a workspace over its seat limit, an unaccepted invite, a member whose role just changed. Every environment runs twice, with and without your change, so any difference is attributable to the change.
4. Read the failures. Results come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it.
5. Close the loop with your agent. The failing case goes back to your coding agent in an actionable form; it fixes, re-runs, and repeats until nothing fails.

The command is haystack verify. The site points to separate pages for haystack verify and its flags, "Working with runs" for reading results, and "Agents" for the MCP server and the agent loop.

Why the access bar is narrow

The site's own example explains the pitch: a permissions refactor touching two files reached 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. In that scenario a frontend stopped sending a member's role, the access API treated a missing role as editor, and anyone with a pending invite could edit or delete anything in the workspace. Nothing threw, every request returned 200, and the test suite passed. Catching that class of quiet failure requires real environments and real traffic, which is why Haystack is onboarding teams selectively rather than opening a free tier.

Practical next step

If you want access, start at https://haystack.sh/ and sign up for early access, or book a demo if you would rather see it working first. Come prepared with a specific change you would want verified — the product's value is easiest to judge against a real diff and real traffic, not a hypothetical.