How do you use Haystack to verify a code change?

Haystack verifies a code change by reading your diff, forking a copy of production for every customer journey that change can reach, and running each case with and without the change so any difference is attributable to it. The main entry point is the CLI command haystack verify. It fits teams whose changes touch shared services, permissions, or data flows where a passing test suite has not been enough. Haystack is early and onboarding a small number of teams, so access starts with early access signup or a demo booking rather than open self-serve use.

What Haystack does with your change

The workflow has four moving parts, and you only supply the change itself.

• Blast radius. Haystack reads the diff and works out every case the change can affect. The cases are real customer journeys learned from production traffic, narrowed to the ones the change can reach — not scenarios someone invented for a test. In one worked example, a permissions refactor touching two files reached 3 services and 27 code paths, producing 14 cases worth running, mapped in about five seconds. You do not write the cases.
• Fork your infrastructure. Haystack snapshots your services and the data behind them — Postgres, Redis, queues, object storage — then forks that snapshot once per case. In the same example, 14 copies of prod were ready in 26 seconds. These are real environments, not mocks, and nothing inside one reaches prod.
• Run every case at once. Each environment is seeded into a different state: a workspace over its seat limit, an invite nobody accepted, a member whose role just changed. Every environment runs twice, with and without your change, so any difference is caused by the change.
• See what actually broke. Failures come with the screen before and after, the network calls that changed, and the events that stopped firing. The environment stays live, so you can open it.

Running the verification

The command to start with is:

haystack verify

The site points to haystack verify for the command and its flags, so check that reference for the exact options your version accepts rather than assuming flag names. The expected result is a set of runs, one per affected case, each executed in its own forked environment with and without your change.

To read what came back, the site directs you to Working with runs. That is where you interpret results: which cases failed, and the evidence attached to each failure — the before/after screen, the changed network calls, and the events that stopped firing.

Closing the loop with your coding agent

The failing case goes back to your coding agent in a form it can act on. The agent fixes, re-runs, and keeps going until nothing fails. The site covers this under Agents, which documents the MCP server and the agent loop.

So the full cycle is: read the diff and compute blast radius → fork production snapshots → run all cases in parallel → inspect failure evidence → hand the failing case to your agent → fix and re-run until clean.

Why this catches what tests miss

The failures that reach production are usually quiet. In the worked example, a frontend stopped sending a member's role and the access API treated a missing role as editor, so anyone with a pending invite could edit and delete anything in the workspace. Nothing threw, every request returned 200, and the test suite passed.

That is the gap Haystack targets: a change that is technically valid, passes assertions, and still breaks a real journey. Because each case runs against a real forked environment seeded into a specific state, and runs twice with and without the change, the difference shows up even when no error is raised.

Getting started

Haystack is early and taking on a small number of teams. Sign up for early access at https://haystack.sh/ or book a demo. Until you are onboarded, the CLI, runs, and agent loop described above are the pieces to plan around — the site's own references for haystack verify, Working with runs, and Agents are the places to confirm current command details and flags.