agentic-evidence.fly.dev

HotTea's founding-team access is listed at $199/month, including 25 candidate sessions and an estimated 2M token cap per month. The plan is built around hard cost caps rather than usage-based overage: prompt length, message count, candidate session count, and monthly token ceilings are described as hard stops, so requests stop before you go over. Access activates the moment payment clears through Stripe Checkout. This fits assessment teams that already run practical coding tasks and want a reviewer packet for agentic engineering, not teams looking for a free trial or a proctored exam environment.

What the $199/month covers

The pricing page lists a single founding-access tier. There is no published free tier or per-seat breakdown in the available material, so treat the $199 figure as the entry cost for a paid pilot rather than a starting point for negotiation.

What lands in the reviewer packet

Each candidate gets an isolated developer VM with Claude, Codex, tests, git, and normal shell tooling — no locked browser or mock IDE. The packet combines:

• Transcript of the candidate's AI turns
• Terminal output and git snapshots
• Final diff and test results, including hidden checks
• Reviewer note and a structured fluency analysis with quoted reasoning moments, probe responses, and follow-up questions

The sample packet shows this in practice: a senior backend candidate spent 29 minutes and 3 AI turns on an invite-acceptance regression, scored 9/10 on AI fluency, passed 4/4 hidden checks, and the reviewer note captured where the candidate rejected Claude's first suggestion and moved expiry validation ahead of the retryable membership branch.

Why the cost structure matters for a pilot

The guardrails are the selling point as much as the price. Because prompt length, message count, session count, and monthly tokens are hard stops, a pilot with a fixed candidate cohort has a predictable ceiling — you won't discover an overage bill after a busy month. The trade-off is that if you exhaust 25 sessions or the token cap, requests stop rather than scaling automatically, so plan your cohort size against the monthly allowance.

Who this is for

HotTea is aimed at teams that already use GitHub-based assessments and want an audit trail of how a candidate directs an agent — what they probe, push back on, and verify. It is not a proctoring tool; the candidate works the way they normally work, and you read the receipts.

If your hiring process is still manual review of diffs and tests, the packet adds a layer you may not need yet. If reviewers are already disagreeing about AI-assisted submissions and you want that disagreement documented with evidence, the $199/month pilot is the low-commitment way to test it on one cohort.

HotTea is a paid-pilot assessment tool that replaces proctoring with an evidence trail. Instead of locking the browser and blocking AI, it gives each candidate a real developer VM with Claude, Codex, git, tests, and a normal shell, then records how they direct and verify the agent. The output is a reviewer packet — transcript, terminal output, git snapshots, final diff, tests, hidden checks, and a reviewer note — rather than a pass/fail proctoring verdict. It fits teams that already trust practical coding tasks but want a defensible record of AI-era engineering behavior.

The core difference: prevention vs. observation

Traditional proctored assessments assume AI use is cheating and try to prevent it. HotTea assumes candidates will use AI agents in the job, so it measures how well they do it.

What the reviewer packet actually contains

The sample packet on the site (a Hatchways-style invite regression task) shows the structure. A candidate named Alex Chen, tagged Senior Backend Developer, worked for 29 minutes across 3 AI turns and scored 9/10 on AI fluency with 4/4 hidden checks passed.

The timeline entries are the point:

• 18:02 — Workspace VM provisioned with Claude Code and recorder.
• 18:09 — Candidate asked Claude to map the invite flow, then rejected the first suggestion, citing wrong branch order.
• 18:18 — Moved expiry validation ahead of the retryable membership branch — identified as the root cause.
• 18:31 — Git snapshot, tests, and reviewer note attached to packet.

That is the AI-fluency evidence: how the candidate directs the agent, what they probe, what they push back on, and what they verify. A proctoring tool would have logged none of it — or flagged the AI use as a violation.

Why this matters for the hiring decision

GitHub-based assessments already show what a candidate can build. The stated gap HotTea targets is the AI-era audit trail: the record of agentic engineering behavior that a final repo alone doesn't reveal. Two candidates can submit identical diffs while one drove the agent deliberately and verified the result, and the other accepted the first suggestion without checking. The packet distinguishes them.

The design choice follows from that: no proctoring, no locked browsers, no fake editors. The candidate works the way they really work, and the reviewer reads the receipts.

Cost and guardrails

Founding-team access is listed at $199/mo, including 25 candidate sessions per month and an estimated 2M monthly token cap. The site describes hard caps on prompt length, message count, candidate session count, and monthly tokens — requests stop before overage rather than billing through. Access activates when payment clears via Stripe Checkout.

When HotTea is the better fit

Choose it if your team:

• Already runs practical coding tasks and trusts them as signal
• Wants evidence of how candidates work with AI agents, not proof they avoided AI
• Needs a defensible record for hiring decisions — quoted reasoning moments, probe responses, and follow-up questions
• Is willing to run a paid pilot and review packets manually

Stick with proctoring if your constraint is a compliance or exam-integrity requirement that mandates monitored, AI-free conditions. HotTea is not built for that; it is built for teams hiring agentic engineers who will use these tools on the job.

HotTea collects evidence from a real agentic coding session — not a proctored exam — and bundles it into a reviewer packet. Candidates work in an isolated developer VM with Claude, Codex, tests, git, and normal shell tooling, and the platform captures the transcript, terminal output, git snapshots, final diff, tests, hidden checks, and a reviewer note. A structured fluency analysis then quotes specific reasoning moments and adds probe responses and follow-up questions for the next interview.

The workspace the evidence comes from

Each applicant gets an isolated developer VM provisioned with Claude Code, Codex, tests, git, and a normal shell. There is no mock IDE and no locked browser, so the candidate works the way they actually work. The sample packet shows this in practice: a workspace VM is provisioned with Claude Code and a recorder, and the session runs for a bounded window (the sample shows 29 minutes and 3 AI turns).

Because the environment is a real repo rather than a simulated editor, the artifacts it produces are the same ones an engineer would leave behind on the job — which is what makes them usable as hiring evidence.

What lands in the reviewer packet

One packet per candidate, containing:

• Transcript — the candidate's conversation with the agent, including what they asked for and how they responded to what came back.
• Terminal output — the commands run and their results.
• Git snapshots — intermediate states, not just the final answer.
• Final diff — the actual change the candidate landed.
• Tests — the test runs and their outcomes.
• Hidden checks — checks the candidate did not see, used to verify the work independently.
• Reviewer note — a human-written note attached to the packet.

The sample packet illustrates the shape of this: a Hatchways-style invite regression task, a candidate listed as a senior backend developer, 9/10 AI fluency, 4/4 hidden checks passed, and 3 agentic turns observed. Timestamped entries show the workspace being provisioned, the candidate asking Claude to map the invite flow and rejecting the first suggestion because of wrong branch order, then moving expiry validation ahead of the retryable membership branch to fix the root cause. A git snapshot, tests, and reviewer note are attached at the end.

How AI fluency is actually measured

The fluency analysis is not a single score pulled from nowhere. It is built from observable behaviors during the session:

• How the candidate directs the agent — the framing and specificity of their prompts.
• What they probe — which parts of the agent's output they interrogate rather than accept.
• What they push back on — moments where they reject or correct a suggestion, as in the sample where the candidate rejected the first suggestion citing wrong branch order.
• What they verify — how they confirm the change actually works, via tests and hidden checks.

The structured analysis quotes these reasoning moments directly, so a reviewer can read the candidate's own words rather than a summary. It also records probe responses and generates follow-up questions — what the candidate proved, what they did not prove, where reviewers disagreed, and what to ask next.

What the packet is designed to support

The stated purpose is a defensible technical hiring decision. GitHub-based assessments already show what a candidate can build; HotTea adds the AI-era audit trail for agentic engineering on top of that. The reviewer packet is meant to answer the questions a practical coding task alone cannot: how the candidate worked with an agent, where they exercised judgment, and what remains unverified.

Cost guardrails on the evidence collection

Sessions run under hard caps rather than soft warnings: per-session, per-message, and monthly token ceilings, plus limits on prompt length and candidate session count. Requests stop before overage. The founding access tier is listed at $199/mo, including 25 candidate sessions per month and an estimated 2M monthly token cap. Access activates when payment clears through Stripe Checkout, and a sample packet is available to open before committing.

HotTea is a technical hiring assessment platform that turns completed coding assessments into Hiring Signal Reports. It is built for teams that already run practical coding tasks and want a defensible record of how a candidate worked with AI agents — not a proctoring tool. If your assessment already shows what a candidate can build, HotTea adds the missing piece: an audit trail of AI fluency.

The problem it addresses

GitHub-based assessments show what a candidate can build. What they don't capture is how the candidate directs an AI agent, what they probe, what they push back on, and what they verify. HotTea's stated position is that practical coding tasks still work — the gap is the AI-era audit trail.

What lands in the reviewer packet

Each candidate gets one packet containing:

• Transcript — the conversation with the AI agent, including quoted reasoning moments
• Terminal output — what actually ran
• Git snapshots and final diff — the state of the work
• Tests and hidden checks — including pass/fail results
• Reviewer note — a structured fluency analysis

The report is organized around four questions: what the candidate proved, what they did not prove, where reviewers disagreed, and what to ask next.

How the assessment environment works

Candidates work in an isolated developer VM with Claude, Codex, tests, git, and normal shell tooling — described as a real VM, not a proctor. There are no locked browsers and no mock IDE. The candidate works the way they really work, and the reviewer reads the receipts.

A sample packet in the page evidence shows this in practice: a candidate asked Claude to map an invite flow, rejected the first suggestion citing wrong branch order, then moved expiry validation ahead of a retryable membership branch to fix the root cause. The packet recorded 9/10 AI fluency, 4/4 hidden checks passed, and 3 agentic turns observed.

Cost guardrails

The platform enforces hard stops rather than soft warnings. Prompt length, message count, candidate session count, and monthly token caps are ceilings — requests stop before overage. The page describes these as hard stops, not suggestions.

Pricing

Access is purchased through Stripe Checkout and activates the moment payment clears. The page does not state whether a free tier or trial exists beyond the paid pilot entry point.

Who it is for

Assessment teams and hiring managers who run practical coding tasks and need the hiring decision to be defensible. It complements existing GitHub-based assessments rather than replacing them — the value is the reviewer packet for agentic engineering, not the coding task itself.