How AudioArena's Benchmarking Methodology Works

AudioArena benchmarks voice models through AudioAgentBench, a set of six static multi-turn voice agent scenarios scored across five dimensions. You can inspect results turn by turn, per model and per run, and the site publishes a dedicated methodology page documenting the approach. The key caveat: these are static, scripted conversations rather than live calls, so the scores reflect performance on fixed scenarios, not open-ended real-world phone behavior.

What AudioAgentBench measures

AudioAgentBench is built from six named benchmarks, each representing a distinct voice-agent task:

Each benchmark is a static multi-turn conversation. "Static" means the turns are pre-defined rather than generated live by a caller, so every model faces the same script. That makes runs comparable across models, but it also means the benchmark can't capture how a model handles unexpected caller behavior.

How scoring works

Every benchmark is scored across five dimensions. The site's own summary describes it as "scored across 5 dimensions," and points to a methodology page for the details of how those dimensions are defined and weighted. If you need the exact rubric — what each dimension captures and how a turn is judged — that methodology page is the authoritative source, since the benchmark description alone doesn't spell out the criteria.

How to read the results

Results are organized so you can compare models and inspect individual runs:

• Per model — see how a given voice model performs across the benchmarks.
• Per run — because a model can be run more than once, you can view results for a specific run rather than only an aggregate.
• Turn-by-turn transcripts — select a benchmark, a model, and a run to see the conversation turn by turn.

That last point matters for interpretation. Aggregate scores tell you which model scored higher; transcripts tell you why. A model can lose points on a single turn — a misheard request, a dropped constraint, an awkward recovery — and the transcript is where you see it.

What this methodology can and can't tell you

Good for: comparing models on identical, reproducible tasks; spotting turn-level failure patterns; checking consistency across multiple runs of the same model.

Not designed for: predicting live-call performance. Because the conversations are static, the benchmark doesn't test barge-in, silence handling, accent variation, or recovery from a caller who goes off-script. Treat the scores as a controlled comparison, not a simulation of production traffic.

Practical checklist for interpreting a score

1. Identify which benchmark the score comes from — a strong Appointment result says little about Grocery performance.
2. Check whether you're looking at a single run or an aggregate across runs; a single run can be an outlier.
3. Open the turn-by-turn transcript for any surprising result and find the turn where the score was lost.
4. Read the methodology page before treating the five dimensions as equally weighted or self-explanatory.
5. Compare models only within the same benchmark and the same run conditions.

AudioArena is published by The Intelligence Company, with the benchmark work attributed to Arcada Labs and a GitHub presence referenced on the page.