audioarena.ai

To view a turn-by-turn transcript on AudioArena, you select three things in sequence: a benchmark, a model, and a specific run. The transcript then shows how the voice agent handled each turn of that conversation. This works for any of the six static multi-turn voice agent benchmarks on the site — Appointment, Assistant, Conversation, Event, Grocery, and Product — and any model or run available for them.

Steps

1. Select a benchmark. Choose one of the six AudioAgentBench benchmarks: Appointment, Assistant, Conversation, Event, Grocery, or Product. Each is a static multi-turn voice agent scenario, so pick the one closest to the interaction you want to inspect.
2. Select a model. Choose the voice model whose behavior you want to examine. Results are scored across five dimensions, so the model you pick determines which scores and transcripts you'll see.
3. Select a run. A model can have more than one run on a benchmark. Pick the specific run you want to inspect — the transcript belongs to that run, not to the model in general.
4. Read the transcript. The turn-by-turn transcript opens for that benchmark/model/run combination, letting you follow the conversation one turn at a time.

What you get

The transcript is the record of a single run, so two runs of the same model on the same benchmark can be compared by opening each separately.

Common snags

• No transcript appears: confirm all three selections are made. The transcript is tied to a specific run, so a benchmark and model alone aren't enough.
• Comparing models: keep the benchmark and, where possible, the run conditions consistent so differences reflect the model rather than the scenario.
• Understanding the scores: the five scoring dimensions and the benchmark setup are explained in AudioArena's methodology, which is worth reading before drawing conclusions from a single transcript.

AudioAgentBench is published by Arcada Labs, By The Intelligence Company, with the project available on GitHub.

AudioArena lets you compare voice AI models by running them against the same static multi-turn voice agent benchmarks and reading the results side by side. To get a useful comparison, match the benchmark to your real use case, then compare models across all five scoring dimensions rather than a single overall number, and finish by reading the turn-by-turn transcripts to see why one model scored higher.

Step 1: Pick the benchmark that matches your use case

AudioAgentBench includes six static multi-turn voice agent benchmarks. Each one simulates a different kind of conversation, so the right starting point depends on what you're actually building or buying for.

If your scenario doesn't map cleanly onto one benchmark, run more than one and treat the results as a range rather than a single verdict.

Step 2: Select a model and a run

On AudioArena you select a benchmark, a model, and a run to view the results. Because runs are recorded, the same model and benchmark combination can be inspected repeatedly — which is what makes side-by-side comparison possible: hold the benchmark constant, change the model, and compare.

Step 3: Compare across all five scoring dimensions

Each benchmark is scored across five dimensions. Comparing only the aggregate score hides the trade-offs that usually decide a real deployment — a model that sounds natural but drops task details will look fine on a summary score and fail in production.

A practical comparison method:

1. List the models you're considering as rows.
2. List the five scoring dimensions as columns.
3. Fill in each cell from the benchmark results.
4. Mark which dimensions are non-negotiable for your use case (for example, task completion over conversational polish).
5. Eliminate any model that fails a non-negotiable dimension, then rank the rest.

This keeps the decision tied to your requirements instead of to whichever model happens to top the overall ranking.

Step 4: Read the turn-by-turn transcripts

Scores tell you that a model struggled; transcripts tell you where. Turn-by-turn transcripts show the actual conversation, so you can spot patterns such as:

• Losing track of information the user gave earlier in the conversation
• Asking for details that were already provided
• Handling interruptions or topic changes poorly
• Completing the task but in an unnatural or overly verbose way

If two models score similarly, the transcripts are usually what breaks the tie — one may fail gracefully and recover, while the other derails the conversation.

Step 5: Turn the comparison into a decision

Use the results to answer the question AudioArena poses directly: which AI would you call for this task? A defensible decision looks like this:

• Benchmark chosen: the one closest to your real scenario
• Models compared: the shortlist you'd actually deploy
• Deciding dimension: the score that matters most for your use case
• Transcript evidence: a specific moment that shows the difference
• Condition: the choice holds only for scenarios like the benchmark you ran — a model that wins on Appointment isn't automatically the best choice for Grocery or Product

Common pitfalls

• Comparing across different benchmarks. Scores from different benchmarks aren't interchangeable; keep the benchmark fixed when comparing models.
• Trusting a single run. Look at more than one run before drawing conclusions about a model.
• Ignoring the methodology. AudioArena publishes a methodology document and links to its GitHub (Arcada Labs, by The Intelligence Company). Read it before treating scores as definitive, since it defines how the benchmarks and dimensions are scored.
• Over-reading static benchmarks. These are static multi-turn benchmarks, not live calls — they're a structured comparison tool, not a guarantee of real-world performance.

AudioArena benchmarks voice models through AudioAgentBench, a set of six static multi-turn voice agent scenarios scored across five dimensions. You can inspect results turn by turn, per model and per run, and the site publishes a dedicated methodology page documenting the approach. The key caveat: these are static, scripted conversations rather than live calls, so the scores reflect performance on fixed scenarios, not open-ended real-world phone behavior.

What AudioAgentBench measures

AudioAgentBench is built from six named benchmarks, each representing a distinct voice-agent task:

Each benchmark is a static multi-turn conversation. "Static" means the turns are pre-defined rather than generated live by a caller, so every model faces the same script. That makes runs comparable across models, but it also means the benchmark can't capture how a model handles unexpected caller behavior.

How scoring works

Every benchmark is scored across five dimensions. The site's own summary describes it as "scored across 5 dimensions," and points to a methodology page for the details of how those dimensions are defined and weighted. If you need the exact rubric — what each dimension captures and how a turn is judged — that methodology page is the authoritative source, since the benchmark description alone doesn't spell out the criteria.

How to read the results

Results are organized so you can compare models and inspect individual runs:

• Per model — see how a given voice model performs across the benchmarks.
• Per run — because a model can be run more than once, you can view results for a specific run rather than only an aggregate.
• Turn-by-turn transcripts — select a benchmark, a model, and a run to see the conversation turn by turn.

That last point matters for interpretation. Aggregate scores tell you which model scored higher; transcripts tell you why. A model can lose points on a single turn — a misheard request, a dropped constraint, an awkward recovery — and the transcript is where you see it.

What this methodology can and can't tell you

Good for: comparing models on identical, reproducible tasks; spotting turn-level failure patterns; checking consistency across multiple runs of the same model.

Not designed for: predicting live-call performance. Because the conversations are static, the benchmark doesn't test barge-in, silence handling, accent variation, or recovery from a caller who goes off-script. Treat the scores as a controlled comparison, not a simulation of production traffic.

Practical checklist for interpreting a score

1. Identify which benchmark the score comes from — a strong Appointment result says little about Grocery performance.
2. Check whether you're looking at a single run or an aggregate across runs; a single run can be an outlier.
3. Open the turn-by-turn transcript for any surprising result and find the turn where the score was lost.
4. Read the methodology page before treating the five dimensions as equally weighted or self-explanatory.
5. Compare models only within the same benchmark and the same run conditions.

AudioArena is published by The Intelligence Company, with the benchmark work attributed to Arcada Labs and a GitHub presence referenced on the page.

AudioAgentBench is the benchmark suite on AudioArena that evaluates voice agents through static multi-turn conversations. It contains six named benchmarks — Appointment, Assistant, Conversation, Event, Grocery, and Product — and each one is scored across five dimensions. You can select a benchmark, a model, and a run to read turn-by-turn transcripts, and a separate methodology page explains how the scoring works.

The six benchmarks

Each benchmark is a self-contained scenario type rather than a single prompt, and each is multi-turn, so a model is judged on how a conversation develops rather than on one isolated reply.

The names map to everyday voice-agent jobs: booking something, handling a general request, holding a conversation, organizing around an event, assembling a grocery order, or dealing with a product. Because all six are static multi-turn benchmarks, the same conversation is replayed to each model, which is what makes cross-model comparison meaningful.

The five scoring dimensions

Every benchmark is scored across five dimensions. The site states the count and points to a methodology page for the details of how scoring works; the specific dimension names and their definitions live on that methodology page rather than in the benchmark listing itself.

Practical consequence: when you compare two models, don't collapse the five dimensions into one number. A model can be strong on some dimensions and weak on others within the same benchmark, and the turn-by-turn transcripts are where you see why.

How to read a result

1. Pick a benchmark that matches the job you care about — for example, Grocery if you're evaluating order-taking.
2. Pick the models you want to compare.
3. Pick a run and open the turn-by-turn transcript.
4. Read the transcript alongside the dimension scores, so a low score is tied to a specific turn rather than left as an abstract rating.

Expected result: you get a per-benchmark, per-model view of a full conversation, not a single headline score.

Common sticking points

• Benchmark name ≠ difficulty level. Appointment and Grocery are different task types, not easier or harder versions of each other. Choose by task fit.
• Static multi-turn means fixed scripts. Results reflect performance on those scripted conversations; they don't automatically generalize to unscripted live traffic.
• Dimension definitions matter. Before drawing conclusions from a score, read the methodology page so you know what each dimension is actually measuring.
• Transcripts are the evidence. If two models score similarly, the turn-by-turn transcripts are what tell you whether the difference matters for your use case.

AudioAgentBench is published by Arcada Labs, with a GitHub presence and attribution to The Intelligence Company.

AudioArena is a platform for benchmarking and comparing state-of-the-art voice AI models through real conversations. Its core asset is AudioAgentBench, a set of six static multi-turn voice agent benchmarks — Appointment, Assistant, Conversation, Event, Grocery, and Product — scored across five dimensions. If you want to judge how a voice model actually behaves across a full conversation rather than on a single prompt, this is the kind of resource you'd use.

What AudioArena offers

• Conversation-based comparison. Rather than leaderboard scores alone, the site frames evaluation around "which AI would you call?" — i.e., how a voice agent performs in a realistic back-and-forth.
• AudioAgentBench. Six named benchmarks covering common voice-agent tasks:
  • Appointment
  • Assistant
  • Conversation
  • Event
  • Grocery
  • Product
• Multi-dimensional scoring. Each benchmark is scored across five dimensions (the site references a separate methodology page for details).
• Turn-by-turn transcripts. You can select a benchmark, a model, and a run to read the full transcript of the interaction.

How to use it

1. Pick one of the six benchmarks that matches the task you care about (e.g., Grocery if you're evaluating order-taking).
2. Select a model and a specific run.
3. Read the turn-by-turn transcript to see how the agent handled each exchange, and check the dimension scores for that run.

The practical value is in the transcripts: scores tell you that a model performed well or poorly, while the transcript shows where it broke down — a missed clarification, a wrong turn in a multi-step task, or a dropped context.

Who it's for

• Teams choosing a voice model for a specific agent use case (booking, support, ordering).
• Researchers or builders who want transcript-level evidence rather than aggregate rankings.
• Anyone comparing voice models on multi-turn behavior instead of single-turn responses.

Caveats and context

• The benchmarks are described as static, so they measure performance on fixed scenarios rather than live, open-ended interaction.
• AudioArena is associated with Arcada Labs / The Intelligence Company, and the site links to GitHub for further detail.
• The page evidence does not state pricing or login requirements, so check the site directly before assuming access terms.

For methodology specifics — how the five dimensions are defined and weighted — read the methodology page linked from the site.