What is AudioArena?

Updated

AudioArena is a platform for benchmarking and comparing state-of-the-art voice AI models through real conversations. Its core asset is AudioAgentBench, a set of six static multi-turn voice agent benchmarks — Appointment, Assistant, Conversation, Event, Grocery, and Product — scored across five dimensions. If you want to judge how a voice model actually behaves across a full conversation rather than on a single prompt, this is the kind of resource you'd use.

What AudioArena offers

  • Conversation-based comparison. Rather than leaderboard scores alone, the site frames evaluation around "which AI would you call?" — i.e., how a voice agent performs in a realistic back-and-forth.
  • AudioAgentBench. Six named benchmarks covering common voice-agent tasks:
    • Appointment
    • Assistant
    • Conversation
    • Event
    • Grocery
    • Product
  • Multi-dimensional scoring. Each benchmark is scored across five dimensions (the site references a separate methodology page for details).
  • Turn-by-turn transcripts. You can select a benchmark, a model, and a run to read the full transcript of the interaction.

How to use it

  1. Pick one of the six benchmarks that matches the task you care about (e.g., Grocery if you're evaluating order-taking).
  2. Select a model and a specific run.
  3. Read the turn-by-turn transcript to see how the agent handled each exchange, and check the dimension scores for that run.

The practical value is in the transcripts: scores tell you that a model performed well or poorly, while the transcript shows where it broke down — a missed clarification, a wrong turn in a multi-step task, or a dropped context.

Who it's for

  • Teams choosing a voice model for a specific agent use case (booking, support, ordering).
  • Researchers or builders who want transcript-level evidence rather than aggregate rankings.
  • Anyone comparing voice models on multi-turn behavior instead of single-turn responses.

Caveats and context

  • The benchmarks are described as static, so they measure performance on fixed scenarios rather than live, open-ended interaction.
  • AudioArena is associated with Arcada Labs / The Intelligence Company, and the site links to GitHub for further detail.
  • The page evidence does not state pricing or login requirements, so check the site directly before assuming access terms.

For methodology specifics — how the five dimensions are defined and weighted — read the methodology page linked from the site.