What Is Vals AI? Independent Benchmarks for Real-World AI Tasks

Vals AI is an independent evaluation platform that benchmarks leading AI models on economically valuable, real-world tasks rather than synthetic puzzles. It runs its own evaluations and builds many of its benchmarks in-house, covering domains such as finance, software, cybersecurity, healthcare, and frontier risks like recursive self-improvement and mental health. If you need a third-party read on how models perform on practical work—and you care about methodology independence—it's worth consulting. If you need vendor-specific pricing, deployment guidance, or guarantees about any single model's fitness for your workflow, it is not that kind of resource.

What Vals AI actually measures

The platform's stated focus is testing AI on tasks with economic value. Its benchmark suite spans:

• Finance — task-based evaluation of financial work
• Software — engineering and coding tasks
• Cybersecurity — including frontier-risk evaluation
• Healthcare — domain-specific task performance
• Frontier risk — recursive self-improvement and mental health

Two claims matter for how much weight to give the results: Vals AI runs all of its own evaluations, and it creates many of its benchmarks in-house. That distinguishes it from aggregators that republish vendor-reported numbers.

How results are presented

Each model entry is scored across four dimensions:

Presenting cost, latency, and token usage alongside accuracy is the practically useful part. A model that tops the scoreboard but costs several times more per task, or responds too slowly for interactive use, may not be the right pick—and the leaderboard lets you see that trade-off rather than a single accuracy number.

The leaderboard, as published

The most recent snapshot on the site (dated Oct 02, 2026) shows the best model from each lab:

Two things to note before you act on this. First, the spread is wide—from 68.90% down to 8.86%—so the benchmark is discriminating between models rather than clustering them. Second, a single aggregate score hides per-domain differences; a model ranked mid-table overall may still lead in the specific domain you care about. Check the full results rather than the summary ranking.

Reports and evaluations

Beyond the leaderboard, Vals AI publishes benchmark releases and model evaluations. Recent examples from the site include:

• Claude Opus 5.5 agent task (Oct 04, 2026) — agents were asked to find a room-temperature magnet with zero net magnetism whose electrons are still sorted by spin (a Luttinger-compensated magnet). The agents proposed YBaMnFeO5, which looked ideal on paper but their follow-up calculations suggested it may be hard to make; they then identified KV[Cr(CN)6], a magnet first made in 1999 whose sample stayed magnetic up to 376 K, as fitting the description in their calculations. The inputs, raw outputs, and analysis code behind the numbers are in a ledger linked from the post.
• Vals Web Search Index (Oct 02, 2026) — an evaluation of search for real work. The stated motivation: web search hasn't been measured well, and providers have built their own benchmarks to work around data contamination, unrealistic tasks, and answers leaking onto the web. When a benchmark measures the wrong thing, customers can't tell which search tool is best for their work.
• "Has the Bitter Lesson Come for AI Detectors?" (Sep 30, 2026)
• Gemini 4 Argon (Sep 30, 2026) and Claude Sonnet 5.5 (Sep 28, 2026) — evaluated across the benchmark suite.

The agent-magnet report is a useful example of the platform's approach: it publishes the inputs, raw outputs, and analysis code, so you can inspect the reasoning rather than trust a headline number.

When to use Vals AI—and when not to

Use it when you want to:

• Compare models on task performance rather than marketing claims
• Weigh accuracy against cost, latency, and token usage in one view
• See whether a model's strength holds in a specific domain (finance, software, cybersecurity, healthcare)
• Check a third-party evaluation before committing to a model for production work

Look elsewhere when you need:

• Pricing or plan details for the platform itself—the site's pricing signals aren't published, so don't assume free or paid access
• Implementation guidance, API documentation, or integration steps for any model
• A verdict on whether a model meets your compliance, security, or latency requirements specifically

How to read the results responsibly

Benchmarks are a starting point, not a decision. Three practical cautions:

1. Check the domain, not just the rank. The overall score aggregates across finance, software, cybersecurity, healthcare, and frontier risk. Your use case lives in one of those, so find the relevant breakdown.
2. Treat cost and latency as first-class. A 1.9-point accuracy gap between the top two models may be irrelevant if one is materially cheaper or faster for your workload.
3. Prefer evaluations with published artifacts. The reports that link their inputs, raw outputs, and analysis code are the ones you can actually audit. Where that transparency is missing, weight the number accordingly.

Vals AI's value is that it produces independent, in-house evaluations with visible methodology and multi-dimensional scoring. That makes it a reasonable input to a model-selection decision—provided you read past the headline ranking and into the domain-level detail.