What the Humanity's Sixth Sense Benchmark Measures and What It Found

Humanity's Sixth Sense (HSS) is a benchmark that tests whether AI models can reason about what a visual scene implies rather than what it explicitly shows. In the results reported in Import AI 476, humans scored a pass@1 of 93.1 overall while the best AI system, GPT 6 Astra, reached 53.6 — a gap of roughly 40 points. The benchmark was built and released by AI startups Elorian and Scale AI.

What the benchmark measures

HSS targets "intuitive" visual reasoning: given an image or short video, can a model infer facts the scene suggests but never states? The benchmark's own framing is that it "measures how a model interprets and reasons through information a visual scene implies but doesn't explicitly show."

Two example task types from the newsletter:

• A picture of a bookcase, with the question of whether it can fit more books.
• A photo of horses pulling a plough, with the question of whether the chains bear the plough's drag weight.

Dataset composition

The four question categories

1. Temporal & causal dynamics — "What happened before the scene, and what happens next."
2. Physical & spatial logic — "What fits, what's in reach, and what someone else can see from where they are standing."
3. Social understanding — "Intent, attention, belonging."
4. Abstract & contextual inference — "The patterns and rules a scene implies."

Results: humans vs. the best AI system

The human baseline is described as "not perfect, but very high." The breakdown by category:

Chinese open-weight models performed notably worse on this benchmark: GLM-5.3-flash scored 26.8 and Kimi K3 scored 25.5.

How the AI systems failed

The benchmark authors report two dominant error patterns:

• Missing a decisive detail — 21% of failures. The model overlooks the one element in the scene that determines the answer.
• Misidentifying an object, person, or role — 20% of failures.

Together these account for about 41% of failures, meaning most AI errors are not exotic reasoning breakdowns but basic perception-and-attention misses on details that carry the answer.

Why the result matters

Import AI frames HSS as a test of whether "world models" are emergent in current systems. The benchmark can be read two ways:

• As a deficiency report: frontier models still fail at inference tasks humans handle almost effortlessly, especially where a small visual detail decides the answer.
• As a capability surprise: many people assumed true visual reasoning would require purpose-built, spatially aware systems. The fact that general-purpose frontier LLMs reach the mid-50s on tasks of this kind is itself informative about how much implicit scene understanding emerges from scale.

The practical takeaway for anyone evaluating models on visual reasoning: aggregate scores hide category-level differences, and the human–AI gap is largest on time-based inference (97 vs. 58.2) and smallest on video (91.9 vs. 51.1) — though even the smallest gap is about 40 points. If your task depends on inferring what a scene implies rather than describing what it contains, expect current systems to miss the decisive detail roughly one time in five.