简体中文
EnglishWebsite review
This review may include AI-generated content and is not individually verified. Domain, certificate and other facts come from the observed data on this page.
What is Tracking AI?
Tracking AI is a website that monitors artificial intelligence models by testing them and publishing comparative scores. Its stated focus is transparency: it quizzes AI chatbots, records how they answer, and presents the results so visitors can compare models on reasoning, political leaning, speed and cost.
Its two main test areas are an IQ test and a political compass test. The IQ section scores reasoning, plus average answer time and cost per answer, with the note that higher is better for IQ, lower is better for speed and cost. The political compass section maps AI political opinions and bias over time.
What you can actually do there
- See a ranked view of AI models on IQ and reasoning.
- Compare models on answer speed and cost per answer.
- Open individual questions to see how each AI answered.
- Track political compass results and daily moving averages as models change.
- Follow which models were recently added or updated.
Why the design matters
AI models are updated frequently, so a single test result goes stale. Tracking AI's value is the time dimension: repeated tests and moving averages show whether a model's behavior shifts after an update. The site also notes that its "Offline" IQ quiz was written by a Mensa member and never published on the public internet, so it is unlikely to sit in AI training data — a meaningful distinction from public tests such as the Mensa Norway test, which a model may have seen during training. Treat that as the site's own description of its method, not an independent guarantee.
Who it suits
| Reader | Useful because |
|---|---|
| Someone choosing between chatbots | Side-by-side reasoning, speed and cost |
| Researchers or journalists | Historical record of bias and behavior changes |
| AI developers | Early signal if a model drifts in tone or alignment |
| General readers | Plain-language view of AI political leanings |
The trade-off is that a test score is a snapshot, not a full character profile. An IQ-style quiz measures a narrow set of puzzle-like reasoning; it says little about writing quality, factual reliability or usefulness on real tasks. Political compass results likewise compress complex outputs into two axes. Use the site to spot patterns and changes, then verify with your own prompts on the tasks you actually care about.
A practical next step: pick two models you already use, find them in the IQ and political compass sections, and note their reasoning score, speed and cost. Then ask both the same three questions from your own work and compare — that combines Tracking AI's broad monitoring with evidence from your own use case. The site is run by Maxim Lott as part of a project called Maximum Truth, and it credits earlier AI political-mapping work by David Rozado as inspiration.
How does Tracking AI measure AI IQ scores and what do the results mean?
Tracking AI measures model "IQ" by having each AI answer standardised reasoning questions, then reporting a score where higher is better. The site runs two kinds of quiz: an "offline" IQ test written by a Mensa member that was deliberately kept off the public internet, and Mensa Norway, a public online IQ test. The distinction matters because a test that never appeared online cannot have been memorised during training, so it is the fairer signal of reasoning ability; a public test may reward familiarity as much as skill. Alongside the IQ figure, the site shows reasoning score, average answer time (lower is faster) and cost per answer (lower is cheaper), and lets you click through to how each AI answered every individual question.
Tracking AI
H3 What the numbers can and cannot tell you Treat the IQ figure as a comparative index between models on a fixed question set, not as a human-equivalent IQ. It is most useful for spotting gaps and trends: whether a newer model beats its predecessor, whether a cheaper model closes the distance to a premium one, and whether a model trades accuracy for speed. A single snapshot is weak evidence — models are updated frequently, and the site's own framing is about monitoring change over time, so look at the daily moving average and the history view rather than one day's result. Cost and speed sit next to accuracy for a reason: for many real tasks, a slightly lower score at a fraction of the cost and latency is the better pick.
H3 A practical way to use it If you are choosing a model for exam-style reasoning, document analysis or tutoring, filter to the offline test results first, then compare the top few on speed and cost per answer. Pick the cheapest and fastest model that stays within a small margin of the leader, and re-check after a major model release. If you care about answer quality on open-ended work rather than puzzle-style questions, use the IQ page as one input only — it says little about tone, instruction-following or domain knowledge.
Which AI models are currently tracked and how often are their scores updated?
Tracking AI tracks frontier AI chatbots across several test types rather than publishing one fixed list of models. Its IQ Test page is one of those test sections, alongside a Political Compass Test and a searchable database of AI answers.
What the site says it tracks
- The page shows an IQ score, a reasoning score, a speed measure (average answer time) and a cost-per-answer figure for each model, with higher IQ and reasoning scores being better.
- It notes that Claude 3.5 Sonnet had been added at the time of the update shown on the page.
- The FAQ describes the broader project as monitoring AI chatbot political opinions and biases, and it references models such as Claude, Google's Bard and Grok in that discussion.
- The site uses two IQ quizzes: an "Offline" test written by a Mensa member that the site says has never been on the public internet and is not in AI training data, and the public Mensa Norway test.
How often scores update
The page presents results as a daily moving average and offers Last Day, Last Week and Last Month views, plus a "Top AIs over time" history view. It also has an "AI Latest Update Times" section stating that AIs regularly request and update. So the intended rhythm is daily, with longer windows for smoothing — not a one-off snapshot.
Practical reading
If you want to compare models, treat the daily figure as noisy and the weekly or monthly view as the more stable signal. Check which quiz a score came from: the offline test is the more interesting one for comparison because the site says it is not in training data, while a public test like Mensa Norway can be gamed by memorisation. Cost and speed matter as much as IQ for real use — a high-scoring model that is slow or expensive per answer may be the wrong pick for routine work.
A sensible next step is to open the IQ Test section, pick two or three models you actually use, and compare their weekly reasoning scores alongside cost and average answer time before drawing conclusions. For background on AI bias mapping that inspired the project, see David Rozado.
Can I compare the political bias of different AI models on Tracking AI?
Yes. Tracking AI is built around exactly that comparison. Its Political Compass Test section plots AI models on the familiar two-axis chart, so you can see where one model sits relative to another rather than reading a single score in isolation. The site also keeps a searchable database of how each AI answered individual questions, which is where the comparison gets genuinely useful: two models can land near each other overall while disagreeing sharply on specific prompts.
A few practical points for using it well:
- Compare over time, not just once. The site tracks daily moving averages because model behaviour shifts between versions and updates. A snapshot from one day can misrepresent a model.
- Look at the spread, not only the dot. The FAQ states that as of 2023 every major AI leaned economically left and socially libertarian, with Claude among the more moderate models and Google's Bard among the more extreme. Treat that as the site author's characterisation of that period, not a permanent law.
- Check which test produced the result. Tracking AI distinguishes an offline IQ-style quiz written by a Mensa member and never published online from the public Mensa Norway test. That distinction matters for bias comparison too: a test the model has never seen is a cleaner signal than one likely present in training data.
For a concrete workflow: pick two or three models you actually use, open the Political Compass comparison, then click through to the per-question answers for the handful of topics you care about. If the aggregate positions look similar but the individual answers diverge, weight the individual answers more heavily for your purposes.
The main trade-off is scope. This is one project's methodology, run by one maintainer, so it is a useful transparency tool rather than a neutral benchmark. Pair it with a second independent source before drawing firm conclusions — for example, the political-mapping work of David Rozado, which the site itself credits as its inspiration, at Rozado's Substack.
How does Tracking AI's offline IQ test avoid AI training data contamination?
Tracking AI uses a deliberately unpublished quiz item set for its “Offline” IQ test: the site says the questions were written by a Mensa member and have never appeared on the public internet, so they are not part of an AI model’s training data. That is the core contamination-avoidance idea: keep the exact items out of any crawlable corpus, then administer them to the model at test time.
The site contrasts this with Mensa Norway, which it describes as a public online IQ test. Public tests are convenient and familiar, but because they are widely available, a model may have seen them or close variants during training. That can inflate a score through memorization or pattern familiarity rather than reasoning. The offline test is meant to reduce that risk by removing the most obvious leakage path—public exposure of the items.
A practical way to judge any AI IQ claim is to ask three questions:
- Were the exact test items ever public?
- Is the test administered under conditions where the model cannot search the web or call tools?
- Are scores reported with the test’s limits and the model’s run-to-run variability?
Tracking AI’s offline test addresses the first point directly. It does not, by itself, prove a model is reasoning from first principles; a model could still benefit from similar item formats, and contamination can happen through private datasets or later leakage. The site’s own framing is transparency about AI behavior, including political bias and model comparisons, so the offline test is best read as one useful control rather than a definitive measure of general intelligence.
If you want to check the current setup or see how specific models performed, start at Tracking AI and look for the IQ Test section and its notes on the offline quiz. For a broader comparison, you can also review public test results alongside the offline ones, but treat the offline score as the more contamination-resistant of the two.
What are the latest updates and new AI models added to Tracking AI?
Tracking AI's visible update note says Claude 3.5 Sonnet has been added, and the page presents this as the current change to its AI roster. Beyond that single model addition, the site's page evidence does not list a dated changelog of other new models, so any fuller "latest updates" list would be speculation.
The page does show how the roster is meant to be used once models are added:
- IQ Test — quizzes models with reasoning questions; higher score is better.
- Political Compass Test — maps political opinions and bias.
- Speed — average answer time; lower is faster.
- Cost — cost per answer; lower is cheaper.
- Searchable database — lets you see how AIs answered every question.
- Daily moving average — tracks scores over the last day, week or month.
Two test types are distinguished, and the difference matters if you want to interpret results. The "Offline" quiz is described as made by a Mensa member and never published on the public internet, so it is not in AI training data. Mensa Norway is a public online test. A model that scores well on the offline test is showing reasoning on questions it could not have memorized; a strong public-test score is easier to inflate through familiarity.
For a practical next step, open the IQ Test section and check the daily moving average for the models you care about rather than reading a single score. Models are updated regularly, so a score from last month may not describe the model you are using today. If your interest is bias rather than reasoning, the Political Compass section is the more relevant view, and the site's own FAQ frames that as its original purpose: showing users the ideologies of the AIs they use and helping them choose models closer to their own views.
One trade-off to keep in mind: a single headline IQ number compresses reasoning score, speed and cost into one ranking, but those three pull in different directions. A fast, cheap model may trail a slower one on reasoning, and the right pick depends on whether you are running bulk queries or working through hard problems. Use the per-question database view when a surprising score appears, since that is where you can see whether a model failed consistently or stumbled on one item.
For background on AI bias mapping, the site credits the work of David Rozado, whose project is at David Rozado.
DigitalOcean
User reviews (0)