Fake It Till You Benchmark It: Big Tech’s AI Catch-Up Act

Lede

Big Tech has discovered that when winning the AI race becomes difficult, winning the benchmark presentation is a perfectly respectable alternative.

Words used

• Frontier model: An AI model operating near the highest level of capability currently available.
• Benchmark: A defined test used to measure and compare AI performance.
• Open-weight model: An AI model whose trained weights are available for others to download and run, subject to its licence.

Hermit Off Script

Fake it until you make it, AI edition. Gemini 4 Argon, Meta Muse Wonder Full, or Wonderfool if the marketing department gets any more creative, and Grok 4.7, whichever bot happens to be wearing the crown this week. I look at these releases and wonder how we reached the point where some of the richest technology companies on Earth are suddenly desperate to prove that they can compete with companies that made them rethink their entire AI strategy. Google had Search, Meta had social media, and both had enormous audiences, mountains of money and years of collecting the rewards of having billions of people inside their platforms. They made billions, created billionaires and built businesses so powerful that competing with their existing products already seemed almost impossible. And yet the arrival of ChatGPT and Claude helped change what people expected from software. Suddenly, controlling the platform wasn’t enough. The platform needed a genuinely intelligent assistant too. What an inconvenience. I suppose users were extremely valuable when they were generating advertising revenue, but asking what those users actually wanted from technology was apparently a more complicated exercise. So now the giants are chasing the newcomers, particularly OpenAI and Anthropic, which have set much of the pace in advanced reasoning and agentic work. And what do you do when you cannot convincingly demonstrate that your model is better at everything? You show the benchmarks where it wins. More charts, more scores, more records, more declarations about frontier intelligence. I don’t doubt that these companies have brilliant engineers. They clearly do. Nor can I pretend that Gemini 4 Argon and Muse Wonder are merely mediocre models, because independent evaluations show genuine progress, including some leading results. That rather spoils the easy joke, but facts have a nasty habit of doing that. My problem is what happens between the laboratory and the user’s desk. A model can score wonderfully on a specialised test and still be awkward at real-world work, forget instructions, struggle with tools or fail to finish small tasks without supervision. And if cheaper open-weight alternatives already handle much of the ordinary work well enough, what exactly is the expensive subscription selling beyond convenience and a familiar corporate name? I suspect the temptation to optimise for favourable measurements is enormous when investors, journalists and users all want one simple winner. That doesn’t prove anyone deliberately manipulated the latest results, and I wouldn’t pretend otherwise. But I would like the industry to spend as much energy checking completed work as it spends announcing intelligence. Maybe the new competitors really will catch up across every important task, and perhaps eventually overtake OpenAI and Anthropic. Competition can produce excellent results. Or perhaps the race will become another expensive corporate parade where every company wins its chosen category while users are left checking the actual output. I suppose only time will tell whether the money buys better intelligence or merely better presentations. The users carried these platforms to the top. Now the platforms want a trophy for learning to carry a task to the finish.

What does not make sense

• Google and Meta spent years dominating their respective markets, with access to enormous audiences and revenues, yet newer AI companies forced a fresh competition over who could build the most capable assistants.
• The same companies that became extraordinarily wealthy through user attention now want people to accept AI integration as progress, even when its practical value and effects on user control remain open questions.
• A model can win one benchmark and trail on another. The press release usually develops a touching attachment to the first one.
• Google and Meta have produced genuinely strong new models. Calling their entire progress fake would be as lazy as believing every claim in their announcements.
• Anthropic and OpenAI have strong positions in advanced agent evaluations, but treating them as permanent winners ignores how quickly the rankings can change.
• A model performing well on legal, coding or reasoning tests is encouraging. It doesn’t guarantee that an ordinary user can give it an untidy, multi-step job and receive a correct finished result.
• A 2026 study examined 60 language-model benchmarks and found nearly half had become saturated. A test that can no longer distinguish strong models makes a rather questionable trophy. [PMLR]
• Better scores can increase subscriptions, investment interest and favourable coverage. None of those outcomes establishes how much time the customer saved after fixing the mistakes.

Sense check / The numbers

1. The Google money machine. Alphabet reported $119.8 billion in revenue for the second quarter of 2026, including $94.5 billion from Google Services. It also reported 950 million monthly active users of the Gemini app. Google didn’t arrive at the AI contest without money or distribution. [Alphabet, SEC]
2. The Meta audience. Meta reported $60.80 billion in second-quarter 2026 revenue and an average of 3.60 billion daily active people across its family of applications in June. Quarterly capital expenditure reached $31.08 billion. These figures demonstrate financial and distribution power, not automatically better AI. [Meta]
3. Google’s actual benchmark victory. On the Text Arena ranking dated 15 October 2026, Gemini 4 Argon High stood first among 416 listed models, with a preliminary score based on 4,904 votes. This is evidence of strong performance in that evaluation, not proof that it leads every type of AI work. [Arena]
4. The agent leaderboard tells a different story. On 19 October 2026, Agent Arena ranked Claude Opus 5.5 Max first and GPT-6 Astra Max second on its overall agent ranking, across 56 models and more than 2.66 million sessions. Rankings based on agent task outcomes need not match text-preference rankings. [Arena]
5. Meta’s progress is measurable. Artificial Analysis reported that Muse Wonder Full Max reached 56 points on its Intelligence Index, compared with 48 for Muse Spark 1.3 Max, and placed among the top three on six of ten evaluations. That is genuine measured improvement, although benchmark gains alone cannot establish reliability in every deployed setting. [Artificial Analysis]

The sketch

Scene 1: The benchmark coronation
A large executive silhouette stands on a podium beneath a giant leaderboard showing a gold trophy. A small user holds an unfinished document below.
Dialogue:
Executive: “We’re number one!”
User: “At what?”

Scene 2: The real-world examination
The same executive stands beside a medal-wearing AI robot facing an unfinished spreadsheet on a desk. The user points towards the screen.
Dialogue:
User: “Finish this, please.”
Robot: “Request timed out.”

Scene 3: The customer service miracle
The executive stands behind an enormous trophy while handing the user another subscription invoice. The robot remains beside the unfinished spreadsheet.
Dialogue:
Executive: “Try premium.”
User: “Try finishing.”

What to watch, not the show

• Advertising and platform distribution: Google and Meta can place their assistants inside services already used by enormous audiences. Being available everywhere can matter commercially even when another model performs better.
• Benchmark selection: Watch which evaluations appear in launch announcements, which are missing, and whether independent testers can reproduce the results.
• Actual task completion: Measure how frequently models finish multi-step work successfully, how often humans must intervene, and whether the final output survives proper checking.
• Money and infrastructure: Expensive training, computing capacity and distribution can favour established corporations, but investment alone doesn’t guarantee technical leadership.
• Open-weight competition: Cheaper, accessible models may challenge premium subscriptions for everyday tasks, particularly where businesses can run models independently.
• Subscriptions and lock-in: Watch whether companies deliver clear improvements or simply move desirable capabilities into increasingly expensive tiers.
• Privacy and user control: Integrating AI with search histories, messages, documents and social platforms raises serious questions about consent, data access and independent scrutiny.
• Frontier AI access: Watch who receives the strongest systems, who gets the cheaper alternatives and whether access rules widen the gap between large organisations and ordinary users.
• Long-term incentives: When beating a rival on a public test attracts attention faster than building a dependable product, the scoreboard risks becoming the product.

The Hermit take

A benchmark crown is worth keeping only when the model can survive an ordinary working day.
The strongest AI won’t need the prettiest scoreboard. Its finished work will do the talking.

Keep or toss

Keep / Toss

Keep the competition, independent benchmarks, real technical progress and increasingly capable affordable models.
Toss the selective scorecard worship, corporate self-congratulation and expectation that users should pay for unfinished promises.

Sources

• Alphabet second-quarter 2026 financial results: https://www.sec.gov/Archives/edgar/data/1652044/000165204426000066/googexhibit991q22026.htm
• Meta second-quarter 2026 financial results: https://investor.atmeta.com/investor-news/press-release-details/2026/Meta-Reports-Second-Quarter-2026-Results/
• Google Gemini 4 Argon official introduction: https://blog.google/innovation-and-ai/models-and-research/gemini-models/introducing-gemini-4-argon/
• Meta Muse Wonder and Muse Wonder Full announcement: https://about.fb.com/news/2026/10/introducing-muse-wonder-models/
• Grok 4.7 official announcement, 21 September 2026: https://x.ai/news/grok-4-7
• Arena Text leaderboard: https://arena.ai/leaderboard/chat/text?scoring=raw
• Arena Agent leaderboard: https://arena.ai/leaderboard/agent?stream=top
• Artificial Analysis independent Muse Wonder evaluation: https://artificialanalysis.ai/articles/muse-wonder/
• ICML 2026 research on benchmark saturation: https://proceedings.mlr.press/v306/akhtar26a.html

Disclaimer: This article contains satire, commentary and editorial opinion. Claims about model performance refer to published evaluations at the dates stated. The expression “fake it” criticises marketing incentives and does not allege proven fraud or deliberate benchmark manipulation by the named companies.