The AI Frontier Built Power Before It Built Any Defence

Lede

Frontier AI companies built machines they describe as strategically important, then appeared surprised when competitors copied their homework through the front door.

Words used

• Distillation: Training a smaller or less capable model using answers produced by a stronger model.
• Model weights: The internal numerical parameters holding much of a trained model’s learned capability.
• Open-weight model: A model whose weights can be downloaded, modified and privately operated.

Hermit Off Script

The weakness of frontier AI is becoming difficult to ignore. Companies tell us their models may alter economies, cyber security, science and national power. Then Anthropic announces that three Chinese AI laboratories allegedly generated more than 16 million exchanges through about 24,000 fraudulent accounts to extract Claude’s capabilities. Apparently, the most advanced intelligence industry in history discovered suspicious automated behaviour only after the automated behaviour had booked a stadium.
Anthropic accuses DeepSeek, Moonshot and MiniMax of using distillation against Claude. Anthropic also admits that distillation itself is a normal method used by frontier companies to make smaller and cheaper models. The objection is therefore not the technique. It is who used it, how they gained access and whether they broke the rules. Fair enough. Theft does not become acceptable because the burglar understands engineering.
But I cannot ignore the larger question. How did companies building systems they consider dangerous fail to prepare properly for the obvious possibility that someone would collect millions of outputs? Fraudulent accounts, proxy services, repeated structured prompts and industrial-scale traffic are not magic. This is not someone tunnelling beneath the headquarters with a teaspoon. They allegedly came through an online service operated by the company itself.
Anthropic says it is detecting and blocking these campaigns. Good. That is the minimum, not a victory parade. A frontier company cannot boast about being months ahead of everyone else while acting like model extraction arrived by surprise. Security should have grown with capability. Instead, capability sprinted ahead while security was still looking for its shoes.
And China is not the darkest version of this problem. Chinese laboratories are commercial and strategic competitors with infrastructure, staff and governments watching them. What happens when a terrorist group, criminal network or hostile proxy uses the same route to build a cheaper specialist model? It may not need the best general AI in the world. It may need only enough capability in cyber operations, chemical research, propaganda or automated targeting to become dangerous.
Maybe a small hostile group will never reproduce an entire frontier model. It may not need to. Distil one useful capability, remove the refusals, combine it with stolen tools and rent enough computing power. Suddenly the billion-pound security problem has a monthly subscription.
Then there is the question frontier companies dislike asking aloud: what happens when a Chinese model genuinely leads?
I have already asked what happens when the rich get frontier AI while everyone else gets safety talks. This is the other side of that locked door: restricted access does not stop capability spreading. It may simply decide who copies it first.
Not trails by several months. Not copies yesterday’s capability more cheaply. Leads. Will American companies demand open competition after years of export controls, regional restrictions and accusations? Or will “AI safety” suddenly become another name for “we were winning before”?
The frontier was sold as a technological summit. Yet the industry still markets every temporary lead like a coronation, as I argued when comparing AI miracle hype with stubbornly mortal tools. At present, the frontier sometimes looks more like a powerful engine left running outside the shop.

What does not make sense

• Anthropic describes Claude’s capabilities as important to national security, yet alleged extraction continued across about 24,000 fraudulent accounts.
• Frontier companies use distillation themselves but speak about it as sinister when a competitor allegedly uses their outputs without permission.
• Regional access restrictions were treated as protection even though proxy networks exist precisely to bypass regional restrictions.
• The industry measures model intelligence obsessively while publishing far less useful evidence about whether its access controls can withstand organised extraction.
• Export controls restrict physical chips, but an application programming interface can still leak valuable behaviour one answer at a time.
• Companies warn that distilled models may lose safeguards, although they supplied the answers from which those capabilities could allegedly be learned.
• The public is asked to trust voluntary corporate security from companies that benefit financially from making model access fast, cheap and widespread.
• China is presented as the threat, while criminal and terrorist access is treated as tomorrow’s policy seminar.
• Frontier companies advertise models as increasingly capable reasoners, while the industry still struggles to distinguish useful intelligence from clever machinery wearing a laboratory coat.

Sense check / The numbers

1. Anthropic said on 23 February 2026 that DeepSeek, Moonshot and MiniMax generated more than 16 million exchanges through approximately 24,000 fraudulent accounts. That works out at an average of more than 666 exchanges per account, although the distribution may have varied widely. [Anthropic]
2. RAND identified 38 meaningfully distinct attack routes against frontier model weights and proposed 5 security levels. Its report also said there was no small collection of measures capable of acting as a complete solution. [RAND]
3. In July 2026, the UK AI Security Institute found that leading open-weight models trailed closed frontier models by only 4 to 7 months in its cyber evaluations, compared with a gap of 6 to 10 months during much of 2025. [AISI]
4. On one simulated 100-million-token cyber run, AISI estimated a cost of about $85 for Claude Opus 4.5 or 4.6, $46 for GLM-5.2 and $1.19 for DeepSeek V4-Pro. Cheap access does not prove equal capability, but it changes who can afford repeated attempts. [AISI]
5. AISI reported that frontier models completed apprentice-level cyber tasks about 50 per cent of the time, compared with just over 10 per cent in early 2024. It also found vulnerabilities in every model it tested. [AISI]

The sketch

Scene 1: The frontier vault
A laboratory executive stands beside a huge steel vault. Its door is open and thousands of answer sheets travel out through a small customer-service window.
Dialogue:
Executive: “Our intelligence is priceless.”
Security guard: “The API is open.”
Printer: “Sixteen million copies.”

Scene 2: The regional fence
A tiny fence marked “Access restriction” stands across a road. A convoy of proxy servers drives around it through an enormous empty field.
Dialogue:
Executive: “We blocked the region.”
Proxy driver: “We changed the address.”
Fence: “I did my best.”

Scene 3: The budget threat
A small masked group feeds copied answer sheets into a cheap machine. Across the room, executives protect an expensive pile of computer chips.
Dialogue:
Executive: “Guard the chips.”
Machine: “Capability imported.”
Masked operator: “That was affordable.”

What to watch, not the show

• Whether frontier laboratories publish measurable extraction-resistance tests rather than victory statements after attacks.
• How quickly providers detect linked fraudulent accounts, proxy networks and repeated capability-targeting prompts.
• Whether governments set mandatory security standards before models cross dangerous capability thresholds.
• Whether application programming interface prices subsidise capability extraction by making millions of targeted requests economical.
• How open-weight cyber models perform as the current 4-to-7-month gap narrows.
• Whether safety restrictions survive distillation, fine-tuning and private deployment, especially as stronger cyber models move behind restricted-access programmes.
• Whether “national security” becomes a serious engineering duty or merely a phrase used against foreign competitors.
• What Western companies do if China produces the strongest model without relying on Western systems.

The Hermit take

Do not call yourself the frontier while your security arrives months behind your model.
Intelligence without defence is capability delivery with nicer branding.

Keep or toss

Keep / Toss.

Keep research access, competition and useful smaller models.
Toss the fantasy that terms of service, regional blocks and corporate promises amount to national security.

Sources

• Anthropic report on alleged distillation campaigns, 23 February 2026: https://www.anthropic.com/news/detecting-and-preventing-distillation-attacks
• UK AI Security Institute analysis of open-weight cyber capability, 17 July 2026: https://www.aisi.gov.uk/blog/how-far-behind-the-frontier-are-leading-open-weight-models-on-cyber
• UK AI Security Institute Frontier AI Trends Report: https://www.aisi.gov.uk/frontier-ai-trends-report
• RAND report on securing frontier AI model weights: https://www.rand.org/pubs/research_reports/RRA2849-1.html