Create

Sign in to ReadmeX

or

How we handle your data: Privacy Policy

ReadmeX

Your AI reading companion

What would you like to catch up on?

Pick a briefing, or ask your own question.

Your community and people briefings stay personal to you.

Story

Open-weight models near closed models in cyber tests

AI summary

Vals tested AI models on multi-step network intrusion tasks in two simulated scenarios involving mock corporate systems. OpenAI's trusted-access Daybreak completed an average of 81% of assigned tasks, while the open-weight DeepSeek completed 73%, with some tasks tied. Vals estimated token costs of $17–$46 for GPT-6-Sol attacks versus $2–$3 for DeepSeek, while experts cautioned that the simulations do not establish real-world impact or whether AI creates a new cyber threat.

In-depth insight

The measured results indicate that open-weight models can already approach leading closed models on complex cyber tasks while costing substantially less in tokens.

Open-weight artificial intelligence models are now almost as capable of hacking into computer networks as the best closed ones, according to new research that heralds a much-anticipated turning point in the cybersecurity landscape.

Vals, a company that measures AI capabilities, set out to test how well different AI models could carry out complex, multi-step network instructions on mock corporate networks.

“The best frontier models, locked up inside the labs and available only through trusted access programs, are now starting to be matched by open models that anybody can download,” says Nikil Ravi, a researcher at Vals.

The company’s findings suggest that closed models no longer have a definitive edge in cyber skills, meaning recent efforts by frontier labs to limit access to their models, and thereby slow the proliferation of such capabilities, may potentially be moot.

The findings are also noteworthy given that the US frontier labs, particularly Anthropic, premised the limited release of new models—beginning with Claude Mythos Preview in April—on the idea that gated access would only buy time before advanced hacking capabilities were broadly available.

#Vals#OpenAI#Daybreak#DeepSeek#GPT-6-Sol#Anthropic#Claude Mythos Preview#Edera

Original · 4 min readRead the original
Group chat

No comments yet. Start the conversation.