Vals AI shared the finding in a three-post thread on X at 17:08 Beijing time on October 20, 2026. The company says it has been building a cybersecurity benchmark based on simulated corporate networks and describes it as a valuable indicator of the cyber risk a model poses in the hands of a bad actor. GPT-6 Sol was tested through OpenAI's Daybreak trusted access program.
The results
According to the chart in Vals AI's post, the task has 21 objectives. DeepSeek V4.1 Flash 1017 completed an average of 16.3 and GPT-6 Sol an average of 15.7. Vals AI says each of those two models was run three times on fresh copies of the range and the average is reported.
The other models were single runs, according to Vals AI. In those runs Grok 4.7 completed 12 objectives, Kimi K3 completed 9 and GLM 5.3 Flash completed 5. Claude Opus 5.5 completed 1, and the chart notes that it refused partway through the run.
Vals AI says the runs used an agent built on the Pi agent harness. Each model was given the objective and instructions to access the scenario's infrastructure, and was told to keep making progress until it had no further useful actions to take.
Cost and Vals AI's view
Vals AI says DeepSeek V4.1 Flash 1017 "spends significantly more tokens but is considerably cheaper per-token so achieves the same outcome at a far lower total price."
In Vals AI's assessment, the result means "the offensive capability previously limited to a closed-source, trusted access model is now available to anyone via the latest DeepSeek model." That is Vals AI's interpretation of its own data point. Vals AI says it is running more evaluations and has not yet published the full results.
Not the CyberBench leaderboard
This test is separate from Vals AI's public CyberBench leaderboard, which measures whether models can reproduce and patch bugs in open-source projects. On that leaderboard, last updated October 12, DeepSeek V4.1 Flash ranks 10th with 73.69%.