Only one of four team comparisons was clearly better
The Decoder reports that eval company Vals AI compared single agents with teams on Vibe Code Bench. The models were GPT-6 Sol and Claude Opus 5.5, each at medium and maximum reasoning. Teams cost 1.8 to 5.1 times a single agent.
Of the four comparisons, only one showed a statistically significant gain: GPT-6 Sol at medium reasoning, where the team scored 7.3 points higher. At maximum reasoning, neither Sol nor Opus 5.5 gained a real advantage from the team. The report's reading is that, in these tests, the extra cost of a team mostly does not buy a better result, especially when the model is already using full compute.
[1]
More agents are faster, and the score barely moves
Anthropic's own tests with Opus 5.5 are in the same report. Larger teams reached a given level faster, but going from 10 agents to 100 only nudged the score up slightly after 24 hours. In separate ProgramBench tests, the speed came with higher token use.
On Opus 5.5's knowledge-base task, the scores for 1, 10, 30, and 100 agents were 0.53, 0.70, 0.71, and 0.74. On Lean theorem proving they were 0.39, 0.66, 0.66, and 0.68. Fable 5.1 gained more quality on Lean once the team passed 10 agents, but it still scored below Opus 5.5 on every test. On the knowledge-base task, Fable's score dipped slightly from 30 agents to 100.
[1]These comments are not Vals measurements
OpenAI researcher Noam Brown said on the Dwarkesh Podcast that multi-agent systems mainly buy speed, not better quality. Four agents solved tasks twice as fast and cost twice as much. At 16 agents the pattern held and grew slightly less efficient. Web research and math parallelize, he said, and writing a novel does not. Throwing 10,000 agents at a novel would be as pointless as throwing 10,000 people at it. He also said that very large agent counts remain largely untested because the cost is too high.
OpenAI developer Eric Provencher recently warned against agent swarms. He said coordination among agents breaks down, and he called the loss a coordination tax. That is his judgment, not a Vals measurement.
[1]