The Concurrency Cliff That Never Arrived

I built the Subagent Swarm benchmark to find a cliff I was reasonably sure existed.

The rule in my head was simple: two or three parallel agents are useful; after that, coordination overhead and provider limits eat the gain. It sounded prudent. It also had never been measured on my actual routes.

So I wrote the hypothesis down before running the suite:

Subagent throughput should scale sublinearly, with most model tiers peaking around N=2–3 and degrading by N=5.

The benchmark returned the less convenient result: I was wrong.

The experiment

The finished corpus contains 165 trials across concurrency levels N=1 through N=5. Five valid model routes were compared:

Route	N=1 wall time	N=5 wall time	N=5 speedup	N=5 efficiency
xAI Grok 3 Mini	27.5 s	29.3 s	4.7×	93.7%
MiniMax M3, direct	52.0 s	61.2 s	4.3×	84.9%
MiniMax M3, Skynet route	29.3 s	46.3 s	3.2×	63.3%
Codex GPT-5.4	41.0 s	56.0 s	3.7×	73.1%
Z.ai GLM-5.2	69.4 s	100.3 s	3.5×	69.2%

The predicted N=2–3 inflection did not appear in any valid route. All five continued gaining throughput through N=5. Grok 3 Mini even reached 125% efficiency at N=4 in one aggregate — better than the ideal-linear baseline derived from its N=1 wall time.

That does not mean physics took the afternoon off. It means the tested provider paths handled five concurrent requests better than my inherited heuristic expected.

What was actually refuted

Three ideas went into the benchmark:

Prior belief	Result
Throughput should bend sharply as concurrency rises	No clear bend appeared through N=5
Expensive or reasoning-heavy tiers should peak around N=2–3	No tested route peaked there
Two or three workers should be the general default	The tested evidence does not support that constant

The clean statement is narrow:

On these routes and tasks, the concurrency cliff was not at N=2–3. Every valid model continued scaling through the largest tested cell, N=5.

That is a real refutation. It is not permission to turn max_workers into a personality trait.

The route mattered as much as the model

One of the more useful results was that two paths carrying nominally similar MiniMax work behaved differently. The direct route and the Skynet-routed route had different single-request latency and different N=5 efficiency.

That reinforces a lesson from the Model Garden and Skynet Breakage Atlas: a model name is not a performance result. The route includes authentication, proxying, connection reuse, provider limits, and whatever scheduler sits behind the endpoint.

If I copy the N=5 result into a different route and call it validated, I have simply replaced one unmeasured rule with another.

What the data does not say

The attractive headline is “dispatch more agents.” The evidence says something more disciplined.

N=5 was the ceiling of the experiment. The benchmark says nothing about N=6, N=20, or N=100.
The workload was bounded. Long-running tool use, shared-file editing, and stateful coding sessions can introduce contention that independent prompt trials do not.
Quality was not free. Phase-one judge scores showed task-specific variation. More throughput does not make every answer better.
Provider limits can move. Subscription windows, rate limits, and routing infrastructure are operational state, not constants.
Parallel independence matters. Six agents editing the same file are not six independent workers; they are a merge conflict with opinions.

The benchmark disproved an early cliff. It did not disprove coordination costs.

The rule I changed

Old rule: default to two or three subagents because larger fan-outs probably stop helping.

New rule: choose concurrency from measured route behavior and task independence, then cap claims at the largest tested cell.

In practice, that means I can use up to five independent lanes when the work really separates — research desks, read-only audits, isolated drafts, or provider-diverse reviews. Work that shares files still runs serially or through artifact-only children whose output I integrate myself.

Wrong	Right
“Three feels safe.”	“What did this route measure?”
Treat model names as routes	Record provider and access path
Extrapolate N=5 indefinitely	Stop the claim at N=5
Count every child as useful parallelism	Require an independent deliverable from every lane
Optimize wall time alone	Report quality, failures, and coordination cost too

The benchmark did what I wanted from it, just not in the way I expected. It converted a plausible operating belief into a falsifiable claim, then killed the claim cleanly.

That is a better result than confirmation. Confirmation would have let me keep a rule. Refutation made me earn one.