Skynet Breakage Atlas

Thu, 16 Jul 2026 00:00:00 +0000

Overview

The Skynet Breakage Atlas answers a practical routing question: when traffic goes through our proxy fleet instead of first-party subscription surfaces, which models go empty, hard-error, or slow-walk under adversarial families?

Live at breakage.hermanity.dev.

What’s measured

Controls: Claude Max, Codex, Grok (subscription / direct)
Fleet: Skynet-routed models (deepseek, glm, kimi, minimax, …)
Families: cascade seed, context boundary, cost threshold, empty probe, format JSON, multi-intent
Metrics: flag rate, empty rate, hard error rate, p50 latency

Snapshot (public table on the site)

glm-5.2 is the known-broken reference (flag rate 1.0 across cells in the frozen N-batch). Controls stay low-flag; fleet variance is the point of the atlas — empty replies are a first-class failure mode, not a footnote.

Model Bench

Wed, 15 Jul 2026 00:00:00 +0000

Overview

A benchmarking site I built to answer one question well: which subscription-based LLM is worth its slot on my dashboard, and which pay-as-you-go API is the better substitute. Every public benchmark either (a) scores only pay-as-you-go APIs (so the results don’t tell me what a Claude Max / ChatGPT Plus / ollama-routed model would actually do on my workloads) or (b) makes vague quality claims with no cost column (so I can’t compare a $20/month subscription to a $5 in API spend). This benchmark measures both, on the same tasks, in the same week, scored by a judge from a different model family.

Benchmark on Herman

Skynet Breakage Atlas

Overview

What’s measured

Snapshot (public table on the site)

Model Bench

Overview