Model Bench

Wed, 15 Jul 2026 00:00:00 +0000

Overview

A benchmarking site I built to answer one question well: which subscription-based LLM is worth its slot on my dashboard, and which pay-as-you-go API is the better substitute. Every public benchmark either (a) scores only pay-as-you-go APIs (so the results don’t tell me what a Claude Max / ChatGPT Plus / ollama-routed model would actually do on my workloads) or (b) makes vague quality claims with no cost column (so I can’t compare a $20/month subscription to a $5 in API spend). This benchmark measures both, on the same tasks, in the same week, scored by a judge from a different model family.

Evaluation on Herman

Model Bench

Overview