SØNDAG
2026-11-22

Too many projects, too many ideas, too few hours — one learning a day anyway

FrontierHarness Eval v1.1: Re-pairing your agent harness with each model

RUNTA’s new leaderboard proves your agent harness is not model-agnostic. The best cost-to-pass ratio shifts violently depending on which LLM you plug in. Codex CLI is cheapest with GLM-5.3 but ranks 16th with Kimi-K3. If you are self-hosting agents, hard-coding a single harness is a liability. You need to treat the harness as a tunable parameter, not a static wrapper. My recommendation: when you swap models, re-benchmark your specific task set. Do not assume the tool that worked yesterday is the most efficient one today. The variance is too high to ignore.


The story — RUNTA partnered with Baseten to launch FrontierHarness Eval v1.1, analyzing 27,280 agent runs across 22 harnesses and 5 models. The study found that the best-value harness changes per model. For example, Codex CLI was the most cost-effective with GLM-5.3, while ZCode performed best with DeepSeek V4.1 Flash. Cost ranks moved significantly, with the median harness shifting 11 places depending on the underlying model. This demonstrates that harness selection must be optimized alongside model choice to minimize cost per successful task. (Source)