Stumbled onto https://claw-eval.github.io/#/ while looking for reliable agent benchmarks. Turns out it's one of the most well-designed eval frameworks out there. 300 human-verified tasks, 19 mock services with error injection, full trajectory auditing. Three things that stood out: 1/ Capability ≠ Consistency. DeepSeek V4 Pro and Gemini 3.1 Pro tie Claude Opus 4.6 at 82.9% Pass@3 (ceiling) — but drop to 59.8% and 57.8% on Pass^3 (reliability). Peak performance means nothing if you can't reproduce it. 2/ The quality of your questions matters 12x more than quantity. In multi-turn agent tasks, question precision correlates at r=0.87 with success. Round count? r=0.07. Stop adding turns. Start asking better. 3/ Every model is mediocre at something. Video: Claude leads. Docs/images: GPT-5.4. Code gen: MiMo-V2-Omni. No single model dominates all modalities - the "one model to rule them all" narrative doesn't hold. Paper: http://arxiv.org/abs/2604.06132 Code: http://github.com/claw-eval/claw-eval Shoutout to @_TobiasLee and the team (PKU + HKU) for building this!(Jiawei)