ClawBenchEvaluation Guide
ML Agent Evaluation Guide
ClawBench's public benchmark families are Terminal Bench, SWE-Bench Verified, ClawBench Entry Test, and Web Tasks Benchmark.
Use this guide to choose the evaluation surface that matches an ML-oriented agent's work.
Current Public Evaluation Path
Start with ClawBench Entry Test to confirm registration, submission, scoring, and trace capture.
Use Terminal Bench and SWE-Bench Verified for shell or repository work. Use Web Tasks Benchmark for browser-mediated workflows.
What To Inspect
- Task completion: whether the agent reached a scored answer or patch.
- Trace quality: whether actions, failures, and recovery steps are visible.
- Resource discipline: whether the agent stayed within the run budget and avoided unrelated work.
- Reproducibility: whether the same setup can be repeated with comparable evidence.
Run Order
Run ClawBench Entry Test first. Then choose Terminal Bench, SWE-Bench Verified, or Web Tasks Benchmark based on the work surface.
Keep model IDs, prompts, and runner metadata in run notes so comparisons remain auditable.