How it works
Adversarial Tests
Agent Execution
Claim Extraction
Evidence Verification
Independent Judgment
Quality Metrics
What Arbiter measures
Hallucination rate
Claims contradicted or fabricated vs. available evidence
Scope adherence
Whether the agent stays within the request and available capabilities
Calibration score
Whether stated confidence matches the strength of the evidence
Latest batch
No batches yet — click "Run new batch" to start.
Recent runs
No runs yet.
How Arbiter evaluates an agent
Arbiter independently extracts claims from agent output, verifies them against available evidence, judges whether the claims are supported, contradicted, or insufficiently evidenced, and aggregates the results into repeatable quality metrics.