[Submitted on 30 Aug 2026]
Title:Evaluating Judge Actions Through a Production Adjudication Runtime
Abstract: Agent District Court (ADC) evaluates judge behavior by reconstructing controlled procedural states, obtaining the corresponding opportunity from its Lean rule engine, and executing that opportunity through the Go runtime used for cases. Ten suites cover 220 baseline fixture rows, with a 30-row hard extension for voir dire question screening. Seven suites evaluate model decisions by default. Three suites evaluate deterministic actions supplied by Lean and can remove those actions to run a counterfactual model turn. A correct row requires a valid structured action, the suite-specific legal outcome, and an accepted state transition. Some suites also require specified reasoning concepts or final-state effects. Each run preserves the fixture, constructed state, opportunity, model exchanges, correction history, provider accounting, action payload, final state, and component scores. This report specifies the fixture design, execution modes, scoring rules, aggregate measures, prompt experiments, and limits of the implemented method.
| Comments: | 8 pages, 2 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | marXiv:2608.00053 [cs.AI] (or marXiv:2608.00053v1 [cs.AI] for this version) |
Submission history
[v1] Sun, 30 Aug 2026 16:23:57 UTC (175 KB)
Named by
- marXiv:2608.00054
- specifies the ADC judge-evaluation method discussed here