A new study examines how much authority one AI agent should have when reviewing another’s decision. It compares broad override authority with an evidence gate that allows a reviewer to replace a proposal only when it provides a mechanically valid record showing that the proposal violates a binding constraint and that its replacement complies.
The researchers tested two locally run open-weight models across 96 spatial-policy tasks and a 96-task portfolio-transfer benchmark. In the main comparison, both review rules received the same reviewer output; only the rule determining whether the recommendation could take effect changed. The evidence gate sharply reduced cases where a correct proposal was replaced with an incorrect one, while retaining most repairs of constraint violations.
The study also tested a review interface that asked models to choose whether to retain, replace or escalate a proposal. In this prospective test, the interface eliminated corruption of correct proposals for Qwen, while Mistral still replaced all 48 correct proposals. Qwen also declined every welfare-only repair that Mistral frequently attempted.
The authors emphasize that the gate’s effects depend not only on its rules but also on the reviewer’s behavior and the states of the proposals it receives. It can protect correct decisions and binding constraints, but it cannot improve a compliant proposal that is suboptimal. The findings therefore do not establish that restricted review is always better: reviewer capability, interface design and the authority to make recommendations binding all shape the outcome.
Sources: arXiv cs.AI.