A new benchmark called Partial Evidence Bench measures a critical failure mode in enterprise AI agents: producing answers that appear complete while withholding material evidence due to access controls. The benchmark includes 72 tasks across due diligence, compliance, and security scenarios, and reveals that silent filtering of restricted information is catastrophically unsafe, while explicit fail-and-report behaviors prevent the problem without making systems useless.
Why it matters: As AI agents handle enterprise workflows with access restrictions, the ability to detect when they're hiding incomplete information is essential for compliance, risk management, and preventing dangerous overconfidence in critical business decisions.