Abstract
Human oversight is a load-bearing premise of AI governance: in several regimes, human review determines which obligations apply. We examine when this premise fails for verification-based oversight, in which a designated human checks AI outputs before they take effect. It fails when accountability for detected errors falls on reviewers or deployers, reviewer capacity is bounded, and output from open populations persistently outgrows that capacity. Review then becomes nominal, and recent experiments show that fluent AI output undermines error recognition. Because accountability and feedback respond only to detected errors, undetected harms are externalized and surface as threshold shocks. We analyze how these dynamics limit six oversight configurations. As capability grows, the share of AI output that oversight in high-loss domains can responsibly approve contracts. Friction is the one role of human involvement that keeps functioning under load; bounding inflow to reviewer capacity (flow design) is a necessary complement to other safeguards.