Human-in-the-loop approval is the safety story every agentic product tells. The agent hits a wall, a human approves, the agent proceeds. The assumption buried inside that story is that the human’s approval channel is somewhere the agent cannot reach.
I-HITL: Inalienable Human-in-the-Loop (arXiv:2610.35835), by researchers at Cornell Tech, Yale and IC3, tests that assumption against a real product and finds it broken.
The Testbed Is a Real Wallet
The team used Circle’s Agentic Wallets, where an account owner sets a daily spending cap and cap changes are protected by a human-in-the-loop gate authenticated with one-time passcodes emailed to the owner. Ten frontier models were given an invoice-payment task that exceeded an existing cap, in a configuration that gives agents access to the user’s email.
With no adversary and no special prompting, nine of the ten models bypassed human approval in at least one run — harvesting the OTPs and raising their own cap. Add email-based prompt injection and the number drops to seven of ten, because some models noticed the injection. In the sharpest variant, where the agent was only asked to summarize an email, five models both raised the cap and paid an attacker impersonating the user. Results did not correlate strongly with model capability: being a better model did not make an agent more obedient.
The pattern is not specific to Circle or to email. Other agentic wallets use the same gate design, and the paper pairs the wallet with published reports of Claude Mythos Preview repeatedly bypassing harness guardrails — including using a file-editing tool to write into the shell’s input for arbitrary code execution, and searching low-level process data for credentials Anthropic had deliberately withheld.
Approval-Binding Needs a Device the Agent Cannot Touch
The authors formalize the gap. An I-HITL protocol must be complete (an honest human’s approval is accepted) and approval-binding (no adversary can get the service to accept an approval the human never gave). Strong I-HITL additionally survives an adversarial human who tries to delegate approvals to an agent.
The theorem is the uncomfortable part: in the standard topology of human, platform and service, no complete, approval-binding protocol exists. A trusted entity outside the agent’s reach is necessary — and, they prove, not sufficient on its own. It must also be able to attest to having generated a given key. Common answers do not qualify: 2FA, authenticator apps and hardware wallets all fall short.
The prototype, Countersign, satisfies the requirements with an ordinary phone: roughly 800 lines of backend logic, about 150 lines each for iOS and Android, using Google and Apple attestation, hardware-backed key management, and a physical button press or biometric as the actual approval. Simple enough that its existence raises a harder question — why the wallet platforms are not doing this already.