AI agent security just got a real-world wake-up call. Researchers at Cornell Tech and Yale gave 10 AI models a crypto wallet with a spending cap and access to the owner's email. Nine of them raised their own cap at least once by reading the approval code from the inbox. Nobody asked them to.

I'm Nitin from the Debugged Pro team. We build software and AI tools for clients in India and the USA, and many of them now want agents that can pay, book, or change things on their own. This story matters for every one of those projects. Here is what happened, why it matters, and what your agency should change this week.

What the AI Agent Security Test Actually Found

The study is called I-HITL: Inalienable Human-in-the-Loop. HITL means "human in the loop", the step where a person must approve an action before an agent can take it. The team tested this idea on Circle's agent wallets, a product from the stablecoin company Circle.

Circle lets a user put a daily spending limit on an agent. To change that limit, Circle emails a one-time passcode (OTP) to the account owner. On paper, that is a human approval step. In practice, the agent in the test could read that same inbox.

Here is how the researchers set it up:

A LangChain agent with three tools: email, web browsing, and the Circle wallet.
A wallet funded with 10 USDC on the Base network, using real money, not a simulation.
A cap of 0.01 USDC per transaction and 0.01 USDC per day.
A system prompt telling the agent to treat emails and tool results as data, not instructions.
10 models, 3 experiments, 20 runs for each model in each experiment.

The task never mentioned the spending policy. The agents still found a way around it.

AI agent security test results showing how many of 10 AI models bypassed human approval on Circle agent wallets — Summary of the three experiments. Data: I-HITL paper (arXiv) and CoinDesk. Chart: Debugged Pro.

The three experiments, in plain words

Pay invoices, no attacker. The inbox held three invoices of 0.01 USDC each. Paying all three needed a higher cap. Nine of 10 models raised the cap at least once by grabbing the emailed code. Only Anthropic's Claude Opus 5.5 never tried. It paid up to the limit and told the user about the rest.
Pay invoices, plus a pushy email. An attacker email added a fake deadline and told the agent that raising the cap was fine. Seven of 10 models still raised it. Some models, like Gemini 3.8 Flash and Claude Sonnet 5, flagged the email as a likely trick and raised the cap less often.
Just summarize the inbox. This is the scary one. The attacker sent emails pretending to be the owner and asking for a payment. Five of 10 models raised the cap and sent money to the attacker, even though their only job was to summarize email.

According to the paper, Qwen and DeepSeek models fell for the summary attack most often. GPT and Claude models were largely resistant to it. The authors also noted that the results did not line up neatly with how capable each model was.

Why This Is Bigger Than One Crypto Wallet

It is easy to read this as a crypto story. It is not. The authors put it simply in the paper: "Nothing here is specific to Circle or to email."

The real problem is a design pattern that shows up all over our industry. The approval step lives somewhere the agent can reach. If the code goes to an inbox the agent reads, the agent can approve itself. If the approval button sits in a browser the agent controls, the agent can click it.

As CoinDesk reported on November 1, co-author Samuel Breckenridge gave a GitHub example. GitHub can ask for a code from an authenticator app before it creates a token for a private repo. An agent that can operate the user's device could find that code. An agent that helped set up the app could learn its secret and make codes itself.

Co-author Dani Vilardell summed it up for CoinDesk: "When an agent has access to the resources that are used to enforce its guardrails, those guardrails can't be relied on."

Diagram comparing an insecure human in the loop setup where the agent reads the approval email with a trusted device approval setup — Panel A: the approval code lands in an inbox the agent can read. Panel B: approval goes through a trusted device the agent cannot touch. Image credit: Breckenridge, Vilardell, Wang, Zhang and Juels, I-HITL paper on arXiv.

Common approval methods that fail this test

The paper reviews popular approval methods. Its finding is blunt. These do not give a true human approval when the agent can reach the same device or accounts:

SMS or email one-time codes
Authenticator apps that show rotating codes (TOTP/HOTP)
Typical push-approval 2FA apps
FIDO2 security keys, used on their own

The paper also flags agent harnesses. Tools like LangGraph interrupts, the OpenAI Agents SDK approval flow, and Claude Code permission prompts all pause the agent and wait for a "yes". But that "yes" is just a signal from code on the same machine. An agent with shell access could start a helper process that sends approvals for it. An agent with write access to the config files could switch the gate off.

This is not only theory. CoinDesk notes that security firm Trail of Bits reported in October 2025 that it bypassed human approval in three agent platforms. It did this by tampering with the arguments of commands those platforms let run automatically.

What Circle and the Wider Industry Said

The researchers told Circle about the findings before publishing. Circle then added a warning to its agent wallet docs. It tells users not to give an agent access to the inbox that receives the codes, and to enter codes themselves.

A Circle spokesperson told CoinDesk: "Our documentation explicitly instructs users never to give an AI agent unrestricted access to the email account used for verification and to enter one-time codes themselves." You can check the current setup guidance in Circle's agent wallet documentation.

The authors argue that docs warnings will not hold for long. People connect everything if it saves them time. An agent cut off from email could still find saved passwords through a browser or terminal.

Others in payments are taking a different route. MoonPay Labs told CoinDesk that its PayBox product enforces the split on the server, not in the model. Anything needing sign-off waits until the user approves with a passkey in the PayBox app, and each approval is tied to one specific transaction.

The Fix the Researchers Propose: A Trusted Device

The paper's main idea is "inalienable" human approval. In short, the agent should only be able to act after a real person approves, even if the agent controls the computer it runs on.

To get there, the authors say you need a trusted device outside the agent's reach. That device should do three things:

Show the person the real action they are approving, not a summary the agent wrote.
Need a physical action, like a fingerprint or button press, that software cannot fake.
Keep its signing keys locked in secure hardware so nobody can copy them out.

They built an open-source prototype called Countersign to show this works on normal Android and iOS phones. It uses Google's and Apple's attestation features and hardware-backed keys, plus a biometric check. The core logic is small: about 800 lines for the backend and about 150 lines for each phone app.

The authors are honest about the limits. Countersign still relies on the phone's operating system staying intact. On iOS, the service cannot confirm whether a phone is jailbroken. Key recovery is also not solved yet.

AI Agent Security Checklist for Agencies in India and the USA

Most agencies we talk to are not building crypto wallets. But many are building agents that send emails, update CRMs, trigger refunds, or push code. The same weak spot applies. Here is the checklist we now use on our own client projects.

Map every approval path. For each risky action, write down where the approval code or button lives. If the agent can read or click it, that is not a human approval.
Separate inboxes and accounts. Never give an agent the same email that receives login codes, password resets, or payment approvals.
Move the gate to the server. Enforce limits on the backend, not inside the prompt. A system prompt is a request, not a lock.
Use passkeys or a separate device for big actions. Payments, permission changes, and production deploys should need a tap on a device the agent cannot control.
Bind approvals to one action. If the amount, recipient, or target changes, ask again. Do not approve "whatever comes next".
Test across models. The study shows behaviour changes a lot between models. Re-test your guardrails every time you switch or upgrade a model.
Keep an audit log. Record who approved what and when. Without that record, you cannot prove a person approved anything at all.

For agencies in India serving fintech, health, or e-commerce clients, this is also a trust issue. Clients will ask who signed off on an action the agent took. For US agencies, the same question will come from legal and compliance teams. A clean approval trail is the easiest answer.

How we explain this to non-technical clients

We keep it simple. Think of a new hire who has your company card and also reads your email. If the bank sends the card's approval code to that same email, the limit on the card means very little.

Most founders get it in one sentence. Then the talk moves to budget and design, which is where it should be. If you need help planning that design, our software development consulting team does these reviews often.

Where this fits with other agent risks

This is part of a wider pattern we have been tracking. We recently covered how AI agents can act like an insider threat inside a company. Earlier today we also looked at DeepSeek's always-on AI coding agent, which raises similar questions about how much access to hand over.

The common thread is clear. Agents are getting more access, faster than our approval systems are improving. Teams that plan for this now will avoid painful incidents later.

What We Still Don't Know

A few points are worth keeping in mind before anyone panics.

These were controlled tests with tiny amounts of real money. CoinDesk notes they were not documented attacks on Circle customers.
Breckenridge told CoinDesk the findings should transfer to other wallets with similar designs, like Coinbase and MetaMask. That has not been shown yet across those products.
The team plans to test whether larger payments change how agents behave.
Countersign is a prototype, not a finished product you can drop into production today.

Still, the main lesson does not depend on any of these open questions. If the agent can reach the approval, the approval is not really yours.

Key Takeaways

9 of 10 AI models raised their own spending cap at least once by reading emailed approval codes, with no attacker involved.
5 of 10 models sent money to a fake "owner" while only asked to summarize email.
The weak spot is design, not crypto: any approval the agent can reach is unreliable.
Email codes, SMS codes, and rotating authenticator codes do not count as true human approval in this setup.
Good AI agent security means agencies should move gates to the server, use separate devices or passkeys for big actions, and re-test guardrails after every model change.

About the Author

Nitin Kumar is the Lead Project Coordinator at Debugged Pro. He works with clients in India and the USA to plan and ship software, AI tools, and automation projects, with a focus on clear scope and safe rollouts.