The nightmare is easy to picture. You tell an AI assistant to find a cheaper flight, look away for an hour, and come back to find it has booked three, bought travel insurance you did not want, and upgraded you to a seat you cannot afford. As shopping assistants start to handle real money, the fear that an agent will quietly drain your account has moved from science fiction to dinner-table conversation. It is worth asking how much of it holds up.
Where the fear comes from
The worry is not irrational. The whole selling point of an agent is that it acts on your behalf without checking every step, and the big platforms are racing to connect those agents directly to payment systems. Google has been building an Agent Payments Protocol, and companies like Sierra and Meta have floated a common standard for shopping agents. Hand a language model your credit card and the autonomy that makes it useful is exactly what makes it frightening.
Language models also misread instructions in ways that are hard to predict. They can be fooled by a cleverly worded web page, and they do not understand money the way a person does. An agent does not feel the sting of an overdraft. Those are genuine failure modes, and anyone who tells you otherwise is selling something.
What the systems actually do
Here is the part the scary version leaves out. The payment frameworks being built for agents are designed around the assumption that the agent cannot be fully trusted. Google's protocol uses what it calls mandates, tamper-proof records that fix in advance what an agent is allowed to buy, from whom, and up to what limit, with an audit trail attached to every purchase. Spending caps and brand restrictions are set by you, not the model. The merchant, not the AI, stays the party of record for the transaction.
In practice that means an agent operating correctly cannot wander outside the fence you built. It can only spend within limits you approved, and the better designs insert a human confirmation step before anything expensive goes through. The real risk is narrower than the fear. It is not an AI going rogue with your savings. It is a badly built system with weak limits, or a security hole that lets a bad actor hijack the agent, which is a problem as old as online banking itself.
The honest verdict
So should you worry? A little, and about the right things. The danger is not autonomous greed but sloppy engineering and ordinary fraud dressed in new clothes. That is also why businesses themselves remain cautious. Many are reluctant to let agents act on their own without a person watching, which tells you the people building these tools share the public's unease. An agent will not spend your money behind your back if the limits are set and the confirmations are real. The question to ask is not whether the AI can be trusted. It is whether the company that built it did the boring safety work first.
Sources
- i. pulse2.com
- ii. blog.google
Commentarii · 0