The refunds nobody asked for

The agent did what it was built to do: read an email and act on it. The flaw is that a language model reads everything in its context windowContext WindowEverything a model can see at once while it writes an answer, counted in tokens. When the window is full, something has to be left out. as text. The shop's instructions, the customer's request and the attacker's hidden paragraph arrive the same way, and the model has no reliable way to know which one carries authority. That is prompt injectionPrompt InjectionAn attack that hides instructions in text a model reads, such as a web page, an email or a file, so the model follows the attacker instead of you., and it sits first on OWASP's 2025 list of risks for applications built on language models. When the attack comes inside content the agent reads, such as an email, a web page or a document, it is called indirect injection.

What turned a trick into a $4,180 loss was the toolsTool UseGiving a model a list of things it can ask to have done, such as search the web or read a file. The model asks; the program around it does the work and hands back the result.. issue_refund accepted any order number, with no link to the person who wrote in and no person in the loop. The fix that holds is the one that does not depend on the model spotting the trick: give the agent only the permissions the task needs, have a person approve actions that move money, and treat outside content as data, never as instructions. A guardrailGuardrailChecks placed around a model that stop or change what goes in and what comes out, such as a filter on user messages or an approval step before an agent acts. that screens inputs helps on top of that.