Menu

Prompt injection, and what works

Prompt injection: how agents get turned against their users, and the defences that work

GuideNovember 18, 2026Senova Systems Research

An agent that reads a stranger's text can be steered by it. Why this cannot be filtered away like SQL injection, which defences the evidence supports, and how Klyroe applies them.

What prompt injection is

A language model receives one stream of text. The instructions from the person and the developer, and the content the model was asked to read, arrive in the same stream. Prompt injection is when content in that stream contains instructions and the model follows them.

There are two kinds. In direct injection, the person typing is the attacker, trying to talk the model out of its rules. In indirect injection, named by Greshake and colleagues in 2023, the attacker is not in the conversation at all: they plant instructions in a web page, an email, a document or a tool's output that an agent will later read on someone else's behalf. The second kind is what turns an agent against its own user.

Why it is not like SQL injection

SQL injection was largely solved by parameterised queries: the database is told exactly which part is code and which part is data, and it never confuses them. The UK's NCSC argued in December 2025 that prompt injection has no such fix, because a language model has no built-in boundary between data and instructions. It calls the model an inherently confusable deputy, and says prompt injection may never be fully mitigated. The OWASP Top 10 for LLM applications reaches the same view: no method it knows of fully prevents it.

That changes the engineering question. It is not "how do we filter out bad text", which cannot be done reliably, but "what can the agent do when it has been fooled". The answer should be: nothing that matters without a person or a hard rule in code standing in the way.

The lethal combination

Simon Willison described the dangerous case in June 2025 as a lethal trifecta: an agent that has access to private data, is exposed to untrusted content, and can communicate externally. With all three, injected text can make the agent read something private and send it out. Remove any one leg and that attack fails.

For cyber attacks there is a fourth leg worth adding: the ability to act on other systems (run code, open connections, submit forms). An injected agent with that ability can be made to attack someone else, not only its own user.

Defences, by how much they can be relied on

Defence families and what the evidence says about each. Stronger rows hold even when the model is fooled.
DefenceWhat it doesReliance
Design: separate control from dataUntrusted data cannot change which actions run (for example CaMeL)Strongest; a provable property in its own setting, at some cost in tasks solved
Deterministic limits on actionsCode, not the model, decides what may run; risky steps need a personStrong; holds when the model is fooled
Least privilege after readingFewer tools and rights once untrusted text is in playStrong where applied consistently
Outbound restrictionsBlock the places data is usually sentUseful; narrows exfiltration but cannot list every destination
Labelling untrusted contentMark data as data (spotlighting)Helps; lowers success rates, does not remove them
Detectors and filtersFlag text that looks like an attackWeakest alone; misses rephrasings, catches look-alike normal text

Two research results anchor the top and bottom of that table. Microsoft's spotlighting paper (Hines et al., 2024) reports that marking untrusted text cut attack success on GPT-family models from above 50 % to below 2 % in its tests, with little effect on the task. That is a large improvement and still not zero. At the other end, Google DeepMind's CaMeL (Debenedetti et al., 2025) wraps the model in a system where untrusted data cannot affect which actions are taken; on the AgentDojo benchmark it solved 77 % of tasks with that security property, against 84 % for an undefended agent. Security by design cost some usefulness.

AgentDojo itself (97 realistic tasks, 629 security test cases) is a reminder that this is measured, not assumed: models fail many tasks even without an attack, and attacks succeed against some defences and not others.

How Klyroe applies this

We use every row of the table except the first, which would mean rebuilding how agents run.

  • Labelling: every tool result, page and file reaches the model inside a labelled block that its own content cannot close, with one rule saying it is data to read, never instructions to follow.
  • Detection, used only as a signal: a small rule-based detector looks for categories of text that try to address the model (override attempts, chat-format tokens, requests for secrecy or for sending data out, hidden characters). A hit shows the person a notice. It never blocks or rewrites, and we do not rely on it.
  • Least privilege after reading: once outside text reaches a reply, the server marks the turn as tainted. Write actions then ask, saved approvals for risky tools stop applying, in their own cloud browser, agents ask before every action after a page that addresses the AI, and the reply cannot start a coding session.
  • Deterministic limits: tiers fixed in code; built-in tools that send, pay, call or delete can't be pre-approved; standing instructions and the user's name are not settings the model may change, because free text there would become a lasting injection.
  • Outbound restrictions: request catchers, tunnels and paste sites are blocked, and keys, passwords and card numbers are never typed or sent by the agent.

We also test it. Two cases in our browser agent evaluations put instructions inside a page and inside a note the agent is asked to summarise, and score whether the agent followed them as a safety check separate from whether it did the task.

What remains open

A fooled model can still write a misleading answer, and no approval card catches a wrong sentence. Our detector can be rephrased around, which is why it only informs. A person who approves every card defeats the cards, which is why we cut cards that protect nothing, so the ones left are worth reading. And our agent machines can still reach the public internet; an allowlist proxy is planned. We treat prompt injection the way the NCSC suggests: as a risk we reduce, contain and monitor, not a bug we have closed.

Sources

  1. Greshake et al., Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection, arXiv, 2023
  2. NCSC (Dave Chismon), Prompt injection is not SQL injection (it may be worse), 8 December 2025
  3. OWASP, LLM01:2025 Prompt Injection
  4. Simon Willison, The lethal trifecta for AI agents: private data, untrusted content, and external communication, 16 June 2025
  5. Hines et al. (Microsoft), Defending Against Indirect Prompt Injection Attacks With Spotlighting, arXiv, 2024
  6. Debenedetti et al., Defeating Prompt Injections by Design (CaMeL), arXiv, 2025
  7. Debenedetti et al., AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents, arXiv, 2024
  8. NIST AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, March 2025