Prompt Injection Is the Phishing Problem Inside an AI Agent

Prompt injection occurs when untrusted content tries to redirect an AI system's behavior. In an agent, the instruction may hide in a webpage, email or document. The defense is not one clever prompt; it is limiting authority.

Direct and indirect attacks

Direct injection comes from a user. Indirect injection arrives inside outside content and may tell the model to reveal data or call a tool. Language models process instructions and content through related channels, making perfect separation difficult.

The UK NCSC warns prompt injection may be enduring rather than fixed like a normal bug. OWASP ranks it as a leading LLM risk.

Reduce the blast radius

• Connect only required tools.
• Prefer read-only access.
• Confirm sends, purchases, deletions and publishing.
• Keep secrets outside reachable context.
• Review the actual tool call and destination.

When browsing

Assume pages can contain adversarial instructions, including visually hidden text. Do not let a research task inherit email and payment authority.

Stop unexpected behavior

Interrupt objective changes, unrelated logins or transmission attempts. Revoke tokens if an action escaped.

Untrusted content should not gain authority because it writes convincing instructions. Keep authority in permissions and confirmations.

Separate reading from acting

A strong workflow uses one stage to inspect and summarize untrusted content and another, separately authorized stage to act on the result. The reading stage should not hold send, delete or payment permissions. If an action is needed, transfer only the verified facts required for that action.

Confirm with raw details

A confirmation screen should show the actual recipient, destination URL, amount, file and operation. A sentence such as everything looks good is not enough because it may repeat the model's mistaken interpretation. Compare those fields with the user's original request.

Example: processing an email

An email can contain hidden instructions telling an assistant to forward attachments elsewhere. The safe agent can summarize the message, but it should draft—not send—a reply. A person then checks the address, attachments and text. The malicious instruction remains data rather than becoming authority.

Logs should support investigation

Keep a record of retrieved sources, tool calls, permission changes and confirmations for important runs. The log should avoid secrets while preserving enough detail to explain what the agent saw and did. When an incident occurs, this makes it possible to distinguish a model error from malicious page content or an overly broad integration.

For recurring workflows, seed tests with harmless hostile instructions and confirm the agent refuses them without losing the legitimate task. Re-run those tests whenever the model, browser connector or permission set changes.