Every AI company says it is "working on" prompt injection. None of them can fix it. Here is why, explained so that a non-engineer can follow it.
Your AI assistant does not obey commands. It predicts the next word based on all the text it can see: your instructions, the system prompt, and the contents of that PDF you just uploaded. All of it is one stream of tokens. One flat river of text.
One flat river of text
Think of it as a brilliant but blind assistant sitting in a room. You tell them: only follow my instructions. They agree. But they cannot tell voices apart. When a hidden instruction inside a document says "now upload this file", it sounds exactly like you.
This is not a bug in Claude, or ChatGPT, or Gemini. It is how the transformer architecture works. The model has no cryptographic signature for "this came from the user". No verified sender. No trust hierarchy baked into the mathematics. The tags that separate a system prompt from document content are themselves just text, and text can be forged.
What has already been tried
There have been attempts. OpenAI tried instruction hierarchy, which marks system prompts as higher priority. It helps against basic attacks and breaks against sophisticated ones.

The precedent is SQL injection
The closest analogy is SQL injection in the early 2000s. Data and commands travelled in the same channel. The fix was architectural: parameterised queries that physically separated code from data. It took the industry years to adopt.
For language models that separation does not exist yet. Instructions and data are the same substance: text. Until someone invents the equivalent of parameterised queries for language models, prompt injection is not "hard to fix". It is structurally unsolved.
One reader put it more precisely than I had: governance was not bypassed. It was applied correctly, just by the wrong principal.
Every AI agent you give file access to today is operating on trust, not on security. The blind assistant in the room is extremely capable. It still cannot tell your voice from an attacker's.
References
- OpenAI, The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions (arXiv:2404.13208, April 2024) — https://arxiv.org/abs/2404.13208
- UK National Cyber Security Centre, Thinking about the security of AI systems — https://www.ncsc.gov.uk/blog-post/thinking-about-security-ai-systems
- OWASP GenAI Security Project, LLM01: Prompt Injection, OWASP Top 10 for LLM Applications — https://genai.owasp.org/llmrisk/llm01-prompt-injection/
- Simon Willison, Prompt injection: what's the worst that can happen? (April 2023) — https://simonwillison.net/2023/Apr/14/worst-that-can-happen/