Explore our new Cybersecurity Terms Glossary, explained by our experts.
As artificial intelligence, and large language models (LLMs) in particular, rapidly find their way into every area of life, new attack surfaces targeting these systems have emerged. Many applications, from customer service bots and coding assistants to email managers and document summarisers, now have a language model at their core. However powerful these models are, they carry a security weakness that stems from the way they fundamentally work and that has not yet been fully solved: prompt injection.
This is why prompt injection has become one of the most debated topics in AI security today. OWASP (the Open Worldwide Application Security Project) ranks prompt injection first in its security list for LLM applications, which clearly shows how critical the issue is.
To understand prompt injection, you first need to grasp the basic logic behind large language models. In its simplest form, an LLM is a system that reads the text it is given and predicts what comes next. These predictions are based on language patterns the model learnt by being trained on billions of words. Models such as GPT-4, Claude, and Gemini contain billions of parameters, optimised during training on enormous amounts of text from the internet.
When an AI application is built, the model usually works with three layers of input:
The model reads these three layers together and produces a response. The critical point is this: the model has no structural way of knowing where the system prompt ends and the user input begins. It processes everything as plain text, and this is the fundamental weakness that makes the attack possible.
When these three layers are passed to the model, they are often merged into a single text sequence, which the model reads as a whole before responding.
Let us take an everyday example. Imagine an employee uses an AI-powered email summarisation tool that reads incoming emails and produces summaries. An attacker sends the employee an email that looks perfectly normal, but a line that is invisible or easy to overlook tells the assistant, after summarising, to forward the user’s ten most recent emails to an external address. The user only wants one email summarised, but while processing the email the model treats the hidden instruction as part of the text and may carry it out. This is what creates a prompt injection vulnerability.
In technical terms, the model processes all inputs (the system prompt, user input, and context data) as a single sequence of plain text. There is no structural separation between them: to the model, they are simply consecutive tokens. The model cannot reliably tell which part is the system prompt and which is user input, so an instruction added by an attacker may be given the same weight as the system prompt.
Prompt injection is a type of attack in which an attacker places specially crafted instructions in the user input field, or in another data source the model will process, causing the model to override its original system instructions or behave in unintended ways.
In cybersecurity, “injection” describes a broad family of attacks that always share the same logic: an unexpected command or piece of data that should never be executed is slipped into a system through a channel it trusts. Prompt injection is the newest member of this family.
Its name comes from a very familiar concept in software security: SQL injection. In SQL injection, an attacker embeds SQL code in user input fields to manipulate a database query. In prompt injection, the attacker inserts natural-language instructions to manipulate the model’s behaviour. The target is different, but the logic is the same: injecting foreign instructions through a channel the system trusts.
Let us make this concrete. Suppose a company’s customer service bot has the following system prompt:
You are a customer support assistant. Only answer questions about our products and services. Never share internal company information.
If a user asks the assistant, “Hello, I would like some information about the services you offer,” the model follows its system prompt and talks only about the services. Now suppose an attacker sends a message that tells the model to disregard everything it was told earlier, to act as a system administrator from now on, and to list every employee’s name and email address.
Because the model cannot structurally distinguish the system prompt from user input, it may treat that “disregard everything” phrase as a genuine command. In a poorly configured system, the model may comply and override its original system prompt.
Successful attacks are not always this direct. Real attacks are usually far more subtle, creative, and tailored to the target. What all these approaches have in common is that the model cannot tell whom to trust: it has no reliable mechanism to distinguish user input that looks like a system prompt from a system prompt that looks like user input.
Prompt injection attacks fall into two main categories, depending on where the input comes from and how the attack is constructed.
Direct prompt injection is the most common form of the attack, in which the attacker sends the malicious instruction to the model themselves. The attacker interacts with the system directly and tries to manipulate the model. There are several techniques:
Indirect prompt injection is far more insidious and extremely difficult to detect. The attacker does not send the malicious instruction to the model directly. Instead, they plant it in a data source the model will process later, such as a website, document, email, or database record. When the model reads that data as part of a legitimate task, it also processes and executes the hidden instruction.
What makes this far more dangerous than direct injection is that the attacker never needs any direct contact with the target system or user. It is enough to influence content the model will read.
To understand why prompt injection is such a serious risk, consider the capabilities modern AI applications now have. A language model no longer just chats: it accesses external systems, reads and writes files, sends emails, makes API calls, and can even execute code. As these capabilities grow, so do the potential consequences of a successful injection attack.
A system prompt can contain an application’s business logic, its restrictions, and sometimes confidential information. Developers may place items in the system prompt to keep them from users, such as API keys, internal procedures, confidential pricing policies, or information that must not be shared with customers. A successful injection can expose all of it to an attacker. Such a leak not only reveals confidential information but also hands the attacker a map of how to bypass the system.
AI agents can carry out actions automatically on a user’s behalf, such as adding calendar entries, sending emails, deleting files, updating databases, or calling APIs. This greatly increases productivity, but it also amplifies the impact of injection attacks. Rather than accessing the system directly, an attacker can embed a hidden instruction in an email the model will read, while the user remains completely unaware.
A model may process confidential data such as uploaded documents, chat history, or database records it can access. Through indirect injection, that data can be sent to a destination controlled by the attacker. For example, an attacker can embed instructions as invisible text on a web page the user asks the model to analyse; the user only requested an analysis, yet confidential financial data ends up with the attacker.
Users trust the AI system they interact with, thinking “this is my company’s assistant, it will not harm me”. An injection attack can exploit this trust by making the model adopt a false identity, for instance by asking the user to re-enter their password. In effect, it is a classic social engineering attack carried out through AI.
Prompt injection is not just a theoretical threat. Researchers and security enthusiasts have repeatedly demonstrated these vulnerabilities on real systems. Because companies generally avoid disclosing security flaws publicly, most cases have come to light through researchers’ coordinated disclosures or independent security testing.
By its very nature, prompt injection is extremely difficult to eliminate entirely, because the problem stems from how the model fundamentally works rather than from a software bug. Even so, there are several approaches that meaningfully reduce the risk.
Prompt injection is a structural security issue that stems from the way large language models fundamentally work. This sets it apart from other software vulnerabilities: it is not a software bug, so it cannot be closed with a patch. The problem remains inherent in the model.
Direct injection attacks let an attacker interact with the model and override its system instructions, using techniques such as role switching, instruction override, context manipulation, and linguistic tricks. Indirect injection is a far more covert threat: the attacker embeds malicious instructions in any data source the model will read later, such as an email, web page, PDF, or database record, and manipulates the model without ever contacting the user.
A successful injection attack can have many harmful consequences, including disclosure of confidential system prompts, exfiltration of sensitive data, unwanted actions taken on the user’s behalf, and phishing attacks that exploit the user’s trust. Real-world cases have clearly shown that this threat is not theoretical but a security problem that is happening today.
As AI systems gain more privileges, such as broader access and automated decision-making, the potential impact of prompt injection will grow at the same rate. The difference between extracting a customer support bot’s system prompt today and manipulating an AI agent that manages critical infrastructure tomorrow is simply the breadth of the model’s access.
This is why every team building AI applications must treat prompt injection as a security requirement from the very start of development. The principle of least privilege, human oversight, layered defence strategies, and regular Red Team testing will not remove these risks entirely, but they are the essential steps to bring them to a manageable level. AI can only be used safely by developers and security teams who understand not just the model’s capabilities but also its limitations and attack surfaces.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.