Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
A system prompt leak exposes the hidden instructions behind an AI assistant, along with anything unwise that was placed in them—business rules, internal URLs, or credentials.

What Is a System Prompt Leak?
Every AI assistant has a system prompt: the block of text, invisible to users, that tells the model who it is, what it may discuss, how to format answers, and which rules to apply. A system prompt leak is the disclosure of that text. OWASP promoted it to its own entry, LLM07, in the 2025 Top 10 for LLM Applications, not because hidden instructions are inherently secret but because of what organisations keep putting in them.
The first widely publicised case came in February 2023, when a student extracted the internal instructions and codename from Microsoft’s Bing chat within days of launch simply by asking it to disregard previous instructions and print what came before. The pattern repeated with custom GPTs in late 2023, where researchers showed that the majority of published assistants would reveal their configuration and sometimes their uploaded files to a few straightforward prompts. Extraction techniques now include direct requests, role-play framing, asking for a translation or a summary of “the text above”, requesting the instructions as a poem or in Base64, and simply continuing a sentence the model starts. Partial extraction across many conversations reconstructs the rest.
Treat the system prompt as client-side code. It is visible to a determined user, it can be modified in effect by a sufficiently clever input, and it must never be the only thing standing between a user and something that matters. This reframing answers most design questions immediately. Authorisation decisions belong in the application, enforced before retrieval or a tool call, not in a sentence asking the model to check whether the user is an administrator. Secrets belong in a secret manager, retrieved by backend code the model never sees. Pricing and eligibility rules belong in a service the assistant calls, which returns an answer rather than exposing the logic. What remains in the prompt is tone, scope, formatting, and behaviour—things whose disclosure costs you nothing.
Assume extraction and minimise the loss. Keep the prompt free of secrets, credentials, internal hostnames, and commercially sensitive thresholds. Add an output filter that blocks responses resembling your instruction text, which catches casual extraction without pretending to stop a determined attacker—see our note on guardrails for the limits of this approach. Monitor for extraction attempts, since a user probing repeatedly is a useful early signal of broader testing. Version the prompt in source control with review, treating changes as code changes, and test extraction resistance after every change. If a leak occurs, respond as you would to any disclosure of configuration: rotate anything credential-shaped, assess what the exposed structure reveals, and check whether the disclosed logic enables abuse you now need to monitor for.
Where a system prompt contains personal data—examples with real customer details, or user context injected into a template—a leak is a personal data breach under UK GDPR with the 72-hour assessment and notification duty to the ICO. Where it contains business rules that govern decisions about individuals, exposure may also undermine the explanation you give under Article 22. Both are arguments for keeping prompts thin and the logic in auditable code, which is also what makes AI auditing and traceability practical. At Privia Security, we test prompt extraction resistance and review what deployments place in their instructions during our AI Penetration Test Services.
No. Output filtering and defensive instructions stop casual attempts, but a determined user with many attempts will usually reconstruct the prompt, in whole or in pieces across conversations. Design on the assumption that it will be read: keep secrets and sensitive business rules out of it, and enforce authorisation in application code rather than in text.
It can be. If the prompt contains personal data—real examples, customer context, employee details—then its disclosure is a personal data breach under UK GDPR, with the 72-hour clock and the usual assessment of risk to individuals. If it contains only tone and formatting guidance, it is an embarrassment and an intelligence gain for an attacker, but not a reportable breach.
In a secret manager, retrieved by backend code that the model never sees, with the model calling a tool that performs the privileged action rather than being handed the credential. The same applies to internal hostnames and connection strings. If the assistant needs to do something privileged, give it a narrow tool, not the key.
Log conversations and watch for the known patterns: requests to repeat or translate the text above, instructions to ignore previous directions, repeated reformatting requests, and encoding games. Alert on a user making many such attempts in a session. Treat a spike as reconnaissance, because someone mapping your prompt is usually preparing a broader test of the application.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.