Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
LLM security is the practice of securing applications built on large language models, where the untrusted input is natural language and the component you cannot patch is the model itself.

What Is LLM Security?
LLM security concerns the applications organisations actually ship: a support assistant, a document search tool, a coding helper, an internal agent wired into half a dozen systems. The model is usually somebody else’s, running on somebody else’s infrastructure, and your security work sits in the design around it—what reaches the context window, what the model is allowed to touch, and what happens to its output.
First, instructions and data share a channel. Everything the model receives is text, so there is no equivalent of a parameterised query to separate developer intent from user content. Second, behaviour is probabilistic: the same input can produce different output, so a control that works in testing may not work every time, and findings are expressed as rates rather than as reproducible steps. Third, you cannot patch the component. When a model behaves badly, the remedies are a prompt change, a filter, or a different model—none of which is a fix in the sense a security team is used to.
The 2025 edition of the OWASP Top 10 for LLM Applications is the most practical risk inventory available. Prompt injection leads at LLM01, followed by sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. Two of those deserve emphasis because they are routinely underestimated. Improper output handling is a conventional vulnerability wearing new clothes: model output rendered into a page without encoding produces cross-site scripting, and output concatenated into a query produces SQL injection. Unbounded consumption is the commercial risk nobody budgets for: a loop or a crafted prompt that consumes tokens until the invoice arrives.
Log prompts, retrieved sources, tool calls, and responses, with enough fidelity to reconstruct an incident and enough classification discipline that the log does not become an unmanaged copy of sensitive data. Feed anomalies into the same SIEM as the rest of the estate, alerting on unusual retrieval volumes, outbound destinations never seen before, and repeated refusals that suggest someone is probing. Pin model versions where the provider allows it and treat a version change as a change requiring regression testing, because behaviour shifts without any deployment on your side.
Test both layers. Conventional application testing covers authentication, authorisation, API abuse, and dependencies; AI-specific testing covers injection through every ingestion channel, data leakage, jailbreak resistance, unsafe tool use, and output handling. Retest on model, prompt, corpus, and connector changes rather than annually. On the compliance side, UK GDPR applies the moment personal data enters a prompt or a retrieval corpus, with Article 32 security obligations, a likely Data Protection Impact Assessment, and Article 22 safeguards where the system materially drives a decision about someone. Financial entities bring AI-enabled services into their DORA ICT risk framework, and anyone in scope of the NIS Regulations should treat a model embedded in an essential service as part of that scope. NCSC and CISA guidance on secure AI development remains the baseline for design and operation. At Privia Security, we assess language model deployments end to end through our AI Penetration Test Services.
It extends it rather than replacing it. You still need authentication, authorisation, dependency management, and secure configuration, and most LLM applications fail on those before anything model-specific. The additions are instruction-level threats, probabilistic behaviour, a component you cannot patch, and output that must be treated as untrusted input by everything downstream.
Limiting what the model can reach. Scoped credentials, allow-listed tools, and permission checks enforced at retrieval time determine the impact of every other failure. A perfectly filtered assistant with broad standing access to your document estate is more dangerous than a weakly filtered one that can only read a product manual.
Yes, and it needs deliberate design. Record prompts, retrieved sources, tool calls with arguments, responses, model and prompt versions, and any human approval, because without those an investigation cannot reconstruct what happened. At the same time, classify and retain those logs properly; a full prompt archive can quietly become your largest store of unmanaged personal data.
Focus on your side of the boundary. Control what data you send, which systems the model can reach, what permissions it holds, how output is validated, and what you log. Review the provider’s terms on training, retention, and residency, pin the model version where possible, and test your own deployment adversarially rather than relying on the provider’s safety claims.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.