Explore our new Cybersecurity Terms Glossary, explained by our experts.

View the Glossary View the Glossary
PRIVIA

Cybersecurity Terms Glossary

What Is LLM Security?

LLM security is the practice of securing applications built on large language models, where the untrusted input is natural language and the component you cannot patch is the model itself.

Illustration for the cybersecurity term LLM Security
Home / Blog / Cybersecurity Terms / What Is LLM Security?
17 November 2026

What Is LLM Security?

What Is LLM Security?

LLM security concerns the applications organisations actually ship: a support assistant, a document search tool, a coding helper, an internal agent wired into half a dozen systems. The model is usually somebody else’s, running on somebody else’s infrastructure, and your security work sits in the design around it—what reaches the context window, what the model is allowed to touch, and what happens to its output.

Three Properties That Break Old Assumptions

First, instructions and data share a channel. Everything the model receives is text, so there is no equivalent of a parameterised query to separate developer intent from user content. Second, behaviour is probabilistic: the same input can produce different output, so a control that works in testing may not work every time, and findings are expressed as rates rather than as reproducible steps. Third, you cannot patch the component. When a model behaves badly, the remedies are a prompt change, a filter, or a different model—none of which is a fix in the sense a security team is used to.

The OWASP LLM Top 10 as a Working List

The 2025 edition of the OWASP Top 10 for LLM Applications is the most practical risk inventory available. Prompt injection leads at LLM01, followed by sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, and unbounded consumption. Two of those deserve emphasis because they are routinely underestimated. Improper output handling is a conventional vulnerability wearing new clothes: model output rendered into a page without encoding produces cross-site scripting, and output concatenated into a query produces SQL injection. Unbounded consumption is the commercial risk nobody budgets for: a loop or a crafted prompt that consumes tokens until the invoice arrives.

Containment Architecture

Operating It

Log prompts, retrieved sources, tool calls, and responses, with enough fidelity to reconstruct an incident and enough classification discipline that the log does not become an unmanaged copy of sensitive data. Feed anomalies into the same SIEM as the rest of the estate, alerting on unusual retrieval volumes, outbound destinations never seen before, and repeated refusals that suggest someone is probing. Pin model versions where the provider allows it and treat a version change as a change requiring regression testing, because behaviour shifts without any deployment on your side.

Testing and UK Obligations

Test both layers. Conventional application testing covers authentication, authorisation, API abuse, and dependencies; AI-specific testing covers injection through every ingestion channel, data leakage, jailbreak resistance, unsafe tool use, and output handling. Retest on model, prompt, corpus, and connector changes rather than annually. On the compliance side, UK GDPR applies the moment personal data enters a prompt or a retrieval corpus, with Article 32 security obligations, a likely Data Protection Impact Assessment, and Article 22 safeguards where the system materially drives a decision about someone. Financial entities bring AI-enabled services into their DORA ICT risk framework, and anyone in scope of the NIS Regulations should treat a model embedded in an essential service as part of that scope. NCSC and CISA guidance on secure AI development remains the baseline for design and operation. At Privia Security, we assess language model deployments end to end through our AI Penetration Test Services.

FAQ – Frequently Asked Questions

Is LLM Security Different From Application Security?

It extends it rather than replacing it. You still need authentication, authorisation, dependency management, and secure configuration, and most LLM applications fail on those before anything model-specific. The additions are instruction-level threats, probabilistic behaviour, a component you cannot patch, and output that must be treated as untrusted input by everything downstream.

What Is the Single Most Important Control?

Limiting what the model can reach. Scoped credentials, allow-listed tools, and permission checks enforced at retrieval time determine the impact of every other failure. A perfectly filtered assistant with broad standing access to your document estate is more dangerous than a weakly filtered one that can only read a product manual.

Do We Need Special Logging for LLM Applications?

Yes, and it needs deliberate design. Record prompts, retrieved sources, tool calls with arguments, responses, model and prompt versions, and any human approval, because without those an investigation cannot reconstruct what happened. At the same time, classify and retain those logs properly; a full prompt archive can quietly become your largest store of unmanaged personal data.

How Do We Secure a Model We Did Not Train?

Focus on your side of the boundary. Control what data you send, which systems the model can reach, what permissions it holds, how output is validated, and what you log. Review the provider’s terms on training, retention, and residency, pin the model version where possible, and test your own deployment adversarially rather than relying on the provider’s safety claims.

Related Terms

You May Be Interested In These

What Are Anti-Money Laundering Controls?

Cybersecurity Terms

17 November 2026

What Are Anti-Money Laundering Controls?

Read More
What Is the SLSA Framework?

Cybersecurity Terms

17 November 2026

What Is the SLSA Framework?

Read More
What Is IDOR?

Cybersecurity Terms

17 November 2026

What Is IDOR?

Read More