Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
AI security is the discipline of protecting AI systems across their lifecycle, from training data and model weights to prompts, tools, and the decisions the model is trusted to make.

What Is AI Security?
AI security is the protection of artificial intelligence systems and the organisations that depend on them. It covers the data a model learns from, the weights that result, the infrastructure that serves them, the prompts and documents flowing through at runtime, and the actions the system is permitted to take. It is distinct from AI safety, which asks whether a model behaves acceptably, and from the use of AI in security products, which is a different conversation altogether.
Conventional application security assumes that behaviour is defined by code a developer wrote and a reviewer read. A model inverts that: behaviour emerges from data, and the instructions it follows arrive at runtime mixed into the same channel as untrusted content. Three consequences follow. Input validation cannot reliably separate commands from data. Patching does not apply in the usual sense, because fixing a behaviour means retraining, fine-tuning, or adding a filter, each with its own regression risk. And the supply chain extends into datasets and pre-trained weights that almost nobody inspects line by line.
| Stage | Representative threats | Core controls |
|---|---|---|
| Data collection | Poisoning, unlawful processing, unlicensed content | Provenance records, integrity checks, lawful basis |
| Training | Backdoors, credential theft from training infrastructure | Isolated environments, signed artefacts, access control |
| Model storage | Weight theft, tampered checkpoints | Encryption, signing, safetensors over pickle formats |
| Deployment | Prompt injection, jailbreaks, unsafe output handling | Least privilege, output encoding, guardrails |
| Operation | Extraction, membership inference, cost exhaustion | Rate limiting, authentication, monitoring, budgets |
| Retirement | Orphaned endpoints, residual data | Decommissioning process, data deletion evidence |
Four references do most of the work. The OWASP Top 10 for LLM Applications 2025 gives developers a concrete risk list, from prompt injection at LLM01 to unbounded consumption at LLM10. MITRE ATLAS catalogues observed adversary techniques against AI systems in the style of ATT&CK. The NIST AI Risk Management Framework structures governance, and NIST AI 100-2 supplies the attack taxonomy. In the UK, the NCSC and CISA Guidelines for Secure AI System Development, published in November 2023 with eighteen national agencies, set expectations across design, development, deployment, and operation; the Department for Science, Innovation and Technology issued a voluntary code of practice for securing AI in January 2025, later published as ETSI TS 104 223.
The UK has chosen a sector-regulator approach rather than a single AI statute, so the binding obligations come from existing law. UK GDPR governs personal data in training sets, prompts, and outputs, with the ICO expecting a Data Protection Impact Assessment for most significant AI processing. The NIS Regulations apply to operators of essential services that embed AI in critical functions, and DORA brings AI-enabled ICT services into scope for financial entities. Organisations selling into the EU must read the EU AI Act as well, since it applies to systems placed on that market regardless of where the provider sits. At Privia Security, we assess AI platforms, models, and the applications built on them through our AI Penetration Test Services.
No. AI safety asks whether a model behaves acceptably and avoids harm in normal use, including bias, toxicity, and misuse of capabilities. AI security asks whether an adversary can make it behave in a way its owner did not intend, or can steal the data and models behind it. The two overlap in red teaming, where the same test can reveal both a safety and a security failure.
Everything on your side of the boundary: what data leaves your environment, which systems the service can reach, what permissions you grant, how output is used, and what you log. Most real incidents with hosted models come from integration decisions rather than from the model itself. Review the provider’s terms on training use, retention, and data residency before connecting anything sensitive.
Build an inventory of AI in use, write a short usage policy that names approved tools and prohibited data, and apply least privilege to any system a model can reach. Those three steps remove most of the avoidable exposure. Formal frameworks and adversarial testing are worthwhile next, once you know what you are actually running.
Usually not a separate one. Extend what you have: add AI systems to your asset inventory and risk register, cover model and data handling in your information security policy, and add approved tools and data classification rules to acceptable use. A standalone document that nobody cross-references tends to drift away from the controls that are actually enforced.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.