Explore our new Cybersecurity Terms Glossary, explained by our experts.

View the Glossary View the Glossary
PRIVIA

Cybersecurity Terms Glossary

What Is AI Security?

AI security is the discipline of protecting AI systems across their lifecycle, from training data and model weights to prompts, tools, and the decisions the model is trusted to make.

Illustration for the cybersecurity term AI Security
Home / Blog / Cybersecurity Terms / What Is AI Security?
17 November 2026

What Is AI Security?

What Is AI Security?

AI security is the protection of artificial intelligence systems and the organisations that depend on them. It covers the data a model learns from, the weights that result, the infrastructure that serves them, the prompts and documents flowing through at runtime, and the actions the system is permitted to take. It is distinct from AI safety, which asks whether a model behaves acceptably, and from the use of AI in security products, which is a different conversation altogether.

Why Existing Controls Do Not Cover It

Conventional application security assumes that behaviour is defined by code a developer wrote and a reviewer read. A model inverts that: behaviour emerges from data, and the instructions it follows arrive at runtime mixed into the same channel as untrusted content. Three consequences follow. Input validation cannot reliably separate commands from data. Patching does not apply in the usual sense, because fixing a behaviour means retraining, fine-tuning, or adding a filter, each with its own regression risk. And the supply chain extends into datasets and pre-trained weights that almost nobody inspects line by line.

The Lifecycle View

StageRepresentative threatsCore controls
Data collectionPoisoning, unlawful processing, unlicensed contentProvenance records, integrity checks, lawful basis
TrainingBackdoors, credential theft from training infrastructureIsolated environments, signed artefacts, access control
Model storageWeight theft, tampered checkpointsEncryption, signing, safetensors over pickle formats
DeploymentPrompt injection, jailbreaks, unsafe output handlingLeast privilege, output encoding, guardrails
OperationExtraction, membership inference, cost exhaustionRate limiting, authentication, monitoring, budgets
RetirementOrphaned endpoints, residual dataDecommissioning process, data deletion evidence

Frameworks Worth Using

Four references do most of the work. The OWASP Top 10 for LLM Applications 2025 gives developers a concrete risk list, from prompt injection at LLM01 to unbounded consumption at LLM10. MITRE ATLAS catalogues observed adversary techniques against AI systems in the style of ATT&CK. The NIST AI Risk Management Framework structures governance, and NIST AI 100-2 supplies the attack taxonomy. In the UK, the NCSC and CISA Guidelines for Secure AI System Development, published in November 2023 with eighteen national agencies, set expectations across design, development, deployment, and operation; the Department for Science, Innovation and Technology issued a voluntary code of practice for securing AI in January 2025, later published as ETSI TS 104 223.

What to Do First

The Regulatory Picture for UK Organisations

The UK has chosen a sector-regulator approach rather than a single AI statute, so the binding obligations come from existing law. UK GDPR governs personal data in training sets, prompts, and outputs, with the ICO expecting a Data Protection Impact Assessment for most significant AI processing. The NIS Regulations apply to operators of essential services that embed AI in critical functions, and DORA brings AI-enabled ICT services into scope for financial entities. Organisations selling into the EU must read the EU AI Act as well, since it applies to systems placed on that market regardless of where the provider sits. At Privia Security, we assess AI platforms, models, and the applications built on them through our AI Penetration Test Services.

FAQ – Frequently Asked Questions

Is AI Security the Same as AI Safety?

No. AI safety asks whether a model behaves acceptably and avoids harm in normal use, including bias, toxicity, and misuse of capabilities. AI security asks whether an adversary can make it behave in a way its owner did not intend, or can steal the data and models behind it. The two overlap in red teaming, where the same test can reveal both a safety and a security failure.

We Only Use Hosted AI Services. What Is Our Responsibility?

Everything on your side of the boundary: what data leaves your environment, which systems the service can reach, what permissions you grant, how output is used, and what you log. Most real incidents with hosted models come from integration decisions rather than from the model itself. Review the provider’s terms on training use, retention, and data residency before connecting anything sensitive.

Where Should a Small Team Start?

Build an inventory of AI in use, write a short usage policy that names approved tools and prohibited data, and apply least privilege to any system a model can reach. Those three steps remove most of the avoidable exposure. Formal frameworks and adversarial testing are worthwhile next, once you know what you are actually running.

Do We Need a Separate AI Security Policy?

Usually not a separate one. Extend what you have: add AI systems to your asset inventory and risk register, cover model and data handling in your information security policy, and add approved tools and data classification rules to acceptable use. A standalone document that nobody cross-references tends to drift away from the controls that are actually enforced.

Related Terms

You May Be Interested In These

How Do You Detect Encrypted Containers?

Cybersecurity Terms

17 November 2026

How Do You Detect Encrypted Containers?

Read More
What Is Privileged Access Management (PAM)?

Cybersecurity Terms

17 November 2026

What Is Privileged Access Management (PAM)?

Read More
What Is the Pyramid of Pain?

Cybersecurity Terms

17 November 2026

What Is the Pyramid of Pain?

Read More