Explore our new Cybersecurity Terms Glossary, explained by our experts.

View the Glossary View the Glossary
PRIVIA

Cybersecurity Terms Glossary

What Is Data Poisoning?

Data poisoning is the deliberate corruption of the data a model learns from, either to degrade its accuracy or to plant a hidden behaviour that activates on an attacker-chosen trigger.

Illustration for the cybersecurity term Data Poisoning
Home / Blog / Cybersecurity Terms / What Is Data Poisoning?
17 November 2026

What Is Data Poisoning?

What Is Data Poisoning?

Data poisoning attacks a model before it is ever deployed, by corrupting what it learns from. The attacker does not need access to your servers or your weights—only to a dataset you will train on, a feedback loop you trust, or a public corpus you scrape. Two goals are typical: degrade overall accuracy so the system becomes unreliable, or implant a backdoor that leaves normal behaviour intact and activates only when a chosen trigger appears.

How Little It Takes

The uncomfortable finding of recent years is that poisoning is cheap. Carlini and colleagues showed in 2023 that two practical attacks on web-scale datasets were within reach of an ordinary budget: buying expired domains that a dataset still references, and editing content at the moment a crawler fetches it. More striking, a study published in October 2025 by Anthropic with the UK AI Security Institute and The Alan Turing Institute found that a near-constant number of poisoned documents—on the order of 250—was enough to install a backdoor in models ranging from 600 million to 13 billion parameters. Backdoor strength did not scale down as models grew, which overturns the comforting assumption that a large corpus dilutes an attack.

Attack Types

Why Detection Is Hard

A backdoor leaves aggregate metrics untouched. Validation accuracy, loss curves, and fairness dashboards all look normal, because the poisoned behaviour only appears for inputs containing the trigger, and the trigger is something the defender has not thought of. With retrieval-augmented systems the equivalent problem arrives at runtime rather than at training time: poison a document in the knowledge base and the model will repeat it confidently, which is the overlap between poisoning and RAG data handling. Fine-tuning does not escape the issue either, since a few hundred crafted examples in a fine-tuning set can both install behaviour and strip safety training.

Defending the Pipeline

Most effective controls are data governance rather than machine learning research.

Regulatory Framing

Poisoning sits at LLM04 in the OWASP Top 10 for LLM Applications 2025 and has its own section in NIST AI 100-2. For UK organisations the relevant duties come from existing law: UK GDPR Article 5(1)(d) requires personal data to be accurate, and Article 32 requires integrity of processing, so a training dataset that anyone can modify is difficult to defend after an incident. NCSC and CISA guidance on secure AI development treats data provenance and integrity as a design requirement, not an optional extra. At Privia Security, we review data pipelines, registries, and training environments as part of our Infrastructure and Architecture Review Services.

FAQ – Frequently Asked Questions

How Many Poisoned Samples Does an Attacker Need?

Far fewer than most people assume. Research published in 2025 found that roughly 250 malicious documents were enough to install a backdoor across models of very different sizes, meaning the required proportion of the corpus falls as the model grows. For fine-tuning sets the numbers are smaller still, often a few dozen examples, because the data volume is much lower.

Can We Detect a Backdoor by Testing Accuracy?

No. A well-constructed backdoor is designed to leave normal performance unchanged, so validation accuracy, loss, and standard benchmarks all look healthy. Detection relies on targeted probing with candidate triggers, comparing behaviour against a reference model, and inspecting activations, none of which is guaranteed. Provenance and integrity controls remain the stronger defence.

Does Poisoning Affect Us If We Only Use Hosted Models?

You inherit the provider’s training risk, which you cannot inspect, but your own exposure is mostly in what you add: fine-tuning data, retrieval corpora, and user feedback that influences behaviour. Those you control. Lock down who can write to a knowledge base, review fine-tuning datasets, and never retrain on unmoderated input.

Is Poisoning the Same as an Adversarial Example?

No, they attack different stages. Poisoning corrupts training so the model learns something wrong; an adversarial example manipulates a single input at inference time against an unchanged model. Poisoning is persistent and affects everyone using the model, while an evasion attack must be crafted for each input the attacker wants misread.

Related Terms

You May Be Interested In These

What Is Broken Access Control?

Cybersecurity Terms

17 November 2026

What Is Broken Access Control?

Read More
What Are the CRIME and BREACH Attacks?

Cybersecurity Terms

17 November 2026

What Are the CRIME and BREACH Attacks?

Read More
What Is REST API Security?

Cybersecurity Terms

17 November 2026

What Is REST API Security?

Read More