Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
Data poisoning is the deliberate corruption of the data a model learns from, either to degrade its accuracy or to plant a hidden behaviour that activates on an attacker-chosen trigger.

What Is Data Poisoning?
Data poisoning attacks a model before it is ever deployed, by corrupting what it learns from. The attacker does not need access to your servers or your weights—only to a dataset you will train on, a feedback loop you trust, or a public corpus you scrape. Two goals are typical: degrade overall accuracy so the system becomes unreliable, or implant a backdoor that leaves normal behaviour intact and activates only when a chosen trigger appears.
The uncomfortable finding of recent years is that poisoning is cheap. Carlini and colleagues showed in 2023 that two practical attacks on web-scale datasets were within reach of an ordinary budget: buying expired domains that a dataset still references, and editing content at the moment a crawler fetches it. More striking, a study published in October 2025 by Anthropic with the UK AI Security Institute and The Alan Turing Institute found that a near-constant number of poisoned documents—on the order of 250—was enough to install a backdoor in models ranging from 600 million to 13 billion parameters. Backdoor strength did not scale down as models grew, which overturns the comforting assumption that a large corpus dilutes an attack.
A backdoor leaves aggregate metrics untouched. Validation accuracy, loss curves, and fairness dashboards all look normal, because the poisoned behaviour only appears for inputs containing the trigger, and the trigger is something the defender has not thought of. With retrieval-augmented systems the equivalent problem arrives at runtime rather than at training time: poison a document in the knowledge base and the model will repeat it confidently, which is the overlap between poisoning and RAG data handling. Fine-tuning does not escape the issue either, since a few hundred crafted examples in a fine-tuning set can both install behaviour and strip safety training.
Most effective controls are data governance rather than machine learning research.
Poisoning sits at LLM04 in the OWASP Top 10 for LLM Applications 2025 and has its own section in NIST AI 100-2. For UK organisations the relevant duties come from existing law: UK GDPR Article 5(1)(d) requires personal data to be accurate, and Article 32 requires integrity of processing, so a training dataset that anyone can modify is difficult to defend after an incident. NCSC and CISA guidance on secure AI development treats data provenance and integrity as a design requirement, not an optional extra. At Privia Security, we review data pipelines, registries, and training environments as part of our Infrastructure and Architecture Review Services.
Far fewer than most people assume. Research published in 2025 found that roughly 250 malicious documents were enough to install a backdoor across models of very different sizes, meaning the required proportion of the corpus falls as the model grows. For fine-tuning sets the numbers are smaller still, often a few dozen examples, because the data volume is much lower.
No. A well-constructed backdoor is designed to leave normal performance unchanged, so validation accuracy, loss, and standard benchmarks all look healthy. Detection relies on targeted probing with candidate triggers, comparing behaviour against a reference model, and inspecting activations, none of which is guaranteed. Provenance and integrity controls remain the stronger defence.
You inherit the provider’s training risk, which you cannot inspect, but your own exposure is mostly in what you add: fine-tuning data, retrieval corpora, and user feedback that influences behaviour. Those you control. Lock down who can write to a knowledge base, review fine-tuning datasets, and never retrain on unmoderated input.
No, they attack different stages. Poisoning corrupts training so the model learns something wrong; an adversarial example manipulates a single input at inference time against an unchanged model. Poisoning is persistent and affects everyone using the model, while an evasion attack must be crafted for each input the attacker wants misread.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.