What is data poisoning?
Data poisoning is an attack in which crafted or deliberately mislabeled records are slipped into the data a machine-learning model trains or retrains on, so the model ends up learning what the attacker wants it to learn. It attacks the model's integrity, but through its data, not through the model file or a single input. Any system that continues to learn from input arriving from outside is at risk.
Untargeted, targeted and backdoor poisoning
| Type | What happens to the model |
|---|---|
| Untargeted | Overall quality drops. A spam filter fed enough mislabeled samples simply gets worse, which works as a denial of service against detection. |
| Targeted | Quieter. The model stays right on nearly everything but goes wrong on one class of input the attacker picked. |
| Backdoor | The most dangerous form. Poisoned samples teach the model to respond to a trigger only the attacker knows, such as a rare word, a pixel pattern or a particular sender domain. |
Two cases show the range. In 2016 Microsoft took its Tay chatbot offline within a day, after users had taught it to post offensive messages. A 2025 study found that about 250 poisoned documents were enough to implant a simple backdoor in language models across a wide range of sizes.
Poisoning the retrieval source
Plenty of business chatbots answer questions with retrieval-augmented generation (RAG). For each question, the application looks up the most relevant passages in a knowledge base and adds them to the model's input. That means anyone with write access to an indexed source can alter the answers while the model itself stays untouched. Suppose an attacker adds a believable but false page to an internal wiki: the assistant may cite that page and pass the false information along. The dividing line with a related attack is what the planted text contains. False facts make it poisoning; instructions directed at the model make it indirect prompt injection.
How to defend against it
The controls center on provenance, meaning where data came from, and on change control.
- Track the origin of every training and retrieval source.
- Limit who may add data or change labels, and keep anonymous feedback from feeding directly into retraining.
- Keep versioned snapshots so a bad update can be rolled back.
- Run every retrained model against a fixed evaluation set before it takes over from the previous version.
- For retrieval sources, restrict publishing rights, require review of pages that describe security procedures, keep an edit history, and display the source behind each answer so users can weigh it.
Data poisoning vs. evasion and manipulation
Poisoning changes the data the model learns or looks up, before the real target arrives. Evasion leaves the model untouched and fools it with a crafted input at decision time. Model manipulation tampers with the weights or settings directly. Ask whether the data changed, the input was crafted, or the model itself was altered.
Data poisoning on the CompTIA Security+ exam
Data poisoning is named under objective 2.6. Wording about mislabeled training data, a model retrained on attacker-supplied feedback, or a planted record that teaches a hidden trigger points at it, and a false page planted in a chatbot's knowledge base is retrieval poisoning. The distractors are evasion, when one crafted input is misclassified, and prompt injection, when planted text gives the model instructions rather than false facts.