LLM Misinformation & Hallucinations
A confident wrong answer is its own security problem. Misinformation (OWASP LLM07:2026 — up from #9 in 2025) covers the false, fabricated or insecure content models produce — hallucinated facts, made-up citations, insecure code, invented package names — and the overreliance that lets it slip into decisions and shipped software unchecked.
What it is
Models generate plausible text, not verified truth. Most of the time plausible and true coincide; when they don’t, the output is delivered with the same fluent confidence. The danger multiplies when people treat it as authoritative — pasting AI-written code into production, citing invented sources, or acting on a fabricated fact — so misinformation is as much about human process as about the model.
The threats
Where it shows up in practice.
| Threat | How it works |
|---|---|
| Hallucinated facts | The model states something false as fact, fluently and without hedging. |
| Fabricated citations | Real-looking references, quotes or case numbers that don’t exist. |
| Insecure code suggestions | Working-looking code with vulnerabilities — SQLi, weak crypto, missing authz — copied straight into a codebase. |
| Hallucinated packages (“slopsquatting”) | The model invents a package name; an attacker registers it with malware so the suggested install pulls it in — ties to supply chain. |
| Overreliance / automation bias | Users trust output because it sounds authoritative and skip the check that would have caught it. |
How to defend
- Ground the model. Use retrieval over trusted sources so answers cite real, current material — see vector & embedding weaknesses.
- Keep a human in the loop for anything that matters. High-stakes outputs (legal, medical, financial, security) get expert review before they’re acted on.
- Security-review AI code. Treat model-written code like any untrusted contribution — scan, test and review it, don’t paste-and-ship.
- Verify package names exist before installing anything a model suggests; pin and check dependencies.
- Communicate uncertainty. Design the product to signal confidence limits and cite sources, so users don’t over-trust.
- Cross-check facts and citations against a primary source before relying on them.
Go deeper
- The full 2026 risk map: OWASP LLM Top 10.
- Grounding to reduce it: Vector & embedding weaknesses (LLM09).
- When hallucinated packages become malware: AI supply chain (LLM04).
Aligned to OWASP LLM07:2026 Misinformation — the OWASP GenAI LLM Top 10 2026 edition (released Aug 2026). A plain-language distillation of current practice. Something changed? Tell me.