LLM Security: Could an AI Attack Trick Your App, or You?
This page teaches you to defend against AI attacks — whether you build with LLMs or you just answer the phone. Pick where you’re starting, learn the risk from a cheat sheet, then practise on a target you’re allowed to break. Everything here is free, and nothing is behind a sign-up.
Where do I start?
Pick the one that sounds most like you. Each opens a short path.
I build or test AI apps
- Read the OWASP LLM Top 10 overview (10 minutes) so you know the map.
- Learn the big one: Prompt Injection, then Jailbreak techniques.
- Do it for real in the LLMVault lab, then level up in the AI Red Teaming Playground lab.
- If your app uses tools or agents: AI Agent Security, then the FinBot CTF.
I use AI tools at work
- Know what not to paste in: Sensitive Information Disclosure.
- Know when not to trust the answer: LLM Misinformation.
- Know the scams aimed at your job — deepfake video calls and “urgent transfer” requests: AI scams: how to spot and stop them.
I’m protecting my family
- Read AI scams: how to spot and stop them (6 minutes).
- Tonight: agree a family code word, and agree that nobody ever asks for money by gift card, wire or crypto.
- Share the FTC’s plain-English page on family emergency scams with older relatives.
Track 1 — builders & defenders: the OWASP LLM Top 10
Each risk links to a cheat sheet you can work from: how the attack lands, what to test, and how to defend it.
| Risk | In one line | Cheat sheet |
|---|---|---|
| Overview | The whole Top 10 on one page, as a quick reference. | OWASP LLM Top 10 — overview |
| LLM01 — Prompt Injection | Untrusted input steering the model's instructions, directly or through content it reads. | Prompt injection testing |
| LLM01 — Jailbreak techniques | The specific patterns people use to bypass a model's guardrails — the sharp end of injection. | Jailbreak techniques |
| LLM02 — Sensitive Information Disclosure | The model leaking secrets, PII, or training data it should never surface. | Sensitive information disclosure |
| LLM03 — Excessive Agency (AI agents) | Agents with more tools, permissions, or autonomy than the task needs. | AI agent security |
| LLM04 — Supply Chain | Compromised models, datasets, adapters, or dependencies pulled into your stack. | AI supply chain |
| LLM05 — Data & Model Poisoning | Tainted training or fine-tuning data that plants behaviour you didn't ask for. | Data & model poisoning |
| LLM06 — Unbounded Consumption | Cost and denial-of-service through unchecked tokens, requests, or recursion. | Unbounded consumption |
| LLM07 — Misinformation | Confident, wrong output that gets trusted and acted on. | LLM misinformation |
| LLM08 — Hidden Context Exposure (system prompt leakage) | Getting the model to reveal its hidden instructions and the secrets baked into them. | System prompt leakage |
| LLM09 — Vector & Embedding Weaknesses | Attacks on the RAG layer: poisoned documents, embedding inversion, retrieval abuse. | RAG security |
| LLM10 — Improper Output Handling | Trusting model output downstream — into a shell, a browser, a DB — without sanitising it. | Improper output handling |
Practise it: StashGrid labs
- LLMVault lab — work the OWASP LLM Top 10 against a deliberately vulnerable app, on your own machine.
- AI Red Teaming Playground lab — Microsoft’s own training challenges, run locally, at rising difficulty. Ends with a defence sheet.
- OWASP FinBot CTF — a vulnerable financial AI agent: prompt injection, insecure tool use and more.
Track 2 — everyone: AI scams
Cloned voices, deepfake video calls and flawless AI-written phishing make old scams convincing. You don’t beat them by spotting the fake; you beat them with habits that make the fake irrelevant: a family code word, calling back on a number you already have, and never moving money on one channel. The full guide: AI scams: how to spot and stop them.
Free training we recommend
Outside courses and ranges worth your time, checked 30 Sep 2026. Free first; anything with a catch says so.
| Training | Best for | Why we picked it |
|---|---|---|
| Microsoft AI Red Teaming 101 | Builders & defenders | Free 10-episode series from Microsoft’s AI Red Team: direct and indirect prompt injection, multi-turn attacks, and the “spotlighting” defence. Pairs with our Playground lab. |
| PortSwigger: Web LLM attacks | Web testers | Free browser labs from the Burp Suite team on LLM APIs, insecure output handling and excessive agency. Maps straight onto our cheat sheets. |
| DeepLearning.AI × Giskard: Red Teaming LLM Applications | Beginners with a little Python | About 1h40 on finding and testing LLM app weaknesses, manually and with an open-source scanner. Free for now, during their platform beta. |
| TryHackMe: AI Security path | Beginners who like guided rooms | Five modules and 25 labs (~19 hours) covering injection, jailbreaks, supply chain and poisoning. Free sign-up; some rooms may need a subscription. |
| Wiz: Prompt Airlines | Anyone curious | A free browser challenge: talk an airline’s support bot into a free (fictional) ticket. The fastest way to feel how prompt injection works. |
| OWASP GenAI Security Project | Defenders & team leads | The source behind the 2026 LLM Top 10, plus free guides on agentic AI security. Where our cheat sheets start. |
| FTC: Family emergency scams | Everyone | Plain-English advice on voice-clone “emergency” calls. The page to send a parent or grandparent. |
Foundations & reference
- OWASP AI Exchange — the community reference for AI security threats, controls and frameworks.
- CAI — agent-based cybersecurity AI framework — a sandbox for prototyping and testing AI-driven security tooling.
What this covers, and what it doesn’t
Covers: how LLM applications get attacked and how to defend them; practice on deliberately vulnerable targets you run yourself or that invite you in; everyday habits that stop AI-powered scams.
Doesn’t cover: attacking systems you don’t own or aren’t invited to test, generating harmful content, or vendor product reviews. Where an outside lab includes harmful-content challenges, we tell you to skip them.
Tell us what we’re missing
Steven brings the ideas, Nova builds the pages and labs, and Nyx checks them — but none of us knows what we don’t know. If something here is wrong, out of date, or there’s training we should add, tell us. Suggestions go on Steven’s idea pile; no promises on timing.
FAQ
Do I need to know how to code?
Not for the cheat sheets, the scams guide or Prompt Airlines. The labs need you to be comfortable pasting commands into a terminal and running Docker; each step shows exactly what to type.
Is it legal to practise these attacks?
Yes, on targets built for it: labs you run on your own machine, and CTFs and ranges that invite you in. Trying the same thing on a real company’s chatbot without permission is a different matter. Don’t.
Which OWASP LLM Top 10 version do you use?
The 2026 edition. Some outside labs still use an older edition’s numbering, so a risk’s number can differ between a lab and our cheat sheets; the names still line up.
What does it cost?
Everything on this page is free to read and use. Labs that call a hosted AI model, like the Playground lab, use your own API key, which costs a little per challenge — set a spending limit first.
Can I tell if a voice on the phone is cloned?
Sometimes, but don’t count on it. Verification beats detection: a family code word and a call back on a number you already have work even when the clone is perfect.
How often is this page updated?
When the OWASP list changes, when an outside course changes its terms, or when we publish a new lab. The date in the training table is when every outside link was last checked.
Steven’s honest footer: some of these I’ve tested end to end, others are notes I wrote while learning the risk. Take what’s useful, and if something here is out of date, tell me.