LLM security requires pipeline-level controls, not model-layer patches. Learn how prompt injection, data exfiltration, and supply chain risks map to defense-in-depth strategies.
Your WAF won't catch it. Your SIEM won't flag it. A user submits a paragraph of natural language, and your model follows instructions it was never meant to follow.
LLM security breaks from traditional application security in a fundamental way. Standard software produces predictable, rule-based outputs. LLMs generate language that is dynamic, often surprising, and sometimes risky. That distinction matters because existing security tooling was built around deterministic systems. Patch cycles, signature-based detection, and input sanitization rules assume the application behaves the same way given the same input. LLMs don't. They operate as black boxes, making it difficult to understand how they generate certain outputs, which complicates both risk management and auditing.
This is why LLM security demands its own threat model, and why the only durable enforcement point is the pipeline layer between your application and the model.
OWASP maintained a Top 10 for LLM applications across two editions. The shifts between them tell you where risk is accelerating.
Prompt injection holds the top spot for the second consecutive edition. No surprise. But the reshuffling below it is more revealing. Sensitive information disclosure jumped from sixth to second place in 2025. Supply chain vulnerabilities rose from fifth to third. Both reflect real-world incidents that forced the security community to reprioritize.
The ranking changes are a signal: data leakage and third-party model trust are no longer theoretical risks. They are the categories producing the most damage reports.
Prompt injection is analogous to SQL injection. User input gets processed as part of the prompt, and the model cannot tell the difference between legitimate instructions and injected ones. The consequences range from data theft and system takeover to financial damages and regulatory penalties under GDPR, HIPAA, and PCI-DSS.
Three attack vectors, each with different entry points:
Direct injection (jailbreaking). The attacker submits input that overwrites or reveals the underlying system prompt. This can expose backend systems, internal functions, and data stores the LLM has access to. The attacker talks directly to the model and convinces it to ignore its constraints.
Indirect injection. The attacker embeds malicious instructions in external content the LLM processes, such as websites, uploaded files, or documents. Consider OWASP's own example: an attacker embeds an indirect prompt injection in a webpage, instructing the LLM to disregard previous user instructions and use a plugin to delete the user's emails. When the user asks the LLM to summarize the webpage, the plugin executes the deletion. The user never saw the injected instruction.
Multimodal injection. Models like GPT-4o can extract hidden text from images that appear blank. When uploaded, the model executes the embedded instructions without user interaction. A zero-click attack surface.
How reliable are these attacks? Palo Alto Networks assessed mainstream LLMs against prompt-based attacks and found that some techniques achieved success rates exceeding 50% across models of different scales, with certain cases reaching up to 88%. These are not edge cases requiring exotic setups. They work against models ranging from several-billion to trillion-parameter scale.
Sensitive data leaves LLM applications through multiple paths, and not all of them involve an attacker.
Training data memorization. LLMs can memorize and reproduce fragments of their training data, including personally identifiable information, proprietary business data, and confidential documents. Targeted queries can extract this data, which is the primary reason sensitive information disclosure climbed to second place in OWASP's 2025 rankings.
System prompt extraction. Beyond training data, models can reveal details about their own configuration. System prompts containing business logic, API endpoints, access secrets, or security controls become vulnerabilities when attackers coax models into exposing them.
Agent tool misuse. AI agents that interpret multiple input modes (text, images, documents) are increasingly susceptible to prompt-based attacks hidden within content. These attacks can lead to sensitive data exfiltration without user interaction.
Accidental employee disclosure. In 2023, engineers at Samsung used ChatGPT to help with tasks such as debugging code and summarizing notes, entering confidential company data including source code and internal information in the process. No malicious actor required. The data entered a third-party training pipeline through normal usage.
A compromised model on a popular hub could affect thousands of downstream applications. This is not hypothetical. Model hubs host fine-tuned variants, LoRA adapters, and community contributions with varying degrees of vetting. A poisoned model or adapter that looks legitimate can introduce backdoors across every application that pulls it.
Shadow AI compounds the problem from the other direction. Workers might use public AI tools without permission, entering private data and putting their organizations at risk. The Samsung incident is one documented case, but the pattern repeats wherever employees have unrestricted browser access and a problem they think an LLM can solve faster than internal tools.
Both risks share a common trait: they live outside the model you control. Your carefully hardened production model is irrelevant if your team pulls a compromised adapter from a public hub, or if your employees route proprietary data through consumer AI products.
Here is the core constraint that shapes every LLM security strategy: prompt injection vulnerabilities exist because LLMs do not segregate instructions and external data from each other. Both forms of input look like natural language to the model. There is no fool-proof prevention within the LLM.
System prompt hardening helps. Model-side guardrails help. Neither is sufficient. As OWASP puts it: you can't patch your way out of prompt injection because it exploits LLM design itself.
When an application passes LLM output directly to other components without validation, it trusts the model not to generate malicious content. If that output reaches a web browser, it might contain JavaScript that executes. If it reaches a database, it might contain SQL that runs. Prompt injection chains with improper output handling to produce cross-system compromise.
Defense has to happen in the pipeline, not in the model.
An AI gateway sits between your application and the LLM provider. It is the one enforcement point that applies consistently regardless of which model sits behind it.
The security controls a gateway layer can enforce span every category of risk discussed above:
| Threat | Pipeline control | What it does |
|---|---|---|
| Prompt injection | Input policy enforcement | Scans and filters requests before they reach the model |
| Data exfiltration | PII redaction | Strips sensitive data from prompts before they leave your infrastructure |
| Improper output handling | Output validation | Inspects model responses before they reach downstream systems |
| Privilege escalation | Least-privilege token scoping | Restricts what plugins and tools the model can invoke |
| Shadow AI | Egress filtering | Blocks unauthorized model endpoints at the network layer |
| Supply chain | Model provenance controls | Restricts which model versions your applications can call |
| Audit gaps | Request/response logging | Captures every interaction for compliance and forensics |
Rate limiting, moderation, and guardrails layer on top. No single control suffices. The point of centralizing them at the gateway is that every model call passes through the same enforcement stack, whether the request comes from your customer-facing chatbot, your internal summarizer, or your agentic workflow.
This is where LLM observability ties in. Monitoring inputs and outputs at the gateway gives you anomaly detection across your entire AI pipeline, not just within individual applications.
Eight steps, ordered by impact and implementation effort:
None of these controls require modifying the model. They operate on the pipeline around it, which is exactly where a purpose-built AI gateway enforces them.