Articles about AI Security — exploring patterns, best practices, and real-world implementations in production systems.
16 posts tagged with ai security. ← All posts
A single pre-launch red-team decays the moment your model, prompt, or tools change — turning adversarial testing into a sustained program is what keeps an AI system safe past day one.
The capstone: making red-teaming a sustained program — the remediation loop where findings become regression tests, blending automated/manual/external modes, rules of engagement, ship-blocking severity thresholds, governance evidence, and culture.
The series finale: how to proactively find AI security failures before attackers do — turning injection, leakage, and excessive-agency risks into a repeatable adversarial test suite that runs in CI, measuring attack success honestly, and standing up incident response for the day a control fails.
Find AI security failures before attackers do: adversarial testing of the whole system, a test taxonomy mapped to the series, automated tooling (PyRIT, garak), a CI security-regression suite that fails the build, honest attack-success-rate measurement, and AI incident response.
Turning red-team attacks into metrics you can act on and track over time — attack success rate, coverage, severity, and trend — plus the honest limits of what any of those numbers can tell you.
Turning attacks into metrics: attack success rate and why it's subtle, scoring success (rule/classifier/LLM-judge with its biases), coverage across the taxonomy, severity weighting, tracking trends per model/prompt version, and honest reporting of residual risk.
Part seven of the AI Security Engineering series: DevSecOps for AI systems — securing the secrets, network, supply chain, prompts, and CI/CD gates that surround the model, so a hardened model doesn't sit inside a soft pipeline.
DevSecOps for AI: secrets in a manager not code, least-privilege runtime identities (no ambient prod creds for agents), egress control, rate/spend limits against model DoS, supply-chain verification in CI, prompts-as-code, and a security-eval gate that fails the build.
Scaling red-teaming beyond manual probing — the building blocks of an automated harness (seed library, mutation, orchestrator, scorer), LLM-driven adaptive attackers, the real tools by role (PyRIT, garak, promptfoo, Giskard), and wiring it all into CI as a repeatable gate.
Scaling red-teaming: the harness building blocks (attack seeds, mutation, orchestrator, scorer), adaptive LLM-driven attackers, the real tools by role (PyRIT, garak, promptfoo, Giskard), and integrating an automated red-team gate into CI.
The defensive layer that screens what goes into a model and what comes out — input rails, output rails, topical rails, and groundedness checks — plus the real tooling ecosystem and a vendor-neutral Python pipeline that wraps a model call and knows how to refuse.
The defensive layer that screens inputs and outputs: input/output/topical/groundedness rails, the real ecosystem (Llama Guard, Granite Guardian, NeMo Guardrails, Guardrails AI, Presidio, hosted moderation), and a vendor-neutral guardrail pipeline — with honest false-positive/negative trade-offs.
Why agents and retrieval turn a prompt injection into real-world action, how to red-team the highest-risk AI surface with benign canaries, and the least-privilege controls that shrink an attacker's blast radius.
The highest-risk modern surface: indirect injection via RAG/tools, tool abuse and excessive-agency exploitation, memory poisoning, multi-step attacks, and data-exfiltration channels — with a canary methodology and least privilege as the primary control.
Part five of the AI Security Engineering series: the two OWASP LLM risks that live in the plumbing around the model — the documents your agent retrieves and the models, datasets, and dependencies it is built from — and the Python patterns that treat both as untrusted until proven otherwise.
Securing the components around the model: RAG as an injection and poisoning vector (treat retrieved content as hostile, enforce provenance and per-user authz) and supply-chain risk (safetensors over pickle, model provenance, pinned and vetted dependencies and tools).
A defender's tour of the attacks that target the model and its data — prompt and context extraction, training-data memorization, membership inference and model inversion, model stealing, poisoning and backdoors, and evasion — with what a red-teamer tests and what actually stops each one.
Attacks on the model and its data: system-prompt and training-data extraction, membership inference and model inversion, model stealing, and poisoning/backdoors — what to test and defend, and the honest risk difference between using hosted models and training your own.
Two tightly-linked OWASP LLM risks that turn a clever prompt injection into real-world damage — and the Python patterns that shrink the blast radius: treat model output as untrusted input, and give agents the least agency they can get away with.
Two OWASP risks that turn an injection into damage: insecure output handling (model output is untrusted input — never eval/shell/SQL it unescaped) and excessive agency (least-privilege tools, allow-lists, human approval for irreversible actions, audit logs).
A defender's field guide to the injection and jailbreak techniques a red-teamer probes for — the taxonomy, why each one works, and how to turn it into a re-runnable test suite that maps every passed test to a concrete fix.
A deeper, test-focused look at injection and jailbreak families — direct vs indirect (the RAG/agent threat), role-play, obfuscation, many-shot, multi-turn, cross-lingual — plus a red-team methodology: build a probe suite, mutate, test input and retrieval paths, measure, re-test.
Part three of the AI Security Engineering series: protecting the data that flows through an LLM system — how sensitive information leaks out of prompts, logs, and retrieval, and the engineering controls (redaction, data minimization, per-user retrieval authz, residency choices) that actually stop it.
Protecting data in LLM systems: sensitive-information disclosure, PII in prompts and logs (your observability can be the leak), redaction with Presidio, data residency, and per-user access control on the retrieval layer so RAG doesn't leak across tenants.
Before you attack an AI system you need a map of it: the components an adversary can influence, the trust boundaries between them, and a taxonomy that sorts attacks by goal and stage so your red-teaming is systematic instead of a grab-bag of the attacks that happen to trend that week.
Mapping the AI attack surface so red-teaming is systematic: threat-modeling the components (model, prompts, training/RAG data, tools, guardrails) and an attack taxonomy by goal (integrity/availability/privacy/abuse) and stage (training vs inference), aligned to NIST AI 100-2 and ATLAS.
Why the #1 risk on the OWASP LLM Top 10 has no clean fix — the model can't tell your instructions apart from the text it reads — and the defense-in-depth pattern that actually shrinks the blast radius.
The #1 LLM risk: direct vs indirect prompt injection (the latter is the real threat for RAG and agents), jailbreak families, and honestly-rated defense-in-depth — delimiting untrusted data, least-privilege tools, human-in-the-loop, and detection's limits.
The opening post of a hands-on series for builders: what it means to systematically stress-test an AI system — model, prompts, retrieval, tools, and guardrails — to surface its failures before adversaries or ordinary users do, how that differs from pentesting and robustness testing, and the frameworks and ethics that keep the work rigorous.
The opener to a defensive AI red-teaming series: stress-testing AI systems (models + prompts + RAG + tools + guardrails) to find failures before adversaries do — how it differs from pentesting, the frameworks (OWASP GenAI, MITRE ATLAS, NIST AI 100-2), and ethical rules of engagement.
Why LLM and agent applications open a genuinely new attack surface, the mental models to reason about it (OWASP Top 10 for LLM Applications, MITRE ATLAS, NIST AI RMF), and how to threat-model an AI system before you write a line of defensive code.
The opener to a series on securing LLM and agent systems: why AI apps have a new attack surface (instructions and data share one channel, tools grant real power), the OWASP Top 10 for LLM Applications, MITRE ATLAS, and how to threat-model an AI system.
All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse all posts or explore related topics using the tag cloud above.