An LLM can say anything, which is exactly the problem. The same open-endedness that makes it useful means it can hallucinate, leak data, follow a malicious instruction hidden in its input, or produce something harmful — and in production, in front of real users, those aren't edge cases, they're inevitabilities. Guardrails are the controls that sit around the model to contain what it might do. This post covers the new failure modes LLMs introduce and the layered defenses that keep a production system safe.
An LLM can say anything, which is the problem. The new failure modes — hallucination, prompt injection, data leakage, harmful output — and layered, fail-closed guardrails (input, output, action) with humans in the loop, treated as part of the lifecycle and red-teamed.
A defense you haven't tested is a hope, not a control. The final discipline of LLM security is treating your guardrails as a system to be measured, attacked, and monitored continuously — because the threat evolves, your application changes, and a defense that worked last quarter can silently rot. This closing post covers red-teaming your own system, operating it in production, and the honest state of the art.
A defense you haven't tested is a hope, not a control. The final discipline of LLM security is treating your guardrails as a system to be measured, attacked, and monitored continuously — because the threat evolves, your application changes, and a defense that worked last quarter can silently rot. Red-teaming, production monitoring, and the honest state of the art.
"Guardrails" is the umbrella term for the runtime checks that sit around a model and screen what goes in and comes out — content classifiers, moderation models, topic and format validators, PII detectors. They're a real and useful layer, distinct from the architectural defenses, and they come with their own design rules: layer them, fail safe, and never mistake them for a wall.
"Guardrails" is the umbrella term for the runtime checks around a model — content classifiers, moderation models, topic and format validators, PII detectors. They're a real and useful layer, distinct from architectural defenses, with their own design rules: layer them, fail safe, and never mistake them for a wall. This post covers building the layered defense in practice.
Injection defense usually focuses on what goes into the model. But an equally dangerous class of bug lives on the way out: whatever the model produces gets passed to another system — a browser, a shell, a database, another service — that trusts it. If the model can be made to emit a malicious payload, and your code renders or executes it, the injection escapes the model and lands in your infrastructure.
Injection defense usually focuses on what goes into the model, but an equally dangerous class of bug lives on the way out: whatever the model produces gets passed to another system that trusts it. If the model can be made to emit a malicious payload and your code renders or executes it, the injection escapes the model and lands in your infrastructure — XSS, SQLi, SSRF, exfiltration.
Everything before this post raised the probability barrier against injection. This post lowers the impact — and impact is what actually protects you. The load-bearing defense against prompt injection isn't a prompt or a filter; it's an architecture where a fully-hijacked model still can't do anything catastrophic, because it was never granted the power to.
Everything before this raised the probability barrier against injection; this post lowers the impact — and impact is what actually protects you. The load-bearing defense isn't a prompt or a filter but an architecture where a fully-hijacked model still can't do anything catastrophic: least privilege, the confused-deputy trap, human-in-the-loop, and the dual-LLM pattern.
Between filtering the input and re-architecting the system sits a tempting middle ground: make the prompt itself more resistant. Delimiters, spotlighting, instruction placement, and defensive system prompts all raise the cost of an attack. None of them close the hole — because they are all still text in the one channel the attacker also writes to — but used well they meaningfully shift the odds.
Between filtering input and re-architecting the system sits a tempting middle ground: make the prompt itself more resistant. Delimiters, spotlighting, instruction placement, and defensive system prompts all raise the cost of an attack — but none close the hole, because they're all still text in the one channel the attacker also writes to. The value is knowing exactly how much they buy.
The first instinct when facing prompt injection is to inspect the input and block the bad stuff. It's a reasonable layer — but a treacherous one, because it creates a feeling of safety far larger than the protection it provides. This post covers the input-side defenses that are genuinely worth having, and draws a hard line around what they can and cannot do.
The first instinct against injection is to inspect the input and block the bad stuff. It's a reasonable layer but a treacherous one — it creates a feeling of safety far larger than the protection it provides. This post covers the input defenses genuinely worth having, and draws a hard line around what they can't do (starting with: indirect injection bypasses them entirely).
You can't defend against what you can't categorize. Prompt-based attacks come in a few structurally distinct shapes — direct injection, indirect injection through content the model reads, and jailbreaks that target the model's safety training — and each demands a different defense. This post maps the attack surface so the rest of the series can defend it systematically.
You can't defend what you can't categorize. Prompt-based attacks come in structurally distinct shapes — direct injection, indirect injection through content the model reads, and jailbreaks targeting the model's safety training — and each demands a different defense. This post maps the attack surface so the rest of the series can defend it systematically.
Prompt injection is the defining security problem of LLM applications, and — unlike SQL injection, which it superficially resembles — it has no clean fix. The reason is structural: a language model reads instructions and data through the same channel, and cannot reliably tell which is which. This opening post explains why that makes injection fundamentally hard, and reframes the goal from "prevent it" to "contain the blast radius."
Prompt injection is the defining security problem of LLM applications, and unlike SQL injection it has no clean fix — because a model reads instructions and data through the same channel and can't reliably tell them apart. This opening post explains why that makes injection fundamentally hard, and reframes the goal from prevent to contain the blast radius.
Offline evaluation tells you whether a change is promising; production tells you whether it actually works. Once your system is serving real users, evaluation becomes continuous: online experiments, guardrail metrics, drift monitoring, and gating deploys on eval scores. This closing post moves evaluation from the lab into the running system and ties the whole series into a working loop.
Offline evaluation tells you whether a change is promising; production tells you whether it works. Once you're serving real users, evaluation becomes continuous — online experiments, guardrail metrics, drift monitoring, CI gating. This closing post moves evaluation from the lab into the running system and ties the series into one loop.
A system can be reliable, safe, and responsible and still fail — by being uneconomical — and the final move is turning one production system into a repeatable capability many teams can build safely.
A system can be reliable, safe, and responsible and still fail by being uneconomical. Phases 10-11: known unit economics and spend guardrails, then a paved road that lets many teams ship governed AI safely.
A system that changes itself can improve itself right off a cliff, so the evaluation and guardrails are not an afterthought to self-evolving agents — they are the thing that makes them safe to run at all.
A system that changes itself can improve right off a cliff. Measuring evolution honestly, reward hacking, drift and collapse, and the guardrails that keep it safe.
A model that scores well in evaluation still has to serve real traffic within a latency budget, isolate tenants, plan for capacity, and enforce safety in the request path — and the guardrails have to be inline, not a filter someone can route around.
A model that scores well still has to serve traffic within a latency budget, isolate tenants, plan capacity, and enforce safety in the request path — inline, not as a filter someone can route around. Phase 5.
Beyond the interactive terminal, Claude Code can run headless in scripts and CI — which unlocks automation, and raises the stakes on permissions, review, and trust.
The capstone: running Claude Code headless in scripts and CI (PR review, batch ops, scheduled jobs) — and the guardrails it demands: least privilege, sandboxing, gating the produced artifact with human review, and the series' layered recap.
Why your Terraform is a security control point, how misconfiguration scanners catch public buckets and open ingress before apply, and how to encode org guardrails as executable policy with OPA/Rego, Conftest, and Sentinel instead of a wiki page nobody reads.
Securing infrastructure definitions and enforcing guardrails automatically: IaC scanning (Checkov/tfsec/Trivy) for misconfig, drift detection, and policy-as-code with OPA/Rego + Conftest so org rules block bad infra before apply — not in a wiki.
The defensive layer that screens what goes into a model and what comes out — input rails, output rails, topical rails, and groundedness checks — plus the real tooling ecosystem and a vendor-neutral Python pipeline that wraps a model call and knows how to refuse.
The defensive layer that screens inputs and outputs: input/output/topical/groundedness rails, the real ecosystem (Llama Guard, Granite Guardian, NeMo Guardrails, Guardrails AI, Presidio, hosted moderation), and a vendor-neutral guardrail pipeline — with honest false-positive/negative trade-offs.
Adding safety to a watsonx app in Python with two complementary layers — the built-in HAP and PII moderations that watsonx.ai applies to generation, and IBM's dedicated Granite Guardian risk-detection models run as classifiers around your main model to catch harm, jailbreaks, and RAG hallucination before a response reaches a user.
Add safety to a watsonx app in Python with two layers: built-in HAP/PII moderations on generation, and Granite Guardian — IBM's dedicated risk models — run as a classifier to screen input and check RAG groundedness, wrapped in a guarded-generate flow.
Adding a safety layer to an NVIDIA-stack Python app with NeMo Guardrails — and why running it in-process, with no HTTP boundary, is the quiet advantage Python gives you over a separate guardrails server.
Add safety with NeMo Guardrails, in-process in Python: the rail types (input/output/dialog/retrieval), the config.yml + Colang model, loading with RailsConfig/LLMRails backed by a NIM model, and detecting when a rail trips instead of treating a refusal as a normal answer.
How to put Amazon Bedrock Guardrails in front of a model from Go — attaching one to a Converse call, screening raw text with ApplyGuardrail, and reading whether the guardrail actually intervened.
Guardrails for Amazon Bedrock in Go: content filters, denied topics, PII/sensitive-information filters, and contextual grounding — attaching a guardrail to a Converse call and screening arbitrary text with ApplyGuardrail, checking for intervention.
Before you attack an AI system you need a map of it: the components an adversary can influence, the trust boundaries between them, and a taxonomy that sorts attacks by goal and stage so your red-teaming is systematic instead of a grab-bag of the attacks that happen to trend that week.
Mapping the AI attack surface so red-teaming is systematic: threat-modeling the components (model, prompts, training/RAG data, tools, guardrails) and an attack taxonomy by goal (integrity/availability/privacy/abuse) and stage (training vs inference), aligned to NIST AI 100-2 and ATLAS.
The opening post of a hands-on series for builders: what it means to systematically stress-test an AI system — model, prompts, retrieval, tools, and guardrails — to surface its failures before adversaries or ordinary users do, how that differs from pentesting and robustness testing, and the frameworks and ethics that keep the work rigorous.
The opener to a defensive AI red-teaming series: stress-testing AI systems (models + prompts + RAG + tools + guardrails) to find failures before adversaries do — how it differs from pentesting, the frameworks (OWASP GenAI, MITRE ATLAS, NIST AI 100-2), and ethical rules of engagement.
A Python engineer's map of NVIDIA's AI platform — NIM inference, NeMo Retriever, NeMo Guardrails, and Triton with TensorRT-LLM — and why the same code runs against the hosted API Catalog or your own self-hosted microservice.
The opener to a series on building LLM and agent applications on NVIDIA's AI stack from Python: how NIM, the API Catalog, NeMo Retriever, NeMo Guardrails, Triton and TensorRT-LLM fit together — and why the openai client and langchain-nvidia-ai-endpoints make it Python-native.
Predicting who will leave is the easy half. The hard half is acting on it: treating the persuadable, respecting a budget, and proving the intervention actually kept anyone.
Predicting attrition and acting on it: churn features and labels, uplift vs propensity (treat the persuadable), and an agentic retention workflow that recommends and executes interventions with guardrails and holdout mea…
Prompt injection can hijack what a model says, but not what it's allowed to do — as long as policy lives in a middleware pipeline the model never sees.
A system prompt saying never delete records is a suggestion the model can be talked out of. A gateway that returns FORBID for the delete tool cannot. Put enforcement where the model can't reach it and prompt injection stops mattering for access control.
Large language models earn their place in regulated finance as fast, well-supervised assistants — not as autonomous agents with a hand on the money.
Where LLMs fit in regulated finance: document analysis/extraction, retrieval-grounded Q&A, and agentic decision support — plus the guardrails (grounding, human-in-the-loop, output validation, audit trails) that keep them…
An agent that burns its turn budget without finishing is neither a success nor an error. Give it a distinct outcome and a typed event stream.
The scariest agent failure is the runaway loop that never finishes. Cap every run at max_turns and make EXHAUSTED a first-class outcome, neither a failure nor a silent success, with a typed event stream you can route on.
If your gateway will block a tool call, don't make the agent discover that by trying. Hand it a capabilities brief and stop paying for turns it can't complete.
If your gateway will block a tool call, don't make the agent find out by trying, because each blocked attempt is a wasted turn. A capabilities brief aligns what the model thinks it can do with what enforcement actually allows.
An autonomous agent injects its own plan-and-execute tools at runtime. If your gateway is fail-closed, you have to find and allowlist them — deliberately.
An autonomous harness injects its own tools at runtime, tools that don't exist at build time. Fail-closed governance is only complete when it turns that blind spot into a visible block, then allowlists the known-safe internals deliberately.
The single most important pattern in LangGraph — a branch plus a back-edge, and the `recursion_limit` that keeps it from running forever.
Branching plus a back-edge is a cycle, and that cycle IS the agent loop: model proposes tool calls, tools run, control returns to the model, repeat until done. Plus recursion_limit, the guardrail that stops a runaway loop.
Stack the guardrails — callbacks, model filters, restricted tools, and clean-room sandboxing — so that if one layer misses, the next one catches
Layered defense-in-depth for agents: input/output guardrails via callbacks, Gemini safety settings, restricting tools, and sandboxing untrusted actions.
A guardrail that blocks a run before the model, chained ahead of a logger — and the upstream nil-update panic I fixed along the way.
A guardrail middleware that blocks an Microsoft Agent Framework Go run before the model, chained ahead of a logger — plus the tool-approval nil-update panic I fixed upstream in PR #472.
before/after the agent, model, and tool steps — and the single short-circuit rule that turns them into guardrails
Callbacks are lifecycle hooks around the agent, model, and tool steps — before/after each — used for guardrails (short-circuit by returning a response), logging, and mutating requests and responses.
Wrapping an agent run with async seams that log, time, guard, and short-circuit — without touching the agent's logic.
Wrapping an Microsoft Agent Framework agent run in Python with async middleware seams — timing, logging, and a guardrail that short-circuits a tool call before it runs.
This lesson teaches a middleware chain where a guardrail can block a run before the model is ever called, chained ahead of a logger.
Chain a guardrail middleware that yields its own refusal and skips the model ahead of a logger — short-circuiting a harmful request offline before Foundry is called.