Archive
1046 posts · Page 30 of 88. ← Blog
Adding a safety layer to an NVIDIA-stack Python app with NeMo Guardrails — and why running it in-process, with no HTTP boundary, is the quiet advantage Python gives you over a separate guardrails server.
Add safety with NeMo Guardrails, in-process in Python: the rail types (input/output/dialog/retrieval), the config.yml + Colang model, loading with RailsConfig/LLMRails backed by a NIM model, and detecting when a rail trips instead of treating a refusal as a normal answer.
How to put Amazon Bedrock Guardrails in front of a model from Go — attaching one to a Converse call, screening raw text with ApplyGuardrail, and reading whether the guardrail actually intervened.
Guardrails for Amazon Bedrock in Go: content filters, denied topics, PII/sensitive-information filters, and contextual grounding — attaching a guardrail to a Converse call and screening arbitrary text with ApplyGuardrail, checking for intervention.
The last post in the series: what changes when the LLM system you built across posts 1-14 has to run for real — reliability, security, cost, observability, evaluation gates, and versioning, from a Go engineer's seat, with code where it earns its place.
The capstone: running an LLM system in production from a Go engineer's seat — reliability (timeouts, retries, fallbacks), security (injection, least-privilege tools, secrets), cost and observability, CI eval gates, and versioning models and prompts.
Errors are part of every API's contract, and gRPC has a specific, structured model for them: a fixed set of status codes, a message, and optional rich details — not the HTTP status codes you know from REST. Using this model well is what makes failures actionable for callers instead of opaque. Getting it wrong turns every error into a debugging session.
Errors are part of every API's contract, and gRPC has a specific, structured model: a fixed set of status codes, a message, and optional rich details — not the HTTP status codes you know from REST. Using it well makes failures actionable for callers instead of opaque; getting it wrong turns every error into a debugging session.
Reasoning models are a genuine advance — and it's precisely because they're so impressive that their limits are worth stating plainly. A model that writes out careful, step-by-step reasoning invites you to trust the reasoning, and that trust is not always warranted. The chain of thought may not be why the model actually answered. More thinking eventually stops helping. And some problems no amount of test-time compute will solve. Knowing the edges is part of using the capability wisely.
Reasoning models are a genuine advance — and precisely because they're so impressive, their limits are worth stating plainly. The chain of thought may not be why the model actually answered. More thinking eventually stops helping. And some problems no amount of test-time compute will solve.
The most expensive AI failures are systems that work technically but solve the wrong problem, cannot show a return, or have no one accountable when they misbehave — and all three are decided before a single model is chosen.
The most expensive AI failures solve the wrong problem, can't show a return, or have no accountable owner — all decided before a model is chosen. Phase 0: strategy, use-case selection, and the named owner.
The Model Context Protocol lets Claude Code reach beyond your codebase — into your databases, issue trackers, docs, and services — through a standard, pluggable interface.
The Model Context Protocol lets Claude Code reach beyond the codebase into databases, trackers, docs, and services through a standard interface — what MCP is, how to connect servers, and treating each server as a least-privilege trust decision.
The demo ran on your laptop with clean data — production means the customer's messy systems, their security rules, and their data as it actually is, which is where most forward deployed work is really won.
Production means the customer's messy data, systems you don't control, and security you must pass: profiling dirty data, loose-coupling integrations, treating security/residency as gates, and deploying + handing over so you can actually leave.
A defender's tour of the attacks that target the model and its data — prompt and context extraction, training-data memorization, membership inference and model inversion, model stealing, poisoning and backdoors, and evasion — with what a red-teamer tests and what actually stops each one.
Attacks on the model and its data: system-prompt and training-data extraction, membership inference and model inversion, model stealing, and poisoning/backdoors — what to test and defend, and the honest risk difference between using hosted models and training your own.
The core of the architect's job is not drawing boxes but making, justifying, and recording the significant, hard-to-reverse decisions a system is built on — deliberately, under uncertainty, and with the reasoning written down.
The core of the job: making and recording decisions under uncertainty — one-way vs two-way doors, structured trade-off analysis, avoiding resume-driven development, and Architecture Decision Records (ADRs) that keep the why alive.
The author's side of code review — turning a pull request from an ego threat into a growth loop by separating your identity from your code, responding to every comment, and pushing back with reasoning instead of feelings.
The author's side: separating your identity from your code, assuming good intent, responding to every comment, pushing back with reasoning (not ego), and turning review into a growth loop instead of an ego threat.
Securing everything you didn't write — from finding known-vulnerable dependencies with SCA, to generating an SBOM you actually act on, to proving provenance with signatures and SLSA so you know and verify what you ship.
Securing everything you didn't write: SCA for known-vulnerable dependencies, the supply-chain threat model (typosquatting, dependency confusion, build compromise), SBOMs (SPDX/CycloneDX), and provenance/integrity with SLSA and Sigstore signing.