A knowledge graph is not a project you finish; it's a system you run. Ontologies evolve, data drifts, quality decays, and the graph has to stay trustworthy while the business changes around it. This closing post covers the operational reality — governing the ontology, keeping the graph correct over time, and the honest question most teams skip: given the cost, when is a knowledge graph actually the right tool? It ties the series into a decision you can make with clear eyes.
A knowledge graph is a system you run, not a project you finish. Governing the ontology as it evolves, keeping the populated graph correct (freshness, re-resolution, provenance), and the honest question: given the cost, when is a knowledge graph the right tool — and when is a table or plain vector RAG cheaper and enough?
A frontier model knows the public internet and nothing about the customer. All the value of an AI deployment comes from the opposite: making the model reason over the customer's own documents, records, and knowledge. Grounding is the technical heart of the AI forward deployed engineer's job — connecting a general model to a specific company's messy, permissioned, incomplete data so its answers are about their reality, not the model's imagination.
A frontier model knows the public internet and nothing about the customer. All the value of an AI deployment comes from the opposite: making the model reason over the customer's own documents, records, and knowledge. Grounding — retrieval-augmented generation over messy, permissioned, incomplete data — is the technical heart of the AI FDE's job. With an interactive reference-architecture diagram.
This is the post the whole series was building toward. Retrieval-augmented generation gives an LLM relevant text to ground its answers, but plain RAG retrieves disconnected chunks and misses the relationships between facts. GraphRAG grounds the model in a knowledge graph instead — so it can follow relationships, reason across multiple hops, and answer questions that no single chunk contains. It's where ontologies stop being theory and become the thing that makes an AI system trustworthy on your domain.
The post the series built toward. Plain vector RAG retrieves disconnected chunks and misses relationships; GraphRAG retrieves a subgraph by traversing the knowledge graph, so multi-hop answers come from walked edges, not guessed connections — typed, verifiable, citable. When to use it, and why hybrid often wins.
A coding agent is only as good as its understanding of the code it's changing, and real codebases are enormous — far larger than any context window. The agent can't read everything, so it must find and load the right code for the task: navigate the repository, search for what's relevant, and build enough understanding to make correct changes. This grounding problem — understanding a large codebase through a small window — is where many coding-agent successes and failures are decided.
A coding agent is only as good as its understanding of the code it's changing, and codebases dwarf the context window. How agents find and understand the right code — search, navigation, strategic reading, repo conventions, semantic retrieval — and why most coding-agent failures are comprehension failures, so grounding is decisive.
The final decision isn't "which algorithm is best" — it's "which point on the recall-latency-memory triangle does my workload need, and what's the simplest thing that hits it." For a huge number of systems the honest answer is far less exotic than the vector-database marketing suggests, and knowing when you've genuinely outgrown Postgres is worth more than knowing HNSW's internals.
The final decision isn't 'which algorithm is best' — it's 'which point on the recall-latency-memory triangle does my workload need, and what's the simplest thing that hits it.' Often the honest answer is far less exotic than the vector-database marketing suggests.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe — the work that starts after the demo impresses everyone.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe.
The vector-storage decision has a boringly practical answer that cuts against the hype: for most systems, the database you already run with a vector extension beats adding a new specialized system — until scale or specific features force the upgrade.
A boringly practical answer that cuts against the hype: for most systems the database you already run with a vector extension beats adding a specialized system — until scale or specific features force the upgrade.
Real search is never pure vector similarity. Users want "similar documents from this project, updated this year" and they expect an exact product code to match exactly. Combining similarity with metadata filters and keyword search — without wrecking recall — is where academic ANN meets production requirements, and it's harder than it looks.
Real search is never pure vector similarity. Users want 'similar documents from this project, updated this year' and expect an exact product code to match exactly — combining similarity with metadata filters and keyword search without wrecking recall is where ANN meets production.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make the orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control that complex applications need.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control complex applications need.
The pieces from this series — routing, query transformation, graded retrieval, multi-hop, and evaluation — assemble into one system that reasons about retrieval as carefully as it reasons about the answer, while spending only as much as each question needs.
Routing, query transformation, graded retrieval, multi-hop, and evaluation assemble into one system that reasons about retrieval as carefully as the answer — while spending only as much as each question needs.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers — behind standard interfaces. The retriever, in particular, is just another Runnable, so RAG becomes a chain like any other.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers. The retriever is just another Runnable.
Vectors are big, and storing millions of them in full precision is where vector search gets expensive. Quantization compresses each vector into a fraction of its size — trading a little recall for large memory savings — and it's the technique that lets both IVF and HNSW scale from millions of vectors to billions without a memory budget that breaks the bank.
Vectors are big, and storing millions in full precision is where vector search gets expensive. Quantization compresses each vector into a fraction of its size — trading a little recall for large memory savings — and it's what lets IVF and HNSW scale to billions.
An on-device model only knows what's baked into its weights — nothing about the user's notes, messages, or documents. On-device RAG fixes that by doing retrieval entirely on the phone: embed the user's data locally, store the vectors locally, and retrieve locally, so the model can reason over personal data that never touches a server. It's the technique that makes a private assistant actually useful.
An on-device model only knows what's in its weights. On-device RAG fixes that by doing retrieval entirely on the phone — embed locally, store locally, retrieve locally — so the model can reason over personal data that never touches a server.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG, and it's built into LlamaIndex.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG.
The two systems everyone builds — retrieval-augmented generation and tool-using agents — are where DSPy's declarative approach pays off most, because they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
RAG and tool-using agents are where DSPy's declarative approach pays off most — they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
Every agentic technique in this series adds cost and complexity, so the only way to know any of it is worth it is to measure — and RAG needs measurement on two fronts at once: did it retrieve the right things, and did it answer faithfully from them?
Every agentic technique adds cost, so the only way to know it is worth it is to measure — and RAG needs measurement on two fronts: did it retrieve the right things, and did it answer faithfully from them?
A "chain" is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small, standard components into pipelines rather than writing monolithic prompt-and-parse code. Thinking in chains is thinking in composable steps, which is what makes LangChain applications modular, testable, and maintainable.
A chain is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small standard components into pipelines rather than writing monolithic prompt-and-parse code.
HNSW is the algorithm behind most modern vector databases, and its idea is borrowed from the "six degrees of separation" that connects any two people through a short chain of acquaintances. Build the right graph of vectors, and you can walk from a random entry point to a query's nearest neighbors in a handful of hops — searching millions of vectors while touching only a few hundred.
HNSW is behind most modern vector databases, and its idea comes from the 'six degrees of separation' that connects any two people through a short chain — build the right graph and you can walk from a random entry to a query's nearest neighbors in a handful of hops.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling memory well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling it well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
The most common architecture mistake in applied AI is reaching for fine-tuning to fix a knowledge problem — so the single most useful rule here is that RAG is for knowledge and fine-tuning is for behavior, and long-context is a convenience, not a strategy.
The most common architecture mistake is reaching for fine-tuning to fix a knowledge problem — so the key rule: RAG is for knowledge, fine-tuning is for behavior, and long-context is a convenience, not a strategy.
Some questions cannot be answered by any single search because the answer is assembled from facts that must be found in sequence, each retrieval informed by the last — and that is what iterative, multi-hop retrieval provides.
Some questions cannot be answered by any single search because the answer is assembled from facts found in sequence, each retrieval informed by the last — that is iterative, multi-hop retrieval.
Some of the biggest cost decisions are architectural — whether to feed knowledge through retrieval or bake it into a model, whether to prompt a big model or fine-tune a small one, and whether to rent tokens or run your own — and each trade turns on your volume and your task.
Some of the biggest cost decisions are architectural — retrieval versus long context, prompting a big model versus fine-tuning a small one, and renting tokens versus self-hosting — each turning on your volume.
Knowing how inverted indexes, BM25, and vector search work is one thing; assembling them into a search system that's fast, relevant, and maintainable is another. This closing post is the practical synthesis — the architecture of a real search system, the choices that matter, and the hard-won operational lessons that no algorithm paper mentions. It turns the series into something you can actually build, whether it's a product search box or the retrieval layer of a RAG application.
Knowing the algorithms is one thing; assembling a fast, relevant, maintainable search system is another. The anatomy (offline index / online retrieve-then-rank), the decisions that matter (buy vs build, keyword/semantic/hybrid), continuous relevance measurement, and why the retriever caps a RAG system's quality.
The simplest way to beat brute force is to avoid searching most of your data — cluster the vectors into regions, and at query time only look inside the few regions nearest the query. That's IVF, and its one tuning knob, how many regions to probe, is a clean, visible dial on the recall-versus-speed trade at the heart of the whole field.
The simplest way to beat brute force is to avoid searching most of your data — cluster the vectors into regions, and at query time only look inside the few nearest the query. That's IVF, and its one knob (nprobe) is a clean dial on the recall-versus-speed trade.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them (postprocessing, response synthesis) are where you tune quality.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them are where you tune quality.
Naive RAG trusts whatever it retrieved, which is how it produces confident answers grounded in the wrong documents; self-correcting retrieval adds the step it was missing — checking the results before using them.
Naive RAG trusts whatever it retrieved, which is how it produces confident answers grounded in the wrong documents. Self-correcting retrieval adds the missing step: check the results before using them.
Keyword search has one incurable weakness: it matches words, not meaning, so it misses relevant results that use different vocabulary than the query. Semantic search fixes this by representing queries and documents as vectors of meaning and matching by proximity in that space — the technology behind modern RAG. But the surprising lesson of production search is that semantic didn't replace keyword; the best systems combine them. This post explains vector search and why hybrid retrieval wins.
Keyword search matches words, not meaning. Semantic search embeds queries and documents into a vector space and matches by proximity — the engine of RAG — but it's weak at exact matches. Why keyword and semantic have opposite strengths, and why hybrid retrieval (not replacement) is the modern default.
The most underrated vector index is no index at all. Brute-force search — compare the query to every vector — is the one method with perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's not just adequate but optimal. Knowing when you don't need an ANN index is as valuable as knowing how they work.
The most underrated vector index is no index at all. Brute-force search has perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's optimal — knowing when you don't need ANN is as valuable as knowing how it works.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which organizes your data for which query pattern is the point.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which fits which query pattern is the point.
Real tasks are pipelines, not single calls, and in DSPy you build them the way you build a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Real tasks are pipelines, not single calls. In DSPy you build them like a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Real systems have more than one place to look, and the answer to "not everything should be retrieved from the same index — or retrieved at all" is to route queries and to treat retrieval as a tool the agent chooses to call.
Real systems have more than one place to look. The answer is to route queries to the right source — and to treat retrieval as a tool the agent chooses to call, or skips entirely.
BM25 ranks by a fixed formula, but real search engines have hundreds of relevance signals — freshness, popularity, personalization, link quality, and dozens of text features — and no human can hand-tune how to weigh them all. Learning to rank hands that job to machine learning: train a model on examples of good and bad rankings, and let it discover the optimal combination of signals. This post is where search ranking becomes a machine-learning problem, and the bridge to the neural retrieval behind modern AI.
Real engines have hundreds of relevance signals no human can hand-weight — so learning to rank trains a model on good/bad rankings. Pointwise/pairwise/listwise framings, gradient-boosted rankers, and the bridge to neural ranking, embedding retrieval, and the rerankers behind modern RAG.
"Nearest" is meaningless until you define "distance," and the metric you choose — cosine, dot product, or Euclidean — must match how your embedding model was trained or your search is quietly wrong. And in high dimensions, distance itself behaves so strangely that the naive intuitions you'd bring from 2D geometry actively mislead you.
'Nearest' is meaningless until you define 'distance,' and the metric you choose must match how your embedding model was trained or your search is quietly wrong — and in high dimensions, distance itself behaves so strangely that 2D intuitions mislead you.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
The user's question is written to be asked, not to be searched, so the first thing an agentic RAG system should do is turn that question into queries that actually retrieve well.
The user's question is written to be asked, not searched, so the first thing an agentic RAG system should do is turn that question into queries that actually retrieve well.
Retrieval is how you put external, current knowledge into a model's window, and doing it well is a context-engineering problem — the goal is not the most documents, but the right ones.
Retrieval is how you put external, current knowledge into a model's window, and doing it well is a context-engineering problem — the goal is not the most documents, but the right ones.
You changed the ranking function and the results look different — but are they better? Search quality feels subjective, yet the entire field rests on measuring it rigorously, because you cannot improve what you cannot measure. Evaluating search means quantifying relevance with metrics that respect what users actually care about: not just whether the right documents appear, but whether they appear near the top. This post is about measuring relevance honestly, offline and online.
You changed the ranking and results look different — but are they better? Precision and recall ignore order (fatal for a ranked list), so rank-aware metrics (precision@k, MRR, NDCG) rule. Offline test collections for fast iteration, online A/B tests as ground truth — the same metrics that evaluate RAG retrieval.
Every RAG system, recommendation engine, and semantic search box rests on one deceptively simple operation: given a query vector, find the closest vectors among millions. Doing it exactly is easy and doesn't scale; doing it fast enough to be useful means giving up exactness on purpose — and understanding that trade is the foundation of vector search.
Every RAG system and semantic search box rests on one operation: given a query vector, find the closest among millions. Doing it exactly is easy and doesn't scale; doing it fast means giving up exactness on purpose — and that trade is the foundation of vector search.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
Agentic RAG is what you get when retrieval stops being a fixed pipeline step and becomes a set of decisions an agent reasons through — whether to retrieve, what to search for, from where, how many times, and whether to trust the result.
Agentic RAG is what you get when retrieval stops being a fixed pipeline step and becomes a set of decisions an agent reasons through — whether, what, from where, how many times, and whether to trust results.
Finding the documents that contain your words is the easy part; putting the best one first is where search lives or dies. For decades, the answer to "how relevant is this document to this query?" has come from two intuitions made precise: a word matters more in a document that uses it often, and matters less if it's common across all documents. TF-IDF and BM25 turn those intuitions into the scoring functions that rank most of the world's text search — and understanding them demystifies why search returns what it does.
Finding matches is easy; putting the best first is where search lives. Two intuitions — a word matters more when frequent in a document, less when common across all — become TF-IDF and then BM25, the decades-long default (and the keyword score in hybrid RAG). Its lexical ceiling is the vocabulary gap.
Modern production AI is rarely "a model" — it is a foundation model wrapped in retrieval, context engineering, tools, and guardrails — and the biggest architectural mistake is reaching for fine-tuning before exhausting the cheaper, more reversible options.
Production AI is rarely a model — it's a composed system, and the biggest mistake is fine-tuning before exhausting cheaper, reversible options. Phase 3: compose before you train.
The retrieve-then-generate pipeline that launched a thousand demos hits a wall on real questions, and understanding exactly where it breaks is the case for making retrieval agentic.
The retrieve-then-generate pipeline that launched a thousand demos hits a wall on real questions. Understanding exactly where naive RAG breaks is the case for making retrieval agentic.
Before a single word goes into the index, it passes through a quiet, decisive step: text analysis. Deciding what counts as a "word," how to normalize it, and which words to keep is what lets a search for "running" find a document about "ran," and what makes "The" not clog the index. Get analysis right and search feels intelligent; get it wrong and no clever ranking can save it. This post is about the unglamorous preprocessing that quietly determines search quality.
Before a word hits the index it passes through text analysis: tokenization (what is a word?), normalization (case folding, stemming vs lemmatization), stop words, and synonyms. The unglamorous step where a huge share of search quality is won — and where query and documents must speak the same vocabulary.
Model quality is bounded by data quality, and the defects you tolerate here — poor lineage, silent drift, unmanaged PII, careless retrieval — resurface downstream as hallucinations, bias, privacy incidents, and un-auditable decisions.
Model quality is bounded by data quality; poor lineage, silent drift, unmanaged PII, and careless retrieval resurface as hallucinations and privacy incidents. Phase 2: the governed data foundation.
One data structure is responsible for the fact that you can search billions of documents in milliseconds: the inverted index. It's a beautifully simple idea — instead of asking "what words are in this document?", precompute "what documents contain this word?" — and it's the engine inside every search engine from Google to Elasticsearch. This post builds the inverted index from scratch and shows why it, and not raw scanning, is how search actually works.
One structure lets you search billions of documents in milliseconds: the inverted index. Instead of 'what words are in this document?', precompute 'what documents contain this word?'. Postings lists, intersecting them for queries, and why it scales to the web — and underlies the keyword half of hybrid RAG.
Why agents and retrieval turn a prompt injection into real-world action, how to red-team the highest-risk AI surface with benign canaries, and the least-privilege controls that shrink an attacker's blast radius.
The highest-risk modern surface: indirect injection via RAG/tools, tool abuse and excessive-agency exploitation, memory poisoning, multi-step attacks, and data-exfiltration channels — with a canary methodology and least privilege as the primary control.
Part five of the AI Security Engineering series: the two OWASP LLM risks that live in the plumbing around the model — the documents your agent retrieves and the models, datasets, and dependencies it is built from — and the Python patterns that treat both as untrusted until proven otherwise.
Securing the components around the model: RAG as an injection and poisoning vector (treat retrieved content as hostile, enforce provenance and per-user authz) and supply-chain risk (safetensors over pickle, model provenance, pinned and vetted dependencies and tools).
Adding safety to a watsonx app in Python with two complementary layers — the built-in HAP and PII moderations that watsonx.ai applies to generation, and IBM's dedicated Granite Guardian risk-detection models run as classifiers around your main model to catch harm, jailbreaks, and RAG hallucination before a response reaches a user.
Add safety to a watsonx app in Python with two layers: built-in HAP/PII moderations on generation, and Granite Guardian — IBM's dedicated risk models — run as a classifier to screen input and check RAG groundedness, wrapped in a guarded-generate flow.
Every time you type into a search box and get useful results in milliseconds from billions of documents, a remarkable amount of engineering is hiding behind the cursor. Information retrieval is the science of finding relevant information in a large collection, and it's one of the oldest, most practical, and — in the age of RAG — most suddenly-relevant-again fields in computing. This series builds search from first principles, and it starts with why "just find the matching documents" is far harder than it sounds.
Finding relevant information in a huge collection in milliseconds is far harder than 'just find the matching documents'. The two problems hiding in search (matching and ranking), why scanning fails, and the offline-index/online-retrieve pipeline that makes search fast — and that RAG reuses directly.
Assembling a full retrieval-augmented generation pipeline in Python on watsonx.ai — ingest and chunk documents, embed them with slate, retrieve by cosine, rerank for precision, then generate a grounded, cited answer with a Granite model, shown both from scratch and with langchain-ibm.
Build a full RAG pipeline in Python on watsonx: chunk (with Docling for parsing), embed and store, retrieve, rerank, and generate a grounded, cited answer with a Granite model — shown both from scratch and via the idiomatic langchain-ibm assembly, with notes on scaling to watsonx.data/Milvus.
Assembling a full retrieval-augmented generation pipeline in Python — chunk and embed with NeMo Retriever, shortlist by cosine, sharpen with a reranker, then ground a ChatNVIDIA answer that cites its sources — first from scratch, then the idiomatic LangChain wiring.
Build a full RAG pipeline in Python on NVIDIA's stack: chunk and embed with NeMo Retriever, shortlist by cosine, rerank, then generate a grounded, cited answer with ChatNVIDIA — shown both from scratch and the idiomatic LangChain assembly, with notes on scaling to a vector DB.
A defender's field guide to the injection and jailbreak techniques a red-teamer probes for — the taxonomy, why each one works, and how to turn it into a re-runnable test suite that maps every passed test to a concrete fix.
A deeper, test-focused look at injection and jailbreak families — direct vs indirect (the RAG/agent threat), role-play, obfuscation, many-shot, multi-turn, cross-lingual — plus a red-team methodology: build a probe suite, mutate, test input and retrieval paths, measure, re-test.
Part three of the AI Security Engineering series: protecting the data that flows through an LLM system — how sensitive information leaks out of prompts, logs, and retrieval, and the engineering controls (redaction, data minimization, per-user retrieval authz, residency choices) that actually stop it.
Protecting data in LLM systems: sensitive-information disclosure, PII in prompts and logs (your observability can be the leak), redaction with Presidio, data residency, and per-user access control on the retrieval layer so RAG doesn't leak across tenants.
Building RAG's retrieval core with watsonx.ai from Python — turning a corpus into vectors with IBM's slate embedding models, scoring a query against them, and then sharpening the shortlist with a reranking model so the LLM gets the right passages, not just plausible ones.
Use watsonx.ai's slate embedding models and reranking from Python to build RAG's retrieval core: embed_documents vs embed_query, numpy cosine scoring, and a two-stage retrieve-then-rerank pipeline — plus the langchain-ibm WatsonxEmbeddings/WatsonxRerank path.
Building RAG's retrieval core in Python — turning a corpus and a query into vectors with NeMo Retriever embedding NIMs, scoring by cosine similarity, then sharpening the shortlist with a cross-encoder reranker NIM.
Use NeMo Retriever from Python to build RAG's retrieval core: NVIDIAEmbeddings (embed_documents vs embed_query for the asymmetric passage/query distinction) with cosine scoring, and NVIDIARerank.compress_documents for a two-stage retrieve-then-rerank pipeline.
How to query a Knowledge Base for Amazon Bedrock from Go — the managed retrieve-then-read layer — using both the low-level Retrieve call and the one-shot RetrieveAndGenerate, with citations wired through.
RAG on Bedrock in Go with Knowledge Bases: the retrieve-then-read pattern via Retrieve, the one-shot managed path via RetrieveAndGenerate with citations, and when to reach for each — plus reading grounding so you keep RAG's trust benefit.
How to know whether an LLM system actually works — building an eval dataset, the four metric families (deterministic checks, text overlap, embedding similarity, LLM-as-judge) in Go, task-specific eval for RAG and classification, and wiring a scored regression gate into CI so you measure instead of vibe.
How to know whether an LLM system works when outputs are non-deterministic: build an eval dataset, score with deterministic checks, embedding similarity, and LLM-as-judge (with its biases), evaluate RAG and classification, and gate regressions in CI.
Before you attack an AI system you need a map of it: the components an adversary can influence, the trust boundaries between them, and a taxonomy that sorts attacks by goal and stage so your red-teaming is systematic instead of a grab-bag of the attacks that happen to trend that week.
Mapping the AI attack surface so red-teaming is systematic: threat-modeling the components (model, prompts, training/RAG data, tools, guardrails) and an attack taxonomy by goal (integrity/availability/privacy/abuse) and stage (training vs inference), aligned to NIST AI 100-2 and ATLAS.
Why the #1 risk on the OWASP LLM Top 10 has no clean fix — the model can't tell your instructions apart from the text it reads — and the defense-in-depth pattern that actually shrinks the blast radius.
The #1 LLM risk: direct vs indirect prompt injection (the latter is the real threat for RAG and agents), jailbreak families, and honestly-rated defense-in-depth — delimiting untrusted data, least-privilege tools, human-in-the-loop, and detection's limits.
The opening post of a hands-on series for builders: what it means to systematically stress-test an AI system — model, prompts, retrieval, tools, and guardrails — to surface its failures before adversaries or ordinary users do, how that differs from pentesting and robustness testing, and the frameworks and ethics that keep the work rigorous.
The opener to a defensive AI red-teaming series: stress-testing AI systems (models + prompts + RAG + tools + guardrails) to find failures before adversaries do — how it differs from pentesting, the frameworks (OWASP GenAI, MITRE ATLAS, NIST AI 100-2), and ethical rules of engagement.
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide answers all seven.
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide...
Why the naive RAG pipeline from post 9 underperforms in production, and the concrete, evaluation-driven fixes — structure-aware chunking, hybrid search, reranking, query transformation, and deliberate context construction — each explained with the reasoning and a real Go sketch.
Why naive RAG underperforms and the techniques that fix it: measure recall@k first, then structure-aware chunking, hybrid dense+BM25 search fused with RRF, over-retrieve-then-rerank, query transformation (HyDE, multi-query), and deliberate context construction against lost-in-the-middle.
Wire the embedding client, vector store, and chat client from the last five posts into one working RAG pipeline in Go — ingest and chunk documents, retrieve the top matches for a question, inject them as grounded context, and generate a cited answer, all from scratch.
Assemble embeddings and vector search into a working RAG pipeline in Go: chunk documents, embed and store them, retrieve the top-k for a query, augment the prompt with grounded context (and cite sources), then generate — a baseline end-to-end Answer() built from scratch.
Post 18 of 26 in "Google ADK, Concept by Concept" — retrieval tools, grounding metadata, rendering citations, and the retrieve→augment→generate loop.
Grounding answers in real data: retrieval tools, grounding metadata returned with responses, rendering citations from that metadata, and the retrieve-augment-generate RAG pattern in ADK.
The final lesson ties the whole Go tutorial into one small product: an assistant that answers questions about your docs — grounded, cited, and refusing to guess.
The capstone: a grounded DocQA agent that answers only from embedded docs via a search_docs tool, cites sources, and refuses to guess — with an optional reviewer.
Vector search treats every chunk as independent. GraphRAG models the relationships between entities, communities, and concepts. For corpus-spanning questions ("what's the relationship between X and Y"), graph wins.
Embedding a question and embedding an answer often produce different vectors. HyDE generates a hypothetical answer to the question, embeds that, and retrieves on it. Retrieval quality goes up disproportionately.
Naive RAG retrieves on every query. Self-RAG decides whether to retrieve. CRAG decides whether the retrieved content is good enough or needs corrective retrieval. Two papers; both worth implementing.
An Indian banking deployment needs to handle Hindi, Marathi, Tamil, Bengali, and English in the same retrieval pipeline. Bhashini (the government's language stack) plus cross-lingual embeddings make it tractable.