The final decision isn't "which algorithm is best" — it's "which point on the recall-latency-memory triangle does my workload need, and what's the simplest thing that hits it." For a huge number of systems the honest answer is far less exotic than the vector-database marketing suggests, and knowing when you've genuinely outgrown Postgres is worth more than knowing HNSW's internals.
The final decision isn't 'which algorithm is best' — it's 'which point on the recall-latency-memory triangle does my workload need, and what's the simplest thing that hits it.' Often the honest answer is far less exotic than the vector-database marketing suggests.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe — the work that starts after the demo impresses everyone.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe.
The vector-storage decision has a boringly practical answer that cuts against the hype: for most systems, the database you already run with a vector extension beats adding a new specialized system — until scale or specific features force the upgrade.
A boringly practical answer that cuts against the hype: for most systems the database you already run with a vector extension beats adding a specialized system — until scale or specific features force the upgrade.
Real search is never pure vector similarity. Users want "similar documents from this project, updated this year" and they expect an exact product code to match exactly. Combining similarity with metadata filters and keyword search — without wrecking recall — is where academic ANN meets production requirements, and it's harder than it looks.
Real search is never pure vector similarity. Users want 'similar documents from this project, updated this year' and expect an exact product code to match exactly — combining similarity with metadata filters and keyword search without wrecking recall is where ANN meets production.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make the orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control that complex applications need.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control complex applications need.
The pieces from this series — routing, query transformation, graded retrieval, multi-hop, and evaluation — assemble into one system that reasons about retrieval as carefully as it reasons about the answer, while spending only as much as each question needs.
Routing, query transformation, graded retrieval, multi-hop, and evaluation assemble into one system that reasons about retrieval as carefully as the answer — while spending only as much as each question needs.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers — behind standard interfaces. The retriever, in particular, is just another Runnable, so RAG becomes a chain like any other.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers. The retriever is just another Runnable.
Vectors are big, and storing millions of them in full precision is where vector search gets expensive. Quantization compresses each vector into a fraction of its size — trading a little recall for large memory savings — and it's the technique that lets both IVF and HNSW scale from millions of vectors to billions without a memory budget that breaks the bank.
Vectors are big, and storing millions in full precision is where vector search gets expensive. Quantization compresses each vector into a fraction of its size — trading a little recall for large memory savings — and it's what lets IVF and HNSW scale to billions.
An on-device model only knows what's baked into its weights — nothing about the user's notes, messages, or documents. On-device RAG fixes that by doing retrieval entirely on the phone: embed the user's data locally, store the vectors locally, and retrieve locally, so the model can reason over personal data that never touches a server. It's the technique that makes a private assistant actually useful.
An on-device model only knows what's in its weights. On-device RAG fixes that by doing retrieval entirely on the phone — embed locally, store locally, retrieve locally — so the model can reason over personal data that never touches a server.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG, and it's built into LlamaIndex.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG.
The two systems everyone builds — retrieval-augmented generation and tool-using agents — are where DSPy's declarative approach pays off most, because they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
RAG and tool-using agents are where DSPy's declarative approach pays off most — they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
Every agentic technique in this series adds cost and complexity, so the only way to know any of it is worth it is to measure — and RAG needs measurement on two fronts at once: did it retrieve the right things, and did it answer faithfully from them?
Every agentic technique adds cost, so the only way to know it is worth it is to measure — and RAG needs measurement on two fronts: did it retrieve the right things, and did it answer faithfully from them?
A "chain" is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small, standard components into pipelines rather than writing monolithic prompt-and-parse code. Thinking in chains is thinking in composable steps, which is what makes LangChain applications modular, testable, and maintainable.
A chain is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small standard components into pipelines rather than writing monolithic prompt-and-parse code.
HNSW is the algorithm behind most modern vector databases, and its idea is borrowed from the "six degrees of separation" that connects any two people through a short chain of acquaintances. Build the right graph of vectors, and you can walk from a random entry point to a query's nearest neighbors in a handful of hops — searching millions of vectors while touching only a few hundred.
HNSW is behind most modern vector databases, and its idea comes from the 'six degrees of separation' that connects any two people through a short chain — build the right graph and you can walk from a random entry to a query's nearest neighbors in a handful of hops.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling memory well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling it well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
The most common architecture mistake in applied AI is reaching for fine-tuning to fix a knowledge problem — so the single most useful rule here is that RAG is for knowledge and fine-tuning is for behavior, and long-context is a convenience, not a strategy.
The most common architecture mistake is reaching for fine-tuning to fix a knowledge problem — so the key rule: RAG is for knowledge, fine-tuning is for behavior, and long-context is a convenience, not a strategy.
Some questions cannot be answered by any single search because the answer is assembled from facts that must be found in sequence, each retrieval informed by the last — and that is what iterative, multi-hop retrieval provides.
Some questions cannot be answered by any single search because the answer is assembled from facts found in sequence, each retrieval informed by the last — that is iterative, multi-hop retrieval.
Some of the biggest cost decisions are architectural — whether to feed knowledge through retrieval or bake it into a model, whether to prompt a big model or fine-tune a small one, and whether to rent tokens or run your own — and each trade turns on your volume and your task.
Some of the biggest cost decisions are architectural — retrieval versus long context, prompting a big model versus fine-tuning a small one, and renting tokens versus self-hosting — each turning on your volume.
The simplest way to beat brute force is to avoid searching most of your data — cluster the vectors into regions, and at query time only look inside the few regions nearest the query. That's IVF, and its one tuning knob, how many regions to probe, is a clean, visible dial on the recall-versus-speed trade at the heart of the whole field.
The simplest way to beat brute force is to avoid searching most of your data — cluster the vectors into regions, and at query time only look inside the few nearest the query. That's IVF, and its one knob (nprobe) is a clean dial on the recall-versus-speed trade.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them (postprocessing, response synthesis) are where you tune quality.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them are where you tune quality.
Naive RAG trusts whatever it retrieved, which is how it produces confident answers grounded in the wrong documents; self-correcting retrieval adds the step it was missing — checking the results before using them.
Naive RAG trusts whatever it retrieved, which is how it produces confident answers grounded in the wrong documents. Self-correcting retrieval adds the missing step: check the results before using them.
The most underrated vector index is no index at all. Brute-force search — compare the query to every vector — is the one method with perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's not just adequate but optimal. Knowing when you don't need an ANN index is as valuable as knowing how they work.
The most underrated vector index is no index at all. Brute-force search has perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's optimal — knowing when you don't need ANN is as valuable as knowing how it works.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which organizes your data for which query pattern is the point.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which fits which query pattern is the point.
Real tasks are pipelines, not single calls, and in DSPy you build them the way you build a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Real tasks are pipelines, not single calls. In DSPy you build them like a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Real systems have more than one place to look, and the answer to "not everything should be retrieved from the same index — or retrieved at all" is to route queries and to treat retrieval as a tool the agent chooses to call.
Real systems have more than one place to look. The answer is to route queries to the right source — and to treat retrieval as a tool the agent chooses to call, or skips entirely.
"Nearest" is meaningless until you define "distance," and the metric you choose — cosine, dot product, or Euclidean — must match how your embedding model was trained or your search is quietly wrong. And in high dimensions, distance itself behaves so strangely that the naive intuitions you'd bring from 2D geometry actively mislead you.
'Nearest' is meaningless until you define 'distance,' and the metric you choose must match how your embedding model was trained or your search is quietly wrong — and in high dimensions, distance itself behaves so strangely that 2D intuitions mislead you.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
The user's question is written to be asked, not to be searched, so the first thing an agentic RAG system should do is turn that question into queries that actually retrieve well.
The user's question is written to be asked, not searched, so the first thing an agentic RAG system should do is turn that question into queries that actually retrieve well.
Retrieval is how you put external, current knowledge into a model's window, and doing it well is a context-engineering problem — the goal is not the most documents, but the right ones.
Retrieval is how you put external, current knowledge into a model's window, and doing it well is a context-engineering problem — the goal is not the most documents, but the right ones.
Every RAG system, recommendation engine, and semantic search box rests on one deceptively simple operation: given a query vector, find the closest vectors among millions. Doing it exactly is easy and doesn't scale; doing it fast enough to be useful means giving up exactness on purpose — and understanding that trade is the foundation of vector search.
Every RAG system and semantic search box rests on one operation: given a query vector, find the closest among millions. Doing it exactly is easy and doesn't scale; doing it fast means giving up exactness on purpose — and that trade is the foundation of vector search.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
Agentic RAG is what you get when retrieval stops being a fixed pipeline step and becomes a set of decisions an agent reasons through — whether to retrieve, what to search for, from where, how many times, and whether to trust the result.
Agentic RAG is what you get when retrieval stops being a fixed pipeline step and becomes a set of decisions an agent reasons through — whether, what, from where, how many times, and whether to trust results.
Modern production AI is rarely "a model" — it is a foundation model wrapped in retrieval, context engineering, tools, and guardrails — and the biggest architectural mistake is reaching for fine-tuning before exhausting the cheaper, more reversible options.
Production AI is rarely a model — it's a composed system, and the biggest mistake is fine-tuning before exhausting cheaper, reversible options. Phase 3: compose before you train.
The retrieve-then-generate pipeline that launched a thousand demos hits a wall on real questions, and understanding exactly where it breaks is the case for making retrieval agentic.
The retrieve-then-generate pipeline that launched a thousand demos hits a wall on real questions. Understanding exactly where naive RAG breaks is the case for making retrieval agentic.
Model quality is bounded by data quality, and the defects you tolerate here — poor lineage, silent drift, unmanaged PII, careless retrieval — resurface downstream as hallucinations, bias, privacy incidents, and un-auditable decisions.
Model quality is bounded by data quality; poor lineage, silent drift, unmanaged PII, and careless retrieval resurface as hallucinations and privacy incidents. Phase 2: the governed data foundation.
Why agents and retrieval turn a prompt injection into real-world action, how to red-team the highest-risk AI surface with benign canaries, and the least-privilege controls that shrink an attacker's blast radius.
The highest-risk modern surface: indirect injection via RAG/tools, tool abuse and excessive-agency exploitation, memory poisoning, multi-step attacks, and data-exfiltration channels — with a canary methodology and least privilege as the primary control.
Part five of the AI Security Engineering series: the two OWASP LLM risks that live in the plumbing around the model — the documents your agent retrieves and the models, datasets, and dependencies it is built from — and the Python patterns that treat both as untrusted until proven otherwise.
Securing the components around the model: RAG as an injection and poisoning vector (treat retrieved content as hostile, enforce provenance and per-user authz) and supply-chain risk (safetensors over pickle, model provenance, pinned and vetted dependencies and tools).
Adding safety to a watsonx app in Python with two complementary layers — the built-in HAP and PII moderations that watsonx.ai applies to generation, and IBM's dedicated Granite Guardian risk-detection models run as classifiers around your main model to catch harm, jailbreaks, and RAG hallucination before a response reaches a user.
Add safety to a watsonx app in Python with two layers: built-in HAP/PII moderations on generation, and Granite Guardian — IBM's dedicated risk models — run as a classifier to screen input and check RAG groundedness, wrapped in a guarded-generate flow.
Assembling a full retrieval-augmented generation pipeline in Python on watsonx.ai — ingest and chunk documents, embed them with slate, retrieve by cosine, rerank for precision, then generate a grounded, cited answer with a Granite model, shown both from scratch and with langchain-ibm.
Build a full RAG pipeline in Python on watsonx: chunk (with Docling for parsing), embed and store, retrieve, rerank, and generate a grounded, cited answer with a Granite model — shown both from scratch and via the idiomatic langchain-ibm assembly, with notes on scaling to watsonx.data/Milvus.
Assembling a full retrieval-augmented generation pipeline in Python — chunk and embed with NeMo Retriever, shortlist by cosine, sharpen with a reranker, then ground a ChatNVIDIA answer that cites its sources — first from scratch, then the idiomatic LangChain wiring.
Build a full RAG pipeline in Python on NVIDIA's stack: chunk and embed with NeMo Retriever, shortlist by cosine, rerank, then generate a grounded, cited answer with ChatNVIDIA — shown both from scratch and the idiomatic LangChain assembly, with notes on scaling to a vector DB.
A defender's field guide to the injection and jailbreak techniques a red-teamer probes for — the taxonomy, why each one works, and how to turn it into a re-runnable test suite that maps every passed test to a concrete fix.
A deeper, test-focused look at injection and jailbreak families — direct vs indirect (the RAG/agent threat), role-play, obfuscation, many-shot, multi-turn, cross-lingual — plus a red-team methodology: build a probe suite, mutate, test input and retrieval paths, measure, re-test.
Part three of the AI Security Engineering series: protecting the data that flows through an LLM system — how sensitive information leaks out of prompts, logs, and retrieval, and the engineering controls (redaction, data minimization, per-user retrieval authz, residency choices) that actually stop it.
Protecting data in LLM systems: sensitive-information disclosure, PII in prompts and logs (your observability can be the leak), redaction with Presidio, data residency, and per-user access control on the retrieval layer so RAG doesn't leak across tenants.
Building RAG's retrieval core with watsonx.ai from Python — turning a corpus into vectors with IBM's slate embedding models, scoring a query against them, and then sharpening the shortlist with a reranking model so the LLM gets the right passages, not just plausible ones.
Use watsonx.ai's slate embedding models and reranking from Python to build RAG's retrieval core: embed_documents vs embed_query, numpy cosine scoring, and a two-stage retrieve-then-rerank pipeline — plus the langchain-ibm WatsonxEmbeddings/WatsonxRerank path.
Building RAG's retrieval core in Python — turning a corpus and a query into vectors with NeMo Retriever embedding NIMs, scoring by cosine similarity, then sharpening the shortlist with a cross-encoder reranker NIM.
Use NeMo Retriever from Python to build RAG's retrieval core: NVIDIAEmbeddings (embed_documents vs embed_query for the asymmetric passage/query distinction) with cosine scoring, and NVIDIARerank.compress_documents for a two-stage retrieve-then-rerank pipeline.
How to query a Knowledge Base for Amazon Bedrock from Go — the managed retrieve-then-read layer — using both the low-level Retrieve call and the one-shot RetrieveAndGenerate, with citations wired through.
RAG on Bedrock in Go with Knowledge Bases: the retrieve-then-read pattern via Retrieve, the one-shot managed path via RetrieveAndGenerate with citations, and when to reach for each — plus reading grounding so you keep RAG's trust benefit.
How to know whether an LLM system actually works — building an eval dataset, the four metric families (deterministic checks, text overlap, embedding similarity, LLM-as-judge) in Go, task-specific eval for RAG and classification, and wiring a scored regression gate into CI so you measure instead of vibe.
How to know whether an LLM system works when outputs are non-deterministic: build an eval dataset, score with deterministic checks, embedding similarity, and LLM-as-judge (with its biases), evaluate RAG and classification, and gate regressions in CI.
Before you attack an AI system you need a map of it: the components an adversary can influence, the trust boundaries between them, and a taxonomy that sorts attacks by goal and stage so your red-teaming is systematic instead of a grab-bag of the attacks that happen to trend that week.
Mapping the AI attack surface so red-teaming is systematic: threat-modeling the components (model, prompts, training/RAG data, tools, guardrails) and an attack taxonomy by goal (integrity/availability/privacy/abuse) and stage (training vs inference), aligned to NIST AI 100-2 and ATLAS.
Why the #1 risk on the OWASP LLM Top 10 has no clean fix — the model can't tell your instructions apart from the text it reads — and the defense-in-depth pattern that actually shrinks the blast radius.
The #1 LLM risk: direct vs indirect prompt injection (the latter is the real threat for RAG and agents), jailbreak families, and honestly-rated defense-in-depth — delimiting untrusted data, least-privilege tools, human-in-the-loop, and detection's limits.
The opening post of a hands-on series for builders: what it means to systematically stress-test an AI system — model, prompts, retrieval, tools, and guardrails — to surface its failures before adversaries or ordinary users do, how that differs from pentesting and robustness testing, and the frameworks and ethics that keep the work rigorous.
The opener to a defensive AI red-teaming series: stress-testing AI systems (models + prompts + RAG + tools + guardrails) to find failures before adversaries do — how it differs from pentesting, the frameworks (OWASP GenAI, MITRE ATLAS, NIST AI 100-2), and ethical rules of engagement.
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide answers all seven.
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide...
Why the naive RAG pipeline from post 9 underperforms in production, and the concrete, evaluation-driven fixes — structure-aware chunking, hybrid search, reranking, query transformation, and deliberate context construction — each explained with the reasoning and a real Go sketch.
Why naive RAG underperforms and the techniques that fix it: measure recall@k first, then structure-aware chunking, hybrid dense+BM25 search fused with RRF, over-retrieve-then-rerank, query transformation (HyDE, multi-query), and deliberate context construction against lost-in-the-middle.
Wire the embedding client, vector store, and chat client from the last five posts into one working RAG pipeline in Go — ingest and chunk documents, retrieve the top matches for a question, inject them as grounded context, and generate a cited answer, all from scratch.
Assemble embeddings and vector search into a working RAG pipeline in Go: chunk documents, embed and store them, retrieve the top-k for a query, augment the prompt with grounded context (and cite sources), then generate — a baseline end-to-end Answer() built from scratch.
Post 18 of 26 in "Google ADK, Concept by Concept" — retrieval tools, grounding metadata, rendering citations, and the retrieve→augment→generate loop.
Grounding answers in real data: retrieval tools, grounding metadata returned with responses, rendering citations from that metadata, and the retrieve-augment-generate RAG pattern in ADK.
The final lesson ties the whole Go tutorial into one small product: an assistant that answers questions about your docs — grounded, cited, and refusing to guess.
The capstone: a grounded DocQA agent that answers only from embedded docs via a search_docs tool, cites sources, and refuses to guess — with an optional reviewer.
Vector search treats every chunk as independent. GraphRAG models the relationships between entities, communities, and concepts. For corpus-spanning questions ("what's the relationship between X and Y"), graph wins.
Embedding a question and embedding an answer often produce different vectors. HyDE generates a hypothetical answer to the question, embeds *that*, and retrieves on it. Retrieval quality goes up disproportionately.
Naive RAG retrieves on every query. Self-RAG decides whether to retrieve. CRAG decides whether the retrieved content is good enough or needs corrective retrieval. Two papers; both worth implementing.
An Indian banking deployment needs to handle Hindi, Marathi, Tamil, Bengali, and English in the same retrieval pipeline. Bhashini (the government's language stack) plus cross-lingual embeddings make it tractable.