The gap between a LangChain demo and a LangChain production system is the same gap as any LLM application — observability, evaluation, cost, and reliability — and LangChain's answer is LangSmith plus the discipline the rest of your engineering already has. This closing post covers operating LangChain applications and the honest verdict on when to use the framework.
The gap between a LangChain demo and a production system is the same as any LLM application — observability, evaluation, cost, and reliability — and LangChain's answer is LangSmith plus the discipline your engineering already has.
A framework earns its place not in the demo but in production — under real traffic, real failures, and the need to observe, control cost, and evolve. Pydantic AI's typed, testable design carries into production well, and paired with observability and the model-agnosticism it's had all along, it makes agents you can actually operate. This closing post covers taking a Pydantic AI agent live.
A framework earns its place in production — under real traffic, failures, and the need to observe, control cost, and evolve. Pydantic AI's typed, testable design carries into production well, paired with observability and model-agnosticism.
A chain answers one call; a conversation needs to remember. LangChain handles memory by treating conversation history as data you manage and pass in — and, for anything beyond simple chat history, hands state management to LangGraph. Knowing which is which keeps your stateful applications clean instead of tangled.
A chain answers one call; a conversation needs to remember. LangChain handles memory by treating conversation history as data you manage and pass in — and, for richer state, hands state management to LangGraph.
Most agent code is tested by running it and eyeballing the output — because testing agents properly is genuinely hard. Pydantic AI's whole design has been quietly building toward making it easy: type safety, dependency injection, and test models combine so you can unit-test agent logic deterministically, offline, without ever calling a real LLM. This is arguably the framework's biggest practical advantage.
Most agent code is tested by running it and eyeballing the output, because testing agents properly is hard. Pydantic AI's design builds toward making it easy: type safety, dependency injection, and test models let you unit-test agent logic deterministically, offline, with no real LLM.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe — the work that starts after the demo impresses everyone.
A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe.
CrewAI makes it easy to build a multi-agent system and just as easy to build one that's slow, expensive, and unreliable — so production CrewAI is mostly about discipline: measure it, keep it as simple as the problem allows, and use Flows for the parts that must be dependable.
CrewAI makes it easy to build a multi-agent system and just as easy to build one that's slow, expensive, and unreliable — so production CrewAI is mostly discipline: measure it, keep it simple, and use Flows where it must be dependable.
Chains follow a path you define; agents decide the path themselves. LangChain gives you both the tools an agent uses and — increasingly through LangGraph — the machinery to run agent loops reliably. Understanding where LangChain's tools end and LangGraph's orchestration begins is the key to building agents that work rather than agents that wander.
Chains follow a path you define; agents decide the path themselves. LangChain gives you the tools an agent uses and — increasingly through LangGraph — the machinery to run agent loops reliably. Knowing where tools end and orchestration begins is the key.
A single agent run answers one question; a conversation needs memory, and a good user experience needs the answer to appear as it's generated. Pydantic AI handles both through its message system — the record of what was said that you pass between runs — and streaming, which delivers typed output progressively. Together they turn one-shot agents into conversational, responsive ones.
A single run answers one question; a conversation needs memory, and good UX needs the answer to appear as it's generated. Pydantic AI handles both through its message system — the record you pass between runs — and streaming that delivers typed output progressively.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make the orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control that complex applications need.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control complex applications need.
A crew is only a real team if its members remember what happened and can hand work to each other — and CrewAI's memory and delegation features are what turn a set of independent agents into something that actually collaborates.
A crew is only a real team if its members remember what happened and can hand work to each other — memory and delegation are what turn a set of independent agents into something that actually collaborates.
A compiled DSPy program is an artifact — you optimize it once, save it, and serve it — which turns the framework's compile step into an ordinary part of a build pipeline rather than something that runs on every request.
A compiled DSPy program is an artifact — optimize once, save, and serve — which turns compilation into a build step, and makes a model swap a recompile rather than a prompt rewrite.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers — behind standard interfaces. The retriever, in particular, is just another Runnable, so RAG becomes a chain like any other.
Answering questions over your own data is the most common LLM application, and LangChain gives you the whole pipeline as composable, swappable components — loaders, splitters, embeddings, vector stores, retrievers. The retriever is just another Runnable.
Dependency injection is the least flashy Pydantic AI feature and quietly one of the most important — it's what lets your agents reach real databases, API clients, and user context without hard-wiring them, and it's the single biggest reason Pydantic AI agents are so testable. Borrowed straight from how good backend frameworks work, applied to agents.
Dependency injection is the least flashy Pydantic AI feature and quietly one of the most important — it lets agents reach real databases and clients without hard-wiring them, and it's the single biggest reason Pydantic AI agents are so testable.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG, and it's built into LlamaIndex.
The moment you expose a query engine as a tool, RAG stops being a fixed pipeline and becomes a decision: the agent decides whether to retrieve, from which source, and whether one search was enough. That is agentic RAG.
Crews give agents autonomy, which is powerful and unpredictable; Flows give you back deterministic control — an event-driven engine where you decide exactly what runs when, with crews slotted in only where autonomy is actually wanted.
Crews give agents autonomy, which is powerful and unpredictable; Flows give you back deterministic control — an event-driven engine where you decide exactly what runs when, with crews slotted in only where autonomy is wanted.
The two systems everyone builds — retrieval-augmented generation and tool-using agents — are where DSPy's declarative approach pays off most, because they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
RAG and tool-using agents are where DSPy's declarative approach pays off most — they are exactly the multi-step pipelines whose prompts are hardest to tune by hand.
Strands's agent loop is deliberately small: a prompt goes in, the model decides, tools run if needed, results feed back, and it repeats until the model is done. What makes it distinctive isn't the loop's shape — every agent has one — but that Strands exposes it plainly and lets the model drive it, with only three ingredients you provide.
Strands's agent loop is deliberately small: a prompt goes in, the model decides, tools run if needed, results feed back, and it repeats until the model is done. Three ingredients you provide, and a loop the model drives.
A "chain" is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small, standard components into pipelines rather than writing monolithic prompt-and-parse code. Thinking in chains is thinking in composable steps, which is what makes LangChain applications modular, testable, and maintainable.
A chain is just Runnables composed with LCEL — but the word names the central idea of LangChain: build applications by wiring small standard components into pipelines rather than writing monolithic prompt-and-parse code.
An agent that can only talk is a chatbot; an agent that can act needs tools. In Pydantic AI, a tool is just a typed Python function you decorate — the framework reads its type hints to tell the model how to call it, validates the model's arguments, and runs it. Function calling stops being schema-wrangling and becomes writing ordinary typed functions.
An agent that can only talk is a chatbot; one that can act needs tools. In Pydantic AI, a tool is just a typed Python function you decorate — the framework reads its type hints to tell the model how to call it, validates arguments, and runs it.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling memory well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
A query engine answers one question in isolation; a chat engine holds a conversation. The difference is memory — and handling it well is what separates a demo chatbot from one that stays coherent and affordable over a long dialogue.
An agent without tools can only think and write; tools are what let it act — search the web, query a database, call an API — and turning a Python function into a CrewAI tool is deliberately almost effortless.
An agent without tools can only think and write; tools are what let it act — search the web, query a database, call an API — and turning a Python function into a CrewAI tool is deliberately almost effortless.
This is the payoff of the whole framework: you hand an optimizer your program, your metric, and some examples, and it searches for the instructions and demonstrations that make the program measurably better — the prompts you never had to write.
The payoff of the framework: hand an optimizer your program, metric, and examples, and it searches for the instructions and demonstrations that make the program measurably better — the prompts you never had to write.
An MCP server can run code and see context on the model's behalf, which makes it powerful and dangerous in equal measure — this is how to deploy one without handing attackers the keys.
An MCP server can run code and see context on the model's behalf. Authentication, prompt-injection and tool-poisoning risks, human-in-the-loop, sandboxing, and a production checklist.
The single idea that defines smolagents is that an agent's action is a snippet of Python, not a JSON blob. It sounds like a minor encoding detail and turns out to change what an agent can do in a single step — because code carries logic, loops, variables, and composition that structured tool calls simply can't express.
The single idea that defines smolagents is that an agent's action is a snippet of Python, not a JSON blob. It sounds like an encoding detail and turns out to change what an agent can do in one step — because code carries logic, loops, and composition JSON can't.
The model-driven approach is not just how Strands works — it's a stance on where intelligence should live in an agent. Put it in the model's reasoning, not in developer-authored control flow. This post unpacks why that stance is increasingly the right one, and where it isn't.
The model-driven approach is a stance on where intelligence should live in an agent: in the model's reasoning, not in developer-authored control flow. This post unpacks why that stance is increasingly right, and where it isn't.
The pipe operator that lets you write `prompt | model | parser` is not syntactic sugar — it's LangChain's core composition model, and everything you pipe together shares one standard interface that gives you streaming, batching, and async for free. Understanding Runnables and LCEL is understanding how LangChain applications are actually built.
The pipe operator that lets you write prompt | model | parser is not syntactic sugar — it's LangChain's core composition model, and everything you pipe together shares one interface that gives you streaming, batching, and async for free.
This is the feature Pydantic AI is named for and built around: you declare a Pydantic model as your agent's output type, and you get back a validated instance of it — not a string to parse, not JSON to hope about, but a real typed object. It turns the single most brittle part of LLM applications into the most reliable.
This is the feature Pydantic AI is named for: you declare a Pydantic model as your agent's output type and get back a validated instance — not a string to parse. It turns the most brittle part of LLM applications into the most reliable.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them (postprocessing, response synthesis) are where you tune quality.
A retriever finds the relevant Nodes; a query engine turns those Nodes into a grounded answer. Together they are the RAG pipeline — and the seams between them are where you tune quality.
Agents and tasks are the pieces; the crew is what assembles them into a working team, and its process — sequential or hierarchical — decides whether they run like an assembly line or a delegating manager.
The crew assembles agents and tasks into a working team, and its process — sequential or hierarchical — decides whether they run like an assembly line or a delegating manager.
DSPy's optimizer improves whatever your metric rewards, which makes the metric the single most consequential thing you write — get it right and compilation makes your program better; get it wrong and it optimizes confidently toward the wrong target.
DSPy's optimizer improves whatever your metric rewards, which makes the metric the single most consequential thing you write — get it right and compilation makes your program better; get it wrong and it optimizes toward the wrong target.
A server is only half the story; the client is what connects to it, discovers its capabilities, and turns a model's intent into real tool calls.
Build an MCP client that launches a server, discovers its tools, and drives them from a language model — the core of what every MCP host does internally.
Most agent frameworks have the model call tools by emitting JSON. smolagents, Hugging Face's deliberately tiny library, makes the model write Python code instead — "agents that think in code." That one design choice, plus a ruthless commitment to minimalism, is what the whole library is about, and it turns out to matter more than it sounds.
Most agent frameworks have the model call tools by emitting JSON. smolagents, Hugging Face's deliberately tiny library, makes the model write Python code instead — 'agents that think in code.' That one choice, plus ruthless minimalism, is the whole library.
Most agent frameworks ask you to design the workflow — the steps, the branches, the orchestration. Strands Agents, AWS's open-source SDK, makes the opposite bet: give the model a goal and tools, and let it drive. That model-driven philosophy is the whole point, and understanding it is understanding why Strands feels different from everything else.
Most agent frameworks ask you to design the workflow. Strands Agents, AWS's open-source SDK, makes the opposite bet: give the model a goal and tools, and let it drive. That model-driven philosophy is the whole point.
Every LangChain application, no matter how elaborate, is built from three humble pieces: a model you call, a prompt you send it, and a parser that turns its reply into something usable. Master these three and the rest of LangChain is just composing them — which is exactly what the framework is designed to let you do.
Every LangChain application, no matter how elaborate, is built from three humble pieces: a model you call, a prompt you send it, and a parser that turns its reply into something usable. Master these three and the rest is just composing them.
The Agent is where everything in Pydantic AI comes together — model, instructions, tools, typed dependencies, and typed output, bundled into one reusable, testable object you define once and run many times. Understanding the Agent as a configured, type-parameterized unit is the key that makes the rest of the framework fall into place.
The Agent is where everything in Pydantic AI comes together — model, instructions, tools, typed dependencies, and typed output, bundled into one reusable, testable object you define once and run many times.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which organizes your data for which query pattern is the point.
An index is the data structure that makes your Nodes findable, and for RAG that almost always means embeddings in a vector store — but LlamaIndex offers more than one index type, and knowing which fits which query pattern is the point.
An agent is a capability; a task is the assignment — and the two fields that define a task, its description and its expected output, are where you turn "a smart agent" into "the specific result I need."
An agent is a capability; a task is the assignment — and the two fields that define a task, its description and its expected output, are where you turn 'a smart agent' into 'the specific result I need.'
Real tasks are pipelines, not single calls, and in DSPy you build them the way you build a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Real tasks are pipelines, not single calls. In DSPy you build them like a neural network — subclass a base module, declare sub-modules, and wire them together in a forward method.
Everything in the series so far comes together here: a small but complete Model Context Protocol server, in Python, exposing a tool, a resource, and a prompt, runnable and testable in minutes.
Build a complete MCP server in Python with the official SDK — a notes service with a tool, a resource, and a prompt — and test it end to end with the MCP Inspector.
LangChain is the framework everyone starts with and everyone has opinions about — a vast toolkit for building LLM applications whose real value isn't any one feature but the standardization it brings: one interface across every model, vector store, and tool, so you write your application once and swap the pieces underneath. Understanding what it is (and its relationship to LangGraph) cuts through the confusion.
LangChain is the framework everyone starts with and everyone has opinions about — a vast toolkit whose real value is the standardization it brings: one interface across every model, vector store, and tool, so you write your application once and swap the pieces underneath.
Most agent frameworks treat the LLM's output as text you hope to parse. Pydantic AI treats it as typed, validated data — bringing the discipline that made Pydantic the backbone of Python data validation to the messy world of LLM agents. If you've ever wished your agent's output was a real typed object instead of a string you cross your fingers over, this framework was built for you.
Most agent frameworks treat the LLM's output as text you hope to parse. Pydantic AI treats it as typed, validated data — bringing the discipline that made Pydantic the backbone of Python data validation to the messy world of LLM agents.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
Everything LlamaIndex retrieves is a Node, and the quality of your Nodes — how you load your data and how you chunk it — sets the ceiling on everything downstream, no matter how good your model or retriever is.
A CrewAI agent is defined less by code than by three sentences — its role, goal, and backstory — and getting those right is the highest-leverage thing you do, because they are the prompt that shapes everything the agent does.
A CrewAI agent is defined less by code than by three sentences — its role, goal, and backstory — and getting those right is the highest-leverage thing you do, because they are the prompt that shapes everything the agent does.
If a signature says what a step does, a module says how to get the model to do it — and because modules are parameterized, swapping one for another changes the reasoning strategy without touching your intent.
If a signature says what a step does, a module says how to get the model to do it — Predict, ChainOfThought, ReAct — and because modules are parameterized, swapping one changes the strategy without touching your intent.
Tools let a model act, but resources and prompts are how a Model Context Protocol server feeds it the right context and gives users repeatable ways to invoke it.
Beyond tools, MCP servers expose resources (read-only context by URI) and prompts (reusable templates). What each is for, how they appear on the wire, and how to choose.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
LlamaIndex began as the fastest way to build RAG and has grown into a full data framework for LLM applications — connect your data, index it, retrieve it, and reason over it, with agents and workflows on top.
CrewAI takes the most intuitive metaphor for multi-agent AI — a team of specialists with roles collaborating on a job — and makes it the programming model, which is both its great strength and the thing to be disciplined about.
CrewAI takes the most intuitive metaphor for multi-agent AI — a team of specialists with roles collaborating on a job — and makes it the programming model, which is both its strength and the thing to be disciplined about.
A DSPy signature is the contract that says what goes in and what comes out of a step — and by declaring the transformation instead of writing the prompt, it becomes something the optimizer can improve.
A DSPy signature declares what goes in and what comes out of a step — and by declaring the transformation instead of writing the prompt, it becomes something the optimizer can improve.
Tools are the part of the Model Context Protocol that lets a model do things instead of just talk about them, and their design is what separates an agent that helps from one that flails.
Tools are the MCP primitive that lets a model act. How to define them, the tools/list and tools/call methods, results versus errors, and designing tools a model can actually use.
Hand-tuned prompt strings are brittle, model-specific, and impossible to systematically improve — DSPy replaces them with declarative modules and an optimizer that writes the prompts for you.
Hand-tuned prompt strings are brittle, model-specific, and impossible to improve systematically. DSPy replaces them with declarative modules and an optimizer that writes the prompts for you.
The same JSON-RPC messages can travel down a subprocess pipe or across the network, and choosing the right transport is mostly a question of where your server lives and who it serves.
The same MCP messages travel over a subprocess pipe or the network — stdio for local tools, streamable HTTP for remote services. How each works and when to choose it.
Taking a watsonx.ai system from a notebook to production in Python — deployment spaces, reliability with retries and fallbacks, cost and throughput control, observability wired to watsonx.governance, and a hardening checklist.
Run a watsonx system in production from Python: IBM Cloud vs Cloud Pak for Data, project_id vs deployment spaces, reliability (tenacity retries, IAM token refresh, fallback), token-based cost, observability wired to watsonx.governance monitors, and securing IAM credentials.
Taking an NVIDIA-stack LLM system from a working prototype to something you trust in production — reliability, cost and throughput, observability, and security — all from Python, with the OpenAI-compatible surface keeping the code stable whether you burst to the API Catalog or run your own NIM.
Run an NVIDIA-stack LLM system in production from Python: hosted vs self-hosted vs hybrid, reliability (client retries, tenacity backoff, readiness probes, fallback), the GPU-hours cost model, Prometheus observability across the pipeline, and securing nvapi-/NGC keys.
Underneath every tool call and resource read is a small, well-defined conversation in JSON-RPC that begins with a handshake and a negotiation over what each side can do.
Under every MCP tool call is a JSON-RPC conversation that starts with an initialize handshake and a capability negotiation. Here is the wire protocol in detail.
Why enterprises pick watsonx for its governance story, what watsonx.governance actually gives a Python team, and how to wire monitoring, factsheets, and generative-quality metrics into an LLM feature — honestly, as the platform-heavy work it is.
Governance is why many enterprises pick watsonx: model inventory, automated factsheets, monitoring (quality, drift, bias, and generative metrics like faithfulness), explainability, and EU AI Act-aligned risk workflows — with the ibm-watsonx-gov evaluation SDK and what to log.
When to graduate from the hosted API Catalog to inference you run yourself — deploying a NIM container, reaching for Triton and its Python client, the TensorRT-LLM levers that raise throughput and cut latency, and the one base_url change that carries your Python client across unchanged.
Move from the hosted API Catalog to self-hosted inference: run a NIM container (your Python client works unchanged by swapping base_url), use Triton directly via tritonclient, and understand the TensorRT-LLM levers — quantization, in-flight batching, paged KV cache, tensor parallelism.
A model is only as useful as the context and tools it can reach, and MCP is the open standard that lets any AI app plug into any tool through one interface instead of a hundred bespoke integrations.
MCP turns the M×N mess of wiring every AI app to every tool into M+N: wrap each system as a server once, make each app a client once, and any app can use any system.
Adding safety to a watsonx app in Python with two complementary layers — the built-in HAP and PII moderations that watsonx.ai applies to generation, and IBM's dedicated Granite Guardian risk-detection models run as classifiers around your main model to catch harm, jailbreaks, and RAG hallucination before a response reaches a user.
Add safety to a watsonx app in Python with two layers: built-in HAP/PII moderations on generation, and Granite Guardian — IBM's dedicated risk models — run as a classifier to screen input and check RAG groundedness, wrapped in a guarded-generate flow.
Adding a safety layer to an NVIDIA-stack Python app with NeMo Guardrails — and why running it in-process, with no HTTP boundary, is the quiet advantage Python gives you over a separate guardrails server.
Add safety with NeMo Guardrails, in-process in Python: the rail types (input/output/dialog/retrieval), the config.yml + Colang model, loading with RailsConfig/LLMRails backed by a NIM model, and detecting when a rail trips instead of treating a refusal as a normal answer.
Assembling a full retrieval-augmented generation pipeline in Python on watsonx.ai — ingest and chunk documents, embed them with slate, retrieve by cosine, rerank for precision, then generate a grounded, cited answer with a Granite model, shown both from scratch and with langchain-ibm.
Build a full RAG pipeline in Python on watsonx: chunk (with Docling for parsing), embed and store, retrieve, rerank, and generate a grounded, cited answer with a Granite model — shown both from scratch and via the idiomatic langchain-ibm assembly, with notes on scaling to watsonx.data/Milvus.
Assembling a full retrieval-augmented generation pipeline in Python — chunk and embed with NeMo Retriever, shortlist by cosine, sharpen with a reranker, then ground a ChatNVIDIA answer that cites its sources — first from scratch, then the idiomatic LangChain wiring.
Build a full RAG pipeline in Python on NVIDIA's stack: chunk and embed with NeMo Retriever, shortlist by cosine, rerank, then generate a grounded, cited answer with ChatNVIDIA — shown both from scratch and the idiomatic LangChain assembly, with notes on scaling to a vector DB.
Building RAG's retrieval core with watsonx.ai from Python — turning a corpus into vectors with IBM's slate embedding models, scoring a query against them, and then sharpening the shortlist with a reranking model so the LLM gets the right passages, not just plausible ones.
Use watsonx.ai's slate embedding models and reranking from Python to build RAG's retrieval core: embed_documents vs embed_query, numpy cosine scoring, and a two-stage retrieve-then-rerank pipeline — plus the langchain-ibm WatsonxEmbeddings/WatsonxRerank path.
Building RAG's retrieval core in Python — turning a corpus and a query into vectors with NeMo Retriever embedding NIMs, scoring by cosine similarity, then sharpening the shortlist with a cross-encoder reranker NIM.
Use NeMo Retriever from Python to build RAG's retrieval core: NVIDIAEmbeddings (embed_documents vs embed_query for the asymmetric passage/query distinction) with cosine scoring, and NVIDIARerank.compress_documents for a two-stage retrieve-then-rerank pipeline.
Letting a Granite model on watsonx.ai invoke your Python functions — the full request-and-response loop with the first-party `ibm-watsonx-ai` chat API, plus the shorter LangChain path with `ChatWatsonx.bind_tools`.
Function/tool calling against watsonx.ai Granite models in Python: the chat() round-trip loop (json.loads the arguments, echo tool_call_id, cap iterations), handling multiple tool calls, and the LangChain ChatWatsonx.bind_tools path — Granite models are tool-trained.
Driving function calling against NVIDIA NIM models from Python — the full request-execute-respond loop with the plain `openai` client, then the same thing automated by `ChatNVIDIA.bind_tools`.
Function/tool calling against NIM models in Python: the openai round-trip loop (json.loads the arguments string, echo tool_call_id, cap the iterations) and the LangChain ChatNVIDIA.bind_tools path — with the honest caveat that model support varies.
Your first real inference calls with the ibm-watsonx-ai SDK — credentials, ModelInference, the generate and chat APIs, streaming, token usage, error handling, and the langchain-ibm path.
Make your first watsonx.ai calls in Python: credentials (IAM key + project_id + regional url), ModelInference with generate_text vs the modern chat API, decoding methods and parameters, streaming, token usage, and the langchain-ibm ChatWatsonx path.
Your first real NVIDIA NIM calls in Python, two idiomatic ways: the plain OpenAI SDK pointed at NVIDIA's endpoint, and the official LangChain integration — with error handling, streaming, and the one change that makes the same code run against a self-hosted model.
Make your first NIM calls in Python two ways: the standard openai client pointed at the API Catalog with an nvapi- key, and the official ChatNVIDIA from langchain-nvidia-ai-endpoints — with streaming, token usage, typed errors, and the base_url swap to a self-hosted NIM.
From the smallest possible agent to a browsable service — the core loop, the four ways to run it, how memory and tools attach, and two ways to put a server in front of it.
From the smallest possible agent to a browsable service — the core loop, the four ways to run it, how memory and tools attach, and two ways to put a server in front of it.
How to wrap an agent run to log, guard, retry, redact, and secure it — using middleware seams that sit entirely outside the agent's own logic.
How to wrap an agent run to log, guard, retry, redact, and secure it — using middleware seams that sit entirely outside the agent's own logic.
A complete guide to what an agent remembers — from a single conversation held in a session, to durable facts injected on every run, to the per-request values that reach a tool without ever touching the model's schema.
A complete guide to what an agent remembers — from a single conversation held in a session, to durable facts injected on every run, to the per-request values that reach a tool without ever...
From a single decorated async function to an explicit graph of executors and agent nodes — the core workflow model in Microsoft Agent Framework, and the two APIs that express it.
From a single decorated async function to an explicit graph of executors and agent nodes — the core workflow model in Microsoft Agent Framework, and the two APIs that express it.
Once you can wire a chain of executors, the graph earns its keep: concurrency, durable state, composition, and control — the patterns that turn a toy pipeline into a system that survives a crash.
Once you can wire a chain of executors, the graph earns its keep: concurrency, durable state, composition, and control — the patterns that turn a toy pipeline into a system that survives a crash.
How to pause a workflow for a human decision, package a whole workflow as an agent, and see exactly what a run did — through OpenTelemetry spans and a rendered graph — in Microsoft Agent Framework.
How to pause a workflow for a human decision, package a whole workflow as an agent, and see exactly what a run did — through OpenTelemetry spans and a rendered graph — in Microsoft Agent Framework.
A complete guide to coordinating many agents — from a fixed pipeline, to parallel fan-out, to a self-routing mesh, to a planner that decides who acts next, to publishing an agent as a network service other agents can call.
A complete guide to coordinating many agents — from a fixed pipeline, to parallel fan-out, to a self-routing mesh, to a planner that decides who acts next, to publishing an agent as a network...
A complete guide to where a Microsoft Agent Framework agent gets its model — from direct Foundry inference to OpenAI-compatible endpoints, service-managed agents, hand-rolled providers, and container hosting.
A complete guide to where a Microsoft Agent Framework agent gets its model — from direct Foundry inference to OpenAI-compatible endpoints, service-managed agents, hand-rolled providers, and...
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide answers all seven.
Once an agent can call tools, the next questions are what it can read, what it returns, how long it can run, where its facts come from, how it's defined, and whether it actually works — this guide...
A guide to the two hosting concerns every agent eventually hits — seeing it run in a local chat window with a live call inspector, and keeping its state alive across crashes on Durable Task infrastructure.
A guide to the two hosting concerns every agent eventually hits — seeing it run in a local chat window with a live call inspector, and keeping its state alive across crashes on Durable Task...
A Python engineer's map of IBM watsonx — what watsonx.ai, watsonx.governance, watsonx.data and watsonx Orchestrate actually are, why enterprises pick them, and the smallest amount of `ibm-watsonx-ai` code that gets a foundation model answering you.
The opener to a series on building LLM and agent applications on IBM watsonx from Python: how watsonx.ai (Granite + third-party models), watsonx.governance, watsonx.data and Orchestrate fit together, and why the ibm-watsonx-ai SDK and langchain-ibm make it Python-native.
A Python engineer's map of NVIDIA's AI platform — NIM inference, NeMo Retriever, NeMo Guardrails, and Triton with TensorRT-LLM — and why the same code runs against the hosted API Catalog or your own self-hosted microservice.
The opener to a series on building LLM and agent applications on NVIDIA's AI stack from Python: how NIM, the API Catalog, NeMo Retriever, NeMo Guardrails, Triton and TensorRT-LLM fit together — and why the openai client and langchain-nvidia-ai-endpoints make it Python-native.
A complete guide to giving a Microsoft Agent Framework agent the ability to act — from a plain Python function the model can call, to provider-hosted sandboxes, remote MCP servers, and higher-level packaging patterns like Skills and CodeAct.
A complete guide to giving a Microsoft Agent Framework agent the ability to act: function tools, provider-hosted tools (code interpreter, file search, web search), local and hosted MCP, plus Skills and CodeAct — with the code and gotchas for each.
Passing an SDK client through a checkpointed agent workflow crashed on circular references. The fix reshaped how I cross @step boundaries.
Threading an SDK client through a checkpointed workflow crashed with maximum recursion depth exceeded. The checkpoint boundary is a serialization boundary: JSON only, values and keys, and anything non-serializable gets injected out-of-band at the composition root.
How ADK's config loader turns a declarative YAML file into a fully-built agent — and why treating an agent as data changes who gets to edit it.
Defining an agent declaratively in YAML and loading it via from_config — the loader reads, resolves, and validates the config into a built agent, so config-as-data works without writing code.
How caching a large, stable prompt prefix cuts latency and cost — and the ADK config that decides when it pays off.
Context caching cuts latency and cost by caching large, stable context — system prompt, reference docs, tool definitions — so repeated calls don't re-send and re-process the same tokens.
The reference capstone — each term in the series, defined in plain English and grouped by what it does.
The capstone of the series: every LangGraph concept defined in one place — the graph model, state and reducers, persistence, human-in-the-loop, agents and tools, parallelism, streaming, and the surrounding ecosystem.
Post 23 of 26 in "Google ADK, Concept by Concept" — how a planner turns one-shot answers into inspectable plan-then-act reasoning.
Structuring an agent's reasoning: planners that make the model plan-then-act (ReAct-style), the built-in thinking feature, and how a planner improves multi-step tool use over naive prompting.
The higher-level building blocks LangGraph stacks on top of the graph engine — pausing for a human, running an agent loop, calling tools, and fanning out dynamically.
The building blocks on top of the core graph: interrupt() to pause for human input, create_react_agent and ToolNode for tool-using agents, and the Send API for dynamic parallel fan-out with a reducer fan-in.
How ADK closes the write-code, run-it, read-the-output loop — and why "unsafe" is a warning, not a typo.
Letting an agent write and run code: built-in and container-based code executors, safe sandboxed execution, how results flow back into the conversation, and the security tradeoffs.
How a checkpointer turns a graph run into something you can stop, reload, and replay from any point in its history.
A checkpointer saves state at every superstep boundary, so a run can pause, resume on a thread_id, and even fork from an earlier checkpoint (time-travel). This is the foundation human-in-the-loop is built on.
How ADK skills bundle instructions, tools, and resources into folders an agent can browse and load on demand.
Skills package reusable capabilities — instructions, tools, resources, including file-based skills — so they can be discovered and attached to agents, promoting reuse across projects.
Watching a LangGraph run happen — the three things `.stream()` can show you, and why they fall out of the superstep model for free.
stream() exposes a run in three modes: values (full state after each node), updates (what each node changed), and debug (the raw event stream). Streaming falls out naturally from the superstep model.
cross-cutting concerns registered once on the Runner instead of copied onto every agent
Plugins are cross-cutting hooks that apply globally across every agent, tool, and runner — logging, policy, metrics, caching — as opposed to per-agent callbacks. When a plugin beats a callback.
How a single `Command` object folds a state update and a routing decision together — and the tiny lowering that makes `goto` just another guarded edge.
Command lets a node return a state update and a goto in one object, moving the routing decision inside the node. It is the cleanest way to express supervisor handoffs and dynamic control flow.
How ADK's model abstraction lets you swap Gemini for Claude, GPT, or Ollama without touching a line of agent code
ADK is model-agnostic: use Gemini natively or plug in Claude, GPT, or Ollama via LiteLLM and a model registry, swapping the model without changing agent code.
The single most important pattern in LangGraph — a branch plus a back-edge, and the `recursion_limit` that keeps it from running forever.
Branching plus a back-edge is a cycle, and that cycle IS the agent loop: model proposes tool calls, tools run, control returns to the model, repeat until done. Plus recursion_limit, the guardrail that stops a runaway loop.
Post 18 of 26 in "Google ADK, Concept by Concept" — retrieval tools, grounding metadata, rendering citations, and the retrieve→augment→generate loop.
Grounding answers in real data: retrieval tools, grounding metadata returned with responses, rendering citations from that metadata, and the retrieve-augment-generate RAG pattern in ADK.
How one router function plus a `path_map` dict lowers to exactly one edge firing per step.
A conditional edge is a router function plus a path map: the router reads state and returns a key, the path map turns that key into the next node. This is how branching (and, next post, loops) are expressed.
Two open protocols that let an agent reach outside its own process — one to borrow tools, one to call other agents as peers.
Two interoperability protocols: MCP lets an agent consume tools from external servers, and A2A — HTTP for agents — lets one agent discover and call another remote agent as a peer over HTTP.
How the four smallest pieces of the LangGraph API turn a bag of nodes into a program you can run.
Edges, START, END, compile() and invoke() are the four smallest pieces that turn a bag of nodes into a runnable program. Here is the full lifecycle of a tiny two-node graph.
Stack the guardrails — callbacks, model filters, restricted tools, and clean-room sandboxing — so that if one layer misses, the next one catches
Layered defense-in-depth for agents: input/output guardrails via callbacks, Gemini safety settings, restricting tools, and sandboxing untrusted actions.
A LangGraph node is just a function — it reads the whole state and returns only the channels it changed.
A node is just a function: it receives the whole current state and returns only the channels it changed. Understand the partial-update contract and the immutable-snapshot guarantee that makes supersteps safe.
How OpenTelemetry traces, structured logs, and token metrics turn an agent's event stream into something you can debug in production.
Seeing inside a running agent: OpenTelemetry tracing with spans for agent, model, and tool steps, structured logging, and exporting traces to debug latency and tool-call trajectories.
The one idea that makes everything else in LangGraph click: nodes don't pass messages, they update a shared state — and reducers decide how.
State is a typed dict of channels; each channel has an optional reducer. No reducer overwrites; a reducer (like add_messages or operator.add) combines. This is the single idea the rest of LangGraph is built on.
How `adk deploy` builds, pushes, and ships an agent in a single step — and the ack-after-invocation rule that keeps event-driven agents reliable.
Deploying an agent: adk deploy with its cloud_run and agent_engine subcommands, containerizing the app, and reliability rules like ack-after-invocation so failures are redelivered, not dropped.
The foundational mental model — why "the graph" is a Pregel program, and how shared state differs from message passing.
LangGraph is shared-state, not message-passing, and both models descend from Google's Pregel/BSP: work advances in supersteps that end at a synchronization barrier. Get this mental model first and the whole API stops being magic.
How ADK turns "did the agent behave correctly?" into a number you can gate a merge on.
Measuring agent quality: eval sets, scoring both the trajectory (right tools, right order) and the final response, criteria configs, and the adk eval CLI — a Python-first workflow today.
token streaming, the accumulate-and-reconcile consumer pattern, and full-duplex live streaming for voice
Consuming output as it is produced: partial events and token streaming, the accumulate-and-reconcile consumer pattern, and bidi/live streaming for voice and interactive UIs.
How an agent actually runs — a Runner drives an invocation and hands you back a stream of events, not a single answer.
How ADK runs an agent: the Runner drives an invocation that yields a stream of Event objects — content, tool calls, state deltas, control signals. The event loop explains streaming, callbacks, and state.
before/after the agent, model, and tool steps — and the single short-circuit rule that turns them into guardrails
Callbacks are lifecycle hooks around the agent, model, and tool steps — before/after each — used for guardrails (short-circuit by returning a response), logging, and mutating requests and responses.
ReadonlyContext, CallbackContext, ToolContext, InvocationContext — and why the read-only vs mutable distinction is a feature, not a limitation.
The context objects ADK passes into tools and callbacks — InvocationContext, ToolContext, CallbackContext, ReadonlyContext — what each exposes and why the read-only vs mutable split matters.
Turning agents into a service you can run and expose, then a full DocQA app that ties the whole series together.
Host Microsoft Agent Framework agents with DevUI, A2A, MCP, and AG-UI, then build DocQA — a grounded, cited multi-agent app that ties the whole Python series together.
Session state is for small text and JSON. When your agent produces a PNG, a PDF, or a WAV, it belongs in the artifact store — binary-native, versioned, and out of the session record.
Artifacts are binary/file data agents produce or consume: ArtifactService saves and versions named artifacts, loaded and saved via context, keeping large blobs out of session state.
Durable workflows in Python: checkpoint and resume, pause for a human with request_info, and package a workflow as an agent.
Durable Microsoft Agent Framework workflows in Python: checkpoint and resume every superstep, suspend on request_info for a human decision, and package a workflow as an agent.
How Python decides what a name means — the LEGB lookup rule, what a closure actually captures, the late-binding loop trap and its fixes, and when you genuinely need `nonlocal` or `global`.
How Python decides what a name means — the LEGB lookup rule, what a closure actually captures, the late-binding loop trap and its fixes, and when you genuinely need `nonlocal` or `global`.
State remembers things inside one chat; Memory is the searchable archive that lets an agent recall what you told it weeks ago.
Memory is long-term recall across sessions, distinct from per-session state: MemoryService stores and retrieves prior context, surfaced via a recall tool so an agent remembers a user over time.
Five prebuilt multi-agent shapes — Sequential, Concurrent, Group Chat, Handoff, Magentic — and when each beats hand-wiring a graph.
Sequential, Concurrent, Group Chat, Handoff, Magentic — the five prebuilt Microsoft Agent Framework orchestrations in Python and when each beats hand-wiring a graph.
A working guide to Python's function signatures — positional and keyword arguments, default values and the mutable-default trap, arbitrary-argument packing and call-site unpacking, keyword-only and positional-only parameters, and treating functions as first-class values.
A working guide to Python's function signatures — positional and keyword arguments, default values and the mutable-default trap, packing with *args/**kwargs, unpacking at call sites, keyword-only and...
A `Session` is the conversation; `state` is the key-value bag agents and tools read and write — and the prefix on a key decides how long it lives.
A Session holds a conversation; state is a scoped key-value store (session/user/app/temp) read and written by tools, injected into instructions via {state} templating, and persisted by SessionService.
Agents as graph nodes: switch-case routing, fan-out/fan-in, and mixing plain functions with agent steps in one workflow.
Agents are just workflow executors: switch-case routing, fan-out/fan-in concurrency, and mixing plain function nodes with agent nodes in one graph.
How Python decides what runs next — conditionals and the ternary, for-each iteration done idiomatically, while loops, break/continue and the surprising loop-else, structural pattern matching with match/case, and where truthiness and comprehensions fit in.
How Python decides what runs next — conditionals and the ternary, for-each iteration done idiomatically, while loops, break/continue and the surprising loop-else, structural pattern matching with...
How a plain function becomes a callable tool, how ToolContext reaches session state, and how long-running tools pause a run for a human.
Tools give agents capabilities: a plain function becomes a tool with its signature as the schema, plus ToolContext, built-in tools, and long-running/human-in-the-loop tools across Python and Go.
The graph model underneath every multi-agent app: executors as nodes, edges as data flow, and typed events streaming out as it runs.
The Microsoft Agent Framework workflow model in Python: executors as nodes, edges as data flow, switch-case routing, and typed streaming events - learned model-free.
Why Python draws a hard line between text and raw bytes, how the encode/decode boundary works, and the string-handling habits that keep that line from cutting you.
Why Python draws a hard line between text and raw bytes, how the encode/decode boundary works, and the string-handling habits that keep that line from cutting you.
How one agent routes work to specialists — and why the description field is the most important string you write.
Agent hierarchies and LLM-driven delegation: sub_agents, how the description field drives auto-transfer, and coordinator/dispatcher patterns — contrasted with deterministic workflow agents.
Turn agent runs into OpenTelemetry spans, block prompt injection with information-flow control, and swap model providers behind one Agent API.
Turn Microsoft Agent Framework agent runs into OpenTelemetry spans, block prompt injection with information-flow control, and swap model providers behind one Agent API.
A working guide to Python's scalar types — arbitrary-precision integers, IEEE-754 floats and the 0.1 + 0.2 trap, when to reach for Decimal and Fraction, the operators that surprise you with negatives, why a boolean is secretly an integer, and how truthiness and None actually work.
A working guide to Python's scalar types — arbitrary-precision integers, IEEE-754 floats and the 0.1 + 0.2 trap, when to reach for Decimal and Fraction, the operators that surprise you with negatives, why a...
When you want fixed control flow, don't ask the model — wire it yourself.
Sequential, Parallel, and Loop agents compose sub-agents in fixed patterns — deterministic orchestration where you, not the model, decide control flow, with state flowing between steps.
Wrapping an agent run with async seams that log, time, guard, and short-circuit — without touching the agent's logic.
Wrapping an Microsoft Agent Framework agent run in Python with async middleware seams — timing, logging, and a guardrail that short-circuits a tool call before it runs.
The single mental model that explains most of Python's surprises — everything is an object with an identity, a type, and a value; a name is a reference bound to an object, not a box that holds one; and assignment binds, it never copies.
The single mental model that explains most of Python's surprises — everything is an object with an identity, a type, and a value; a name is a reference bound to an object, not a box that holds one; and...
description, instruction, generation params, and structured output — the dials on almost every agent you'll build
The four knobs on almost every ADK agent: description (for delegation), instruction with {state} templating, generation params, and structured output — Python Pydantic model vs Go genai.Schema.
Typed results from `response_format`, consuming a stream event by event, and sending an image alongside text.
Three dials on one run() call: typed results via response_format, consuming a stream event by event, and sending an image alongside text.
What actually makes Python distinctive — the design values that shape the language, how it runs, and the everyday tools you'll live in. The first post in a series that treats readability and correctness as features, not afterthoughts.
What actually makes Python *Python* — the design values that shape the language, how it runs, and the everyday tools you'll live in. The first post in a series that treats readability and correctness as...
An `LlmAgent`, a `Runner`, a `Session`, and a CLI that runs it all — the four pieces the other 25 concepts sit on top of.
The smallest ADK agent and the machinery around it: an LlmAgent, the Runner, a Session, and the adk CLI (adk web / adk run) that runs your agent with a dev UI or REPL — no server code.
How a stateless agent remembers: sessions carry one conversation, context providers carry knowledge across all of them.
Microsoft Agent Framework agents are stateless. Sessions carry one conversation; context providers carry memory across all of them. Here is the mental model in real code.
Turn a plain Python function into something the model can call, and watch the tool-call loop close itself.
Turn a plain Python function into a tool the model can call. The @tool decorator, the tool-call loop, multiple tools, and what the model actually sees.
The minimal loop: a Foundry chat client, an Agent with instructions, run non-streaming and streaming — and what actually comes back.
The minimal Microsoft Agent Framework loop in Python: a FoundryChatClient, an Agent whose instructions are its whole personality, run non-streaming and streaming.
Why I learned the whole framework by writing one runnable lesson per concept, against Azure AI Foundry, instead of reading the docs top to bottom.
I learned the whole Microsoft Agent Framework in Python by building one runnable lesson per concept against Azure AI Foundry. Here is the 12-track map.
Single-turn evals check one decision. Multi-turn evals check the whole trajectory. A Python harness with three evaluators, an offline test suite, and the judge prompt that actually works.
AgentSession is short-term memory. MemoryContextProvider + MemoryFileStore is long-term memory. Mem0 is long-term memory for serious workloads. The boundary that matters and how to implement each.
The Microsoft Agent Framework deliberately does not ship an agent registry. Here is why that is the right call, and what to build as a project-local convention when you need one.
Sequential, Concurrent, Handoff, and Custom WorkflowBuilder. Four shapes the Microsoft Agent Framework ships out of the box, when to pick each, and the gotchas that cost me a day.
Microsoft published a 12-chapter reference architecture for multi-agent systems and a separate framework — the Microsoft Agent Framework — to build them. Here is what the 102 Python files actually contain and how they map to the chapters.
We built a small Go + Python service that parses a project's INFORMATION_SCHEMA, asks Gemini to classify each top-spending query against a catalog of anti-patterns, and recommends a rewrite. It is not a magic box; it is a pipeline that cuts the human review time per query from 20 minutes to 90 seconds.