#Observability

Observability gives engineering teams the ability to understand system behaviour from external outputs alone. Posts here cover OpenTelemetry instrumentation, Prometheus-based SLO validation, Grafana dashboards for agentic AI, and the metrics and traces needed to debug multi-agent systems in production.

44 posts tagged with observability. ← All posts

#A2A (14)#ADK (8)#AG-UI (6)#AI (9)#AI Agents (311)#AI Architecture (10)#AI Cost (10)#AI Cost Optimization (8)#AI Engineering (201)#AI Evaluation (8)#AI Governance (29)#AI Red Teaming (9)#AI Research (9)#AI Safety (8)#AI Security (21)#AI in Production (12)#AML (3)#API Design (10)#API Security (8)#APIs (55)#AWS (17)#Accounting (9)#Agent Skills (3)#Agentic AI (24)#Agentic Commerce (12)#Agentic RAG (8)#Agents (4)#Amazon Bedrock (8)#Analytics (3)#Architecture (36)#Audit (3)#Authentication (11)#Authorization (3)#Automation (6)#Azure (11)#Azure AI Foundry (9)#Backend Engineering (310)#Benchmarks (3)#Best Practices (3)#BigQuery (6)#Business Finance (8)#Business Strategy (55)#C (8)#CI/CD (8)#Caching (10)#Capital Markets (14)#Card Payments (12)#Cards (13)#Career (24)#Checkpointing (4)#Claude Code (8)#Cloud (5)#Cloud Architecture (3)#Cloud Native (10)#Code Review (8)#Collaboration (5)#Communication (9)#Compliance (52)#Computer Networking (9)#Computer Science (32)#Computer Vision (5)#Concurrency (37)#Consulting (3)#Containers (10)#Context Engineering (10)#Cost Optimisation (5)#Credit (14)#Credit Risk (14)#CrewAI (8)#Crypto (12)#Cryptocurrency (12)#Cryptography (8)#Custody (9)#DSPy (8)#Data (13)#Data Engineering (12)#Data Structures (9)#Databases (37)#Deployment (3)#Design Patterns (10)#DevOps (16)#DevSecOps (11)#Developer Experience (5)#Developer Tools (5)#Distributed Systems (94)#Documentation (3)#Edge AI (8)#Embeddings (15)#Emotional Intelligence (8)#Energy (8)#Engineering (11)#Engineering Culture (3)#Engineering Practices (16)#Error Handling (4)#Evaluation (53)#Event-Driven Architecture (8)#FREE-AI (8)#FX (5)#Feedback (4)#FinOps (23)#FinTech (6)#Financial AI (14)#Financial Systems (129)#Fine-Tuning (11)#Fintech (131)#Flutter (8)#Foreign Exchange (5)#Forward Deployed Engineer (8)#Forward Deployed Engineering (8)#Fraud (10)#Function Tools (5)#Functional Programming (3)#Fundraising (8)#GCP (5)#Gemma (4)#Generative AI (3)#Git (8)#Go (220)#Go-to-Market (8)#Google ADK (36)#Governance (56)#Granite (6)#GraphQL (3)#Growth (3)#Guardrails (25)#HIPAA (3)#HTTP (3)#Harness Engineering (8)#Hiring (8)#Hugging Face (8)#Human-in-the-Loop (8)#IBM watsonx (8)#Identity (11)#Integration (3)#Intellectual Property (8)#Interfaces (3)#JavaScript (8)#KYC (11)#KYC and AML (12)#Kafka (10)#Kubernetes (17)#LLM (5)#LLM Inference (8)#LLM-as-Judge (3)#LLMs (135)#LangChain (8)#LangGraph (11)#Leadership (26)#Ledger (12)#Legal (8)#Lending (14)#LlamaIndex (8)#MCP (22)#MLOps (31)#Machine Learning (32)#Marketing (16)#Markets (4)#Memory (15)#Memory Management (5)#Metrics (6)#Microservices (3)#Microsoft Agent Framework (150)#Middleware (6)#Migration (9)#Monitoring (3)#Multi-Agent (10)#Multi-Agent AI (14)#Multi-Agent Systems (73)#Multimodal (3)#Multimodal AI (8)#NIM (5)#NVIDIA (8)#OAuth (3)#OWASP (7)#Observability (44)#On-Device AI (8)#Open Source (7)#OpenTelemetry (5)#Operating Systems (8)#Operations (10)#Opinion (6)#Orchestration (10)#Organizational Design (8)#Payment Rails (16)#Payments (54)#People (8)#Performance (43)#Platform Engineering (9)#PreSales (8)#Privacy (5)#Privacy Engineering (3)#Process (4)#Product (27)#Product Management (8)#Production (8)#Programming (10)#Programming Languages (48)#Prompt Engineering (66)#Prompt Injection (6)#Protocol Buffers (3)#Protocols (9)#Providers (4)#Pydantic AI (8)#Python (142)#Quality (3)#RAG (58)#RBI (3)#REST (5)#Rails (16)#Reasoning Models (8)#Reconciliation (3)#RegTech (8)#Regulation (9)#Reliability (48)#Resilience (4)#Responsible AI (5)#Retrieval (3)#Risk (13)#Rust (32)#SRE (21)#Sales (9)#Scalability (3)#Security (81)#Security Engineering (8)#Self-Evolving Agents (16)#Sessions (3)#Settlement (9)#Soft Skills (8)#Software (3)#Software Architecture (36)#Software Engineering (144)#Spanner (4)#Startups (30)#Strands (8)#Streaming (28)#Structured Output (4)#Sustainability (8)#System Design (32)#Systems Programming (48)#Testing (49)#Tool Use (22)#Tooling (4)#Tools (3)#Trading (8)#Treasury (6)#Type System (3)#Type Systems (11)#TypeScript (8)#Vector Databases (21)#Vector Search (11)#Venture Capital (8)#Version Control (8)#Web Development (6)#Workflows (14)#gRPC (13)#smolagents (8)
Pratik Dhanave · ·6 min read

Observability, SRE, and Reliability on the Platform

A platform that ships fast but falls over isn't a good platform. Reliability is a first-class platform capability — and the smartest move platform engineering makes is to build observability and SRE practices into the platform, so every service gets monitoring, SLOs, and reliability by default instead of each team reinventing them. Reliability becomes a paved road too.

A platform that ships fast but falls over isn't a good platform. Reliability is a first-class platform capability — the smartest move is to build observability and SRE practices into the platform, so every service gets monitoring, SLOs, and reliability by default.

Pratik Dhanave · ·6 min read

Observability and Production

A model-driven agent decides its own path, which means you cannot know what it did without watching — so observability isn't a nice-to-have in Strands, it's a requirement. Built on OpenTelemetry and shaped by AWS's own production use, Strands treats seeing inside the agent as first-class, because a loop you can't see is a loop you can't trust.

A model-driven agent decides its own path, so you cannot know what it did without watching — observability isn't a nice-to-have in Strands, it's a requirement. Built on OpenTelemetry and shaped by AWS's own production use.

Pratik Dhanave · ·6 min read

LangChain in Production

The gap between a LangChain demo and a LangChain production system is the same gap as any LLM application — observability, evaluation, cost, and reliability — and LangChain's answer is LangSmith plus the discipline the rest of your engineering already has. This closing post covers operating LangChain applications and the honest verdict on when to use the framework.

The gap between a LangChain demo and a production system is the same as any LLM application — observability, evaluation, cost, and reliability — and LangChain's answer is LangSmith plus the discipline your engineering already has.

Pratik Dhanave · ·6 min read

Pydantic AI in Production

A framework earns its place not in the demo but in production — under real traffic, real failures, and the need to observe, control cost, and evolve. Pydantic AI's typed, testable design carries into production well, and paired with observability and the model-agnosticism it's had all along, it makes agents you can actually operate. This closing post covers taking a Pydantic AI agent live.

A framework earns its place in production — under real traffic, failures, and the need to observe, control cost, and evolve. Pydantic AI's typed, testable design carries into production well, paired with observability and model-agnosticism.

Pratik Dhanave · ·7 min read

Measurement and Unit Economics

Every lever in this playbook shares one precondition: you have to measure. The vendor percentages are directional signposts, not promises — the only number that governs your bill is the one from your own workload. Measurement, and the right unit of measurement, is what turns a list of tactics into an actual reduction.

Every lever shares one precondition: you have to measure. The vendor percentages are directional signposts, not promises — the only number that governs your bill is the one from your own workload, measured as cost per unit of value.

Pratik Dhanave · ·6 min read

Observability in Practice

Knowing the pillars is not the same as having an observable system. In practice, observability is built incrementally, costs real money you have to manage, and only pays off if the whole team treats telemetry as part of building software — not something added after the outage. This closing post turns the concepts into a way of working.

Knowing the pillars is not the same as having an observable system. In practice, observability is built incrementally, costs real money you must manage, and only pays off if the whole team treats telemetry as part of building software — not something added after the outage.

Pratik Dhanave · ·6 min read

Failure and Resilience

In a distributed system, failure is not an exception to handle — it's the steady state. Nodes are always crashing, recovering, slowing down, and being partitioned somewhere in your cluster. Resilience is not preventing failure; it's designing so that the failures happening right now don't become the outage your users see.

In a distributed system, failure is not an exception to handle — it's the steady state. Resilience is designing so the failures happening right now don't become the outage your users see: detection, safe retries, isolation, and graceful degradation.

Pratik Dhanave · ·4 min read

LlamaIndex in Production

A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe — the work that starts after the demo impresses everyone.

A four-line RAG demo and a production RAG system share almost no operational concerns. Getting to production means treating ingestion as a pipeline, retrieval quality as something you measure, and the whole system as something you observe.

Pratik Dhanave · ·5 min read

Kafka in Production

Kafka's defaults will run; whether they'll survive a broker failure, a traffic spike, or a year of growth depends on a handful of decisions — replication, durability, partitioning, and what you monitor — that are far cheaper to make now than to retrofit later.

Kafka's defaults will run; whether they survive a broker failure, a spike, or a year of growth depends on a handful of decisions — replication, durability, partitioning, and what you monitor.

Pratik Dhanave · ·6 min read

Alerting

An alert that fires when nothing is actually wrong trains people to ignore alerts — and an ignored alert is worse than no alert, because it creates false confidence while the real incident scrolls past unnoticed. Good alerting is ruthlessly selective: page a human only for things that are both urgent and real, alert on what users feel, and treat every noisy alert as a bug to fix.

An alert that fires when nothing is wrong trains people to ignore alerts — and an ignored alert is worse than none. Good alerting is ruthlessly selective: page a human only for things both urgent and real, alert on what users feel, and treat every noisy alert as a bug.

Pratik Dhanave · ·6 min read

SLIs, SLOs, and Error Budgets

"Is the system reliable?" is a useless question until you define reliability as a number. SLOs turn reliability from a vague aspiration into a measurable target, and the error budget — the small amount of unreliability you're allowed — turns it into a tool that settles the eternal fight between shipping features and keeping things stable. This is where telemetry becomes a way to run engineering, not just watch it.

'Is the system reliable?' is useless until you define reliability as a number. SLOs turn reliability into a measurable target, and the error budget — the unreliability you're allowed — settles the eternal fight between shipping features and keeping things stable.

Pratik Dhanave · ·5 min read

Cost Observability and AI FinOps

You cannot manage what you cannot see, and the difference between a team that controls its AI spend and one that is surprised by it is almost always whether they measure cost per feature, per user, and per call.

You cannot manage what you cannot see. The difference between a team that controls its AI spend and one surprised by it is whether they measure cost per feature, per user, and per call.

Pratik Dhanave · ·5 min read

OpenTelemetry

Before OpenTelemetry, instrumenting a system meant picking a vendor and wiring their proprietary agent into all your code — and switching vendors meant re-instrumenting everything. OpenTelemetry ended that: one open, vendor-neutral standard for producing metrics, logs, and traces, so you instrument once and send the data anywhere. It's become the default way to make systems observable.

Before OpenTelemetry, instrumenting meant picking a vendor and wiring their agent into all your code — switching vendors meant re-instrumenting everything. OpenTelemetry ended that: one open, vendor-neutral standard, so you instrument once and send the data anywhere.

Pratik Dhanave · ·5 min read

Observability, Reliability, and Incident Response

You cannot operate what you cannot see, and AI systems fail in ways ordinary monitoring misses — quality silently degrades, cost silently climbs, and inputs silently drift — so observability has to watch the things that don't throw exceptions.

AI fails in ways ordinary monitoring misses — quality degrades, cost climbs, inputs drift, all silently. Phase 8: observability that watches the signals that don't throw exceptions, plus runbooks and a kill-switch.

Pratik Dhanave · ·7 min read

Traces

When a request touches ten services and comes back slow, metrics tell you it's slow and logs tell you what each service did — but neither shows you the one thing you need: where, along that journey, the time actually went. Distributed tracing is the pillar built for exactly this, following a single request across every service it touches and showing you the whole path at once.

When a request touches ten services and comes back slow, metrics say it's slow and logs say what each service did — but neither shows where the time went. Distributed tracing follows a single request across every service and shows the whole path at once.

Pratik Dhanave · ·6 min read

Logs

Logs are the oldest and most detailed telemetry — the granular record of what actually happened. But the log line you write for a human to read with grep is nearly useless at scale; the one you write as structured data for a machine to query is the one that saves you at 3 a.m. The shift from text logs to structured logs is the single biggest upgrade most teams can make.

Logs are the most detailed telemetry — the record of what actually happened. But the log line you write for a human to grep is nearly useless at scale; the one you write as structured data for a machine to query is the one that saves you at 3 a.m.

Pratik Dhanave · ·6 min read

Metrics

Metrics are the cheapest, most efficient telemetry you have — a handful of numbers that summarize millions of events and tell you, at a glance, whether your system is healthy. Their power is aggregation; their trap is cardinality; and knowing which numbers to watch (and how to read percentiles) is the difference between a dashboard that warns you and one that lies to you.

Metrics are the cheapest, most efficient telemetry — a handful of numbers summarizing millions of events. Their power is aggregation; their trap is cardinality; and knowing which numbers to watch (and how to read percentiles) is the difference between a dashboard that warns you and one that lies.

Pratik Dhanave · ·6 min read

What Observability Is

Monitoring tells you whether the things you thought to check are okay. Observability lets you ask questions you never anticipated about a system you can't see inside. In a world of distributed services where failures are novel and emergent, that difference — between watching known dashboards and investigating unknown problems — is the difference between guessing and knowing.

Monitoring tells you whether the things you thought to check are okay. Observability lets you ask questions you never anticipated about a system you can't see inside — the difference between watching known dashboards and investigating unknown problems.

Pratik Dhanave · ·14 min read

Continuous Security: Monitoring, Compliance, and Culture

The DevSecOps series finale — shifting right to runtime, turning compliance into code, closing the incident feedback loop, measuring what matters, and the culture that makes secure the default path.

The capstone: shifting right to continuous security — runtime detection and vulnerability management as a loop, security observability and tamper-evident audit, compliance-as-code with evidence from the pipeline, metrics that matter, and the culture (paved roads, champions).

Pratik Dhanave · ·13 min read

API Lifecycle and Governance

How APIs are designed, shipped, and kept consistent at scale — the design review, the enforceable style guide, spec linting as policy-as-code, contract tests, an API catalog, and the org model that makes ten teams' APIs feel like one.

The capstone: designing and shipping consistent APIs at scale — the lifecycle, design review and style guides, linting the spec as policy-as-code in CI, contract testing so changes don't break consumers, an API catalog with ownership, and production observability.

Pratik Dhanave · ·12 min read

Production on watsonx

Taking a watsonx.ai system from a notebook to production in Python — deployment spaces, reliability with retries and fallbacks, cost and throughput control, observability wired to watsonx.governance, and a hardening checklist.

Run a watsonx system in production from Python: IBM Cloud vs Cloud Pak for Data, project_id vs deployment spaces, reliability (tenacity retries, IAM token refresh, fallback), token-based cost, observability wired to watsonx.governance monitors, and securing IAM credentials.

Pratik Dhanave · ·12 min read

Production on the NVIDIA Stack

Taking an NVIDIA-stack LLM system from a working prototype to something you trust in production — reliability, cost and throughput, observability, and security — all from Python, with the OpenAI-compatible surface keeping the code stable whether you burst to the API Catalog or run your own NIM.

Run an NVIDIA-stack LLM system in production from Python: hosted vs self-hosted vs hybrid, reliability (client retries, tenacity backoff, readiness probes, fallback), the GPU-hours cost model, Prometheus observability across the pipeline, and securing nvapi-/NGC keys.

Pratik Dhanave · ·13 min read

Bedrock in Production: IAM, Cost, and Observability

Taking an Amazon Bedrock Go service from a working prototype to something you can run on-call — least-privilege IAM, credentials without static keys, tuning the SDK's built-in retryer, tracking token cost, and wiring up logging and metrics with aws-sdk-go-v2.

Taking a Bedrock Go service to production: least-privilege IAM and role-based credentials, tuning the SDK's built-in retryer for throttling, token-based cost tracking, and observability via model-invocation logging, structured metrics, and request IDs.

Pratik Dhanave · ·15 min read

Versioning and Evolution: Changing an API Without Breaking Its Consumers

The real skill isn't cutting a v2 — it's knowing which changes are safe to ship silently, which ones break clients you'll never meet, and how to retire an old version responsibly instead of forever.

Changing an API without breaking consumers: what counts as breaking, versioning strategies and their trade-offs (URI vs header vs additive evolution), the robustness principle and its limits, and responsible deprecation with Sunset headers and usage telemetry.

Pratik Dhanave · ·15 min read

Production AI

The last post in the series: what changes when the LLM system you built across posts 1-14 has to run for real — reliability, security, cost, observability, evaluation gates, and versioning, from a Go engineer's seat, with code where it earns its place.

The capstone: running an LLM system in production from a Go engineer's seat — reliability (timeouts, retries, fallbacks), security (injection, least-privilege tools, secrets), cost and observability, CI eval gates, and versioning models and prompts.

Pratik Dhanave · ·5 min read

Deadlines, Metadata, and Interceptors

A production RPC is more than a request and a response. Every call should carry a deadline so it can't hang forever, metadata for cross-cutting concerns like auth and tracing, and it should pass through interceptors that apply logging, authentication, and metrics uniformly. These three mechanisms are how a gRPC system becomes observable, secure, and resilient — and they're the pieces beginners most often skip.

A production RPC is more than a request and response. Every call should carry a deadline so it can't hang forever, metadata for cross-cutting concerns like auth and tracing, and pass through interceptors that apply logging, authentication, and metrics uniformly. These three mechanisms make a gRPC system observable, secure, and resilient.

Pratik Dhanave · ·13 min read

Data Security and Privacy

Part three of the AI Security Engineering series: protecting the data that flows through an LLM system — how sensitive information leaks out of prompts, logs, and retrieval, and the engineering controls (redaction, data minimization, per-user retrieval authz, residency choices) that actually stop it.

Protecting data in LLM systems: sensitive-information disclosure, PII in prompts and logs (your observability can be the leak), redaction with Presidio, data residency, and per-user access control on the retrieval layer so RAG doesn't leak across tenants.

Pratik Dhanave · ·11 min read

Building Eval Datasets from Real Traces

Where good eval cases actually come from — seeding by hand, harvesting from production telemetry, and curating a golden dataset in Go that doesn't rot the moment your prompt changes.

Where good eval cases actually come from — seeding by hand, harvesting from production telemetry, and curating a golden dataset in Go that doesn't rot the moment your prompt changes.

Pratik Dhanave · ·8 min read

Human-in-the-Loop & Workflow Observability in Microsoft Agent Framework (Python)

How to pause a workflow for a human decision, package a whole workflow as an agent, and see exactly what a run did — through OpenTelemetry spans and a rendered graph — in Microsoft Agent Framework.

How to pause a workflow for a human decision, package a whole workflow as an agent, and see exactly what a run did — through OpenTelemetry spans and a rendered graph — in Microsoft Agent Framework.

Pratik Dhanave · ·6 min read

EXHAUSTED Is Not Failure: Bounding Agent Runs

An agent that burns its turn budget without finishing is neither a success nor an error. Give it a distinct outcome and a typed event stream.

The scariest agent failure is the runaway loop that never finishes. Cap every run at max_turns and make EXHAUSTED a first-class outcome, neither a failure nor a silent success, with a typed event stream you can route on.

Pratik Dhanave · ·3 min read

Observability, Safety, and Providers — Microsoft Agent Framework in Go

Wrap every run in an OpenTelemetry span, gate risky tool actions behind a permission handler, and swap model providers behind one agent.Agent.

Wrap every Microsoft Agent Framework run in an OpenTelemetry span, gate risky tool actions behind a permission handler, and swap Anthropic, OpenAI, Gemini, Copilot, and Azure behind one agent.

Pratik Dhanave · ·5 min read

Observability in ADK: Seeing Inside a Running Agent

How OpenTelemetry traces, structured logs, and token metrics turn an agent's event stream into something you can debug in production.

Seeing inside a running agent: OpenTelemetry tracing with spans for agent, model, and tool steps, structured logging, and exporting traces to debug latency and tool-call trajectories.

Pratik Dhanave · ·3 min read

Observability, Safety, and Providers — Microsoft Agent Framework in Python

Turn agent runs into OpenTelemetry spans, block prompt injection with information-flow control, and swap model providers behind one Agent API.

Turn Microsoft Agent Framework agent runs into OpenTelemetry spans, block prompt injection with information-flow control, and swap model providers behind one Agent API.

All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse all posts or explore related topics using the tag cloud above.