#Metrics

Articles about Metrics — exploring patterns, best practices, and real-world implementations in production systems.

7 posts tagged with metrics. ← Blog home

#A/B Testing (3)#A2A (14)#ADK (8)#AG-UI (6)#AI (9)#AI Adoption (9)#AI Agents (355)#AI Architecture (22)#AI Cost (10)#AI Cost Optimization (8)#AI Engineering (316)#AI Evaluation (9)#AI Financial Engineering (14)#AI Gateway (8)#AI Governance (43)#AI Red Teaming (9)#AI Research (17)#AI Safety (12)#AI Security (35)#AI Strategy (8)#AI in Production (12)#AML (3)#API Design (17)#API Security (9)#APIs (69)#AWS (17)#Accountability (8)#Accounting (9)#Agent Memory (8)#Agent Skills (11)#Agentic AI (25)#Agentic Commerce (12)#Agentic Engineering (12)#Agentic RAG (8)#Agents (4)#Amazon Bedrock (8)#Analytics (3)#Architecture (45)#Attention (3)#Audit (4)#Authentication (11)#Authorization (3)#Automation (13)#Azure (11)#Azure AI Foundry (9)#Backend Engineering (322)#Benchmarks (3)#Best Practices (5)#BigQuery (6)#Billing (8)#Build Systems (4)#Business Finance (8)#Business Strategy (55)#C (8)#CI/CD (23)#Caching (11)#Capital Markets (14)#Card Payments (12)#Cards (13)#Career (30)#Causal Inference (8)#Cellular Automata (8)#Change Management (3)#Checkpointing (4)#Claude Code (8)#Cloud (5)#Cloud Architecture (3)#Cloud Native (10)#Code Review (9)#Coding Agents (20)#Collaboration (6)#Communication (9)#Complexity (8)#Compliance (54)#Computer Networking (9)#Computer Science (40)#Computer Vision (15)#Computer-Use Agents (8)#Concurrency (42)#Consulting (26)#Containers (10)#Context Engineering (19)#Conversational AI (8)#Cost Optimisation (5)#Cost Optimization (3)#Credit (16)#Credit Risk (16)#CrewAI (8)#Crypto (12)#Cryptocurrency (12)#Cryptography (9)#Custody (9)#DSPy (8)#Data (14)#Data Engineering (25)#Data Governance (3)#Data Science (8)#Data Structures (10)#Databases (48)#Deep Learning (20)#Deployment (5)#Design (3)#Design Patterns (10)#DevOps (24)#DevSecOps (22)#Developer Experience (7)#Developer Productivity (15)#Developer Tools (5)#Differential Privacy (3)#Diffusion Models (10)#Discovery (5)#Distributed Systems (104)#Documentation (3)#Edge AI (8)#Embeddings (21)#Emotional Intelligence (8)#Energy (8)#Engineering (14)#Engineering Culture (3)#Engineering Judgment (7)#Engineering Practices (21)#Enterprise AI (8)#Error Handling (4)#Evaluation (79)#Event-Driven (9)#Event-Driven Architecture (8)#Explainability (3)#Explainable AI (8)#FREE-AI (8)#FX (5)#Feature Engineering (3)#Feature Flags (3)#Feedback (4)#FinOps (23)#FinTech (6)#Financial AI (14)#Financial Engineering (8)#Financial Systems (133)#Fine-Tuning (12)#Fintech (134)#Flutter (8)#Forecasting (10)#Foreign Exchange (5)#Forward Deployed Engineer (43)#Forward Deployed Engineering (8)#Fraud (10)#Fraud Detection (8)#Function Tools (5)#Functional Programming (3)#Fundamentals (23)#Fundraising (8)#GCP (5)#GPU (8)#GUI Automation (8)#Gemma (4)#Generative AI (13)#Git (8)#Go (232)#Go-to-Market (31)#Google ADK (36)#Governance (61)#Granite (6)#Graph Neural Networks (8)#GraphQL (3)#Growth (3)#Guardrails (35)#HIPAA (3)#HTTP (3)#Harness Engineering (8)#Hiring (9)#Hugging Face (8)#Human-in-the-Loop (10)#IBM watsonx (8)#Idempotency (3)#Identity (11)#Image Generation (10)#Inference (8)#Information Retrieval (8)#Integration (3)#Integrations (8)#Intellectual Property (8)#Interfaces (3)#Interpretability (8)#JavaScript (8)#KYC (11)#KYC and AML (12)#Kafka (11)#Knowledge Graphs (11)#Kubernetes (17)#LLM (5)#LLM Inference (8)#LLM Infrastructure (8)#LLM-as-Judge (3)#LLMOps (9)#LLMs (197)#LangChain (8)#LangGraph (11)#Leadership (33)#Ledger (12)#Legal (8)#Lending (14)#Linux (9)#LlamaIndex (8)#Load Balancing (3)#MCP (25)#MLOps (39)#Machine Learning (119)#Marketing (16)#Markets (4)#Mathematical Finance (8)#Memory (16)#Memory Management (5)#Metrics (7)#Microservices (3)#Microsoft Agent Framework (150)#Middleware (6)#Migration (9)#Migrations (8)#Mixture of Experts (5)#Model Risk (3)#Monetization (9)#Monitoring (5)#Monorepo (3)#Multi-Agent (10)#Multi-Agent AI (14)#Multi-Agent Systems (76)#Multimodal (3)#Multimodal AI (11)#NIM (5)#NVIDIA (8)#Networking (3)#OAuth (3)#OWASP (7)#Observability (53)#On-Device AI (8)#Ontology (8)#Open Source (7)#OpenAPI (3)#OpenTelemetry (5)#Operating Systems (9)#Operations (14)#Opinion (6)#Orchestration (10)#Organizational Design (8)#Payment Rails (16)#Payments (55)#People (8)#Performance (56)#Personalization (9)#Platform Engineering (17)#Policy as Code (3)#PreSales (8)#Pricing (4)#Privacy (14)#Privacy Engineering (3)#Process (4)#Product (30)#Product Management (9)#Production (22)#Programming (10)#Programming Languages (48)#Prompt Engineering (86)#Prompt Injection (16)#Proof of Concept (3)#Protocol Buffers (3)#Protocols (9)#Providers (4)#Pydantic AI (8)#Python (142)#Quality (3)#Quantitative Finance (22)#RAG (72)#RBI (3)#REST (5)#Rails (16)#Real-Time (8)#Reasoning Models (8)#Recommender Systems (8)#Reconciliation (5)#RegTech (8)#Regulation (10)#Reinforcement Learning (9)#Reliability (68)#Resilience (4)#Responsible AI (5)#Retrieval (3)#Risk (22)#Risk Management (11)#Rust (32)#SLSA (3)#SRE (23)#SaaS (9)#Sales (9)#Sales Engineering (8)#Scalability (6)#Search (10)#Security (103)#Security Engineering (8)#Self-Evolving Agents (24)#Semantic Web (8)#Sessions (3)#Settlement (9)#Soft Skills (8)#Software (3)#Software Architecture (36)#Software Delivery (9)#Software Engineering (152)#Spanner (4)#Specification Engineering (3)#Speech (8)#Startups (30)#Statistics (8)#Strands (8)#Strategy (6)#Stream Processing (9)#Streaming (38)#Structured Output (4)#Supply Chain Security (10)#Sustainability (9)#System Design (32)#Systems Programming (56)#Technical Sales (6)#Testing (79)#Threat Modeling (3)#Time Series (9)#Tool Use (23)#Tooling (11)#Tools (4)#Trading (8)#Transformers (3)#Treasury (6)#Trust (4)#Type System (3)#Type Systems (12)#TypeScript (8)#Validation (3)#Valuation (3)#Vector Databases (23)#Vector Search (12)#Venture Capital (8)#Version Control (9)#Versioning (7)#Voice AI (9)#Web Development (6)#Webhooks (9)#Workflow (4)#Workflows (14)#eBPF (8)#gRPC (13)#smolagents (8)
Pratik Dhanave · ·7 min read

Measuring Go-to-Market

Without measurement, GTM is guessing — you can't tell a channel that works from one that flatters you, a healthy business from one quietly bleeding, or whether your last change helped. But GTM measurement has a trap engineers fall into from the opposite side: drowning in dashboards of vanity metrics that feel rigorous while missing the two or three numbers that actually decide whether the business works. This closing post is about measuring what matters — and the unit economics that separate a real business from an expensive way to lose money.

Without measurement, GTM is guessing — but the engineer's trap is drowning in dashboards of vanity metrics that feel rigorous while missing the two or three numbers that actually decide whether the business works. This is about measuring what matters — and the unit economics that separate a real business from an expensive way to lose money.

Pratik Dhanave · ·5 min read

Evaluating Models — Accuracy Is the Metric That Lies

The most dangerous number in machine learning is accuracy, because it's the one everyone reaches for and the one that most often misleads. A fraud model that's 99.9% accurate by never catching any fraud is useless and common. Choosing the right evaluation metric — and understanding what each one hides — is the difference between a model that looks good and a model that works. This post is about measuring models honestly.

The most dangerous number in ML is accuracy — a 99.9%-accurate fraud model that catches nothing is common. The confusion matrix, precision vs recall (a business decision), F1/ROC, regression metrics, segment-level evaluation (how bias is caught), and why every offline metric is a proxy (Goodhart's law).

Pratik Dhanave · ·6 min read

What to Measure: A Taxonomy of Metrics

Before you can score an LLM, you have to decide what "good" even means for your task — and that choice determines everything downstream. Metrics fall into a few families, from exact string matching to reference overlap to semantic similarity to task-specific checks, each measuring something different and each with blind spots. Picking the wrong metric is worse than no metric: it gives you a confident number that points the wrong way.

Before you can score an LLM you must decide what "good" means — and that choice determines everything. Metrics fall into families (exact match, reference overlap, semantic similarity, task-specific), each measuring something different with different blind spots. Picking the wrong metric is worse than none: it points confidently the wrong way.

Pratik Dhanave · ·12 min read

Measuring and Scoring Red-Team Results

Turning red-team attacks into metrics you can act on and track over time — attack success rate, coverage, severity, and trend — plus the honest limits of what any of those numbers can tell you.

Turning attacks into metrics: attack success rate and why it's subtle, scoring success (rule/classifier/LLM-judge with its biases), coverage across the taxonomy, severity weighting, tracking trends per model/prompt version, and honest reporting of residual risk.

Pratik Dhanave · ·8 min read

SaaS and Recurring-Revenue Metrics

Subscription businesses changed what "revenue" means. When customers pay every month instead of once, a whole new vocabulary appears — MRR, ARR, churn, net revenue retention, the Rule of 40 — and these metrics, not the raw P&L, are how SaaS companies are actually judged. For any engineer working at or evaluating a subscription business (which is most software today), these are the numbers that matter, and the logic behind them explains why SaaS companies behave the way they do.

Subscription businesses changed what 'revenue' means. When customers pay every month instead of once, a whole new vocabulary appears — MRR, ARR, churn, net revenue retention, the Rule of 40 — and these metrics, not the raw P&L, are how SaaS companies are actually judged.

Pratik Dhanave · ·8 min read

Measuring Marketing

"Half the money I spend on advertising is wasted; the trouble is I don't know which half" is a century-old lament that still captures marketing measurement's core problem. Engineers, arriving with a "measure everything" instinct, often assume the answer is just better tracking — and then discover that the most valuable marketing (brand, demand creation, word-of-mouth) is precisely the hardest to measure, while the easiest-to-measure activities aren't always the most valuable. Measuring marketing well means navigating that tension, not pretending it away.

'Half the money I spend on advertising is wasted; I don't know which half' still captures marketing measurement's core problem. Engineers assume the fix is better tracking — then discover the most valuable marketing (brand, demand creation, word-of-mouth) is the hardest to measure. Measuring well means navigating that tension, not pretending it away.

Pratik Dhanave · ·8 min read

Metrics and Data

How do you know if your product is actually working? Not "did we ship the feature" but "did it make the difference we hoped?" Answering that requires metrics — and product management lives in a productive tension here: data is essential for knowing whether you're succeeding, yet the most metric-obsessed teams often build worse products by optimizing the measurable at the expense of the meaningful. Using data well means measuring what matters, letting it inform judgment, and resisting the traps that catch data-driven teams.

How do you know if your product is actually working? Answering that requires metrics — and PM lives in a productive tension: data is essential, yet the most metric-obsessed teams often build worse products by optimizing the measurable at the expense of the meaningful.

All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse the blog or explore related topics using the tag cloud above.