#Scalability

Articles about Scalability — exploring patterns, best practices, and real-world implementations in production systems.

6 posts tagged with scalability. ← Blog home

#A/B Testing (3)#A2A (14)#ADK (8)#AG-UI (6)#AI (9)#AI Adoption (9)#AI Agents (355)#AI Architecture (22)#AI Cost (10)#AI Cost Optimization (8)#AI Engineering (316)#AI Evaluation (9)#AI Financial Engineering (14)#AI Gateway (8)#AI Governance (43)#AI Red Teaming (9)#AI Research (17)#AI Safety (12)#AI Security (35)#AI Strategy (8)#AI in Production (12)#AML (3)#API Design (17)#API Security (9)#APIs (69)#AWS (17)#Accountability (8)#Accounting (9)#Agent Memory (8)#Agent Skills (11)#Agentic AI (25)#Agentic Commerce (12)#Agentic Engineering (12)#Agentic RAG (8)#Agents (4)#Amazon Bedrock (8)#Analytics (3)#Architecture (45)#Attention (3)#Audit (4)#Authentication (11)#Authorization (3)#Automation (13)#Azure (11)#Azure AI Foundry (9)#Backend Engineering (322)#Benchmarks (3)#Best Practices (5)#BigQuery (6)#Billing (8)#Build Systems (4)#Business Finance (8)#Business Strategy (55)#C (8)#CI/CD (23)#Caching (11)#Capital Markets (14)#Card Payments (12)#Cards (13)#Career (30)#Causal Inference (8)#Cellular Automata (8)#Change Management (3)#Checkpointing (4)#Claude Code (8)#Cloud (5)#Cloud Architecture (3)#Cloud Native (10)#Code Review (9)#Coding Agents (20)#Collaboration (6)#Communication (9)#Complexity (8)#Compliance (54)#Computer Networking (9)#Computer Science (40)#Computer Vision (15)#Computer-Use Agents (8)#Concurrency (42)#Consulting (26)#Containers (10)#Context Engineering (19)#Conversational AI (8)#Cost Optimisation (5)#Cost Optimization (3)#Credit (16)#Credit Risk (16)#CrewAI (8)#Crypto (12)#Cryptocurrency (12)#Cryptography (9)#Custody (9)#DSPy (8)#Data (14)#Data Engineering (25)#Data Governance (3)#Data Science (8)#Data Structures (10)#Databases (48)#Deep Learning (20)#Deployment (5)#Design (3)#Design Patterns (10)#DevOps (24)#DevSecOps (22)#Developer Experience (7)#Developer Productivity (15)#Developer Tools (5)#Differential Privacy (3)#Diffusion Models (10)#Discovery (5)#Distributed Systems (104)#Documentation (3)#Edge AI (8)#Embeddings (21)#Emotional Intelligence (8)#Energy (8)#Engineering (14)#Engineering Culture (3)#Engineering Judgment (7)#Engineering Practices (21)#Enterprise AI (8)#Error Handling (4)#Evaluation (79)#Event-Driven (9)#Event-Driven Architecture (8)#Explainability (3)#Explainable AI (8)#FREE-AI (8)#FX (5)#Feature Engineering (3)#Feature Flags (3)#Feedback (4)#FinOps (23)#FinTech (6)#Financial AI (14)#Financial Engineering (8)#Financial Systems (133)#Fine-Tuning (12)#Fintech (134)#Flutter (8)#Forecasting (10)#Foreign Exchange (5)#Forward Deployed Engineer (43)#Forward Deployed Engineering (8)#Fraud (10)#Fraud Detection (8)#Function Tools (5)#Functional Programming (3)#Fundamentals (23)#Fundraising (8)#GCP (5)#GPU (8)#GUI Automation (8)#Gemma (4)#Generative AI (13)#Git (8)#Go (232)#Go-to-Market (31)#Google ADK (36)#Governance (61)#Granite (6)#Graph Neural Networks (8)#GraphQL (3)#Growth (3)#Guardrails (35)#HIPAA (3)#HTTP (3)#Harness Engineering (8)#Hiring (9)#Hugging Face (8)#Human-in-the-Loop (10)#IBM watsonx (8)#Idempotency (3)#Identity (11)#Image Generation (10)#Inference (8)#Information Retrieval (8)#Integration (3)#Integrations (8)#Intellectual Property (8)#Interfaces (3)#Interpretability (8)#JavaScript (8)#KYC (11)#KYC and AML (12)#Kafka (11)#Knowledge Graphs (11)#Kubernetes (17)#LLM (5)#LLM Inference (8)#LLM Infrastructure (8)#LLM-as-Judge (3)#LLMOps (9)#LLMs (197)#LangChain (8)#LangGraph (11)#Leadership (33)#Ledger (12)#Legal (8)#Lending (14)#Linux (9)#LlamaIndex (8)#Load Balancing (3)#MCP (25)#MLOps (39)#Machine Learning (119)#Marketing (16)#Markets (4)#Mathematical Finance (8)#Memory (16)#Memory Management (5)#Metrics (7)#Microservices (3)#Microsoft Agent Framework (150)#Middleware (6)#Migration (9)#Migrations (8)#Mixture of Experts (5)#Model Risk (3)#Monetization (9)#Monitoring (5)#Monorepo (3)#Multi-Agent (10)#Multi-Agent AI (14)#Multi-Agent Systems (76)#Multimodal (3)#Multimodal AI (11)#NIM (5)#NVIDIA (8)#Networking (3)#OAuth (3)#OWASP (7)#Observability (53)#On-Device AI (8)#Ontology (8)#Open Source (7)#OpenAPI (3)#OpenTelemetry (5)#Operating Systems (9)#Operations (14)#Opinion (6)#Orchestration (10)#Organizational Design (8)#Payment Rails (16)#Payments (55)#People (8)#Performance (56)#Personalization (9)#Platform Engineering (17)#Policy as Code (3)#PreSales (8)#Pricing (4)#Privacy (14)#Privacy Engineering (3)#Process (4)#Product (30)#Product Management (9)#Production (22)#Programming (10)#Programming Languages (48)#Prompt Engineering (86)#Prompt Injection (16)#Proof of Concept (3)#Protocol Buffers (3)#Protocols (9)#Providers (4)#Pydantic AI (8)#Python (142)#Quality (3)#Quantitative Finance (22)#RAG (72)#RBI (3)#REST (5)#Rails (16)#Real-Time (8)#Reasoning Models (8)#Recommender Systems (8)#Reconciliation (5)#RegTech (8)#Regulation (10)#Reinforcement Learning (9)#Reliability (68)#Resilience (4)#Responsible AI (5)#Retrieval (3)#Risk (22)#Risk Management (11)#Rust (32)#SLSA (3)#SRE (23)#SaaS (9)#Sales (9)#Sales Engineering (8)#Scalability (6)#Search (10)#Security (103)#Security Engineering (8)#Self-Evolving Agents (24)#Semantic Web (8)#Sessions (3)#Settlement (9)#Soft Skills (8)#Software (3)#Software Architecture (36)#Software Delivery (9)#Software Engineering (152)#Spanner (4)#Specification Engineering (3)#Speech (8)#Startups (30)#Statistics (8)#Strands (8)#Strategy (6)#Stream Processing (9)#Streaming (38)#Structured Output (4)#Supply Chain Security (10)#Sustainability (9)#System Design (32)#Systems Programming (56)#Technical Sales (6)#Testing (79)#Threat Modeling (3)#Time Series (9)#Tool Use (23)#Tooling (11)#Tools (4)#Trading (8)#Transformers (3)#Treasury (6)#Trust (4)#Type System (3)#Type Systems (12)#TypeScript (8)#Validation (3)#Valuation (3)#Vector Databases (23)#Vector Search (12)#Venture Capital (8)#Version Control (9)#Versioning (7)#Voice AI (9)#Web Development (6)#Webhooks (9)#Workflow (4)#Workflows (14)#eBPF (8)#gRPC (13)#smolagents (8)
Pratik Dhanave · ·15 min read

Designing a System End to End

The capstone — one problem, a home-timeline feed, designed the whole way through with the method from post one: clarify, estimate, contract, then high-level to deep-dive to bottleneck, naming the trade-off at every step and drawing on all seven earlier posts.

The capstone: one worked design end to end — requirements, estimation, API and data model, high-level architecture, and deep dives applying the whole series (scaling, caching, sharding, consistency, async, reliability) with explicit trade-offs.

Pratik Dhanave · ·15 min read

Scaling Fundamentals

How systems grow under load — vertical vs horizontal scaling, why statelessness is the real enabler, load balancing from L4 to L7, consistent hashing, read/write scaling, the scale cube, and when the honest answer is "don't scale yet."

How systems grow: vertical vs horizontal scaling, statelessness as the enabler of horizontal scale, load balancing (L4/L7, consistent hashing), read/write scaling with replicas — and knowing when not to scale.

Pratik Dhanave · ·12 min read

How to Approach System Design

A repeatable method for designing systems and acing the design interview — clarify requirements, estimate on the back of an envelope, pin down the API and data model, then work high-level to deep-dive to bottleneck, always naming the trade-off.

The opener to a system-design series: a repeatable method rather than a grab-bag of components — clarifying functional vs non-functional requirements, back-of-the-envelope estimation with the latency numbers every engineer should know, and the trade-off-driven design flow.

Pratik Dhanave · ·6 min read

Training and Scaling GNNs — The Practical Realities

Graph neural networks look clean on paper and get messy in production, because training on graphs breaks assumptions that ordinary deep learning takes for granted. The data points aren't independent, the graph may not fit in memory, and the model fails in graph-specific ways. This post covers the practical realities of making GNNs actually work at scale — the problems you'll hit and the techniques that address them.

GNNs look clean on paper and get messy in production because the data points aren't independent. Why mini-batching and train/test splitting break on graphs (leakage through edges), how sampling scales GNNs to huge graphs, and the graph-specific failure modes (over-smoothing, over-squashing, homophily) that shape real systems.

Pratik Dhanave · ·6 min read

GraphSAGE — Inductive Learning and Scaling to Big Graphs

The graph convolutional network has two problems that make it impractical for real systems: it can't handle new nodes, and it needs the whole graph in memory at once. GraphSAGE fixes both with one shift in perspective — learn a reusable function that builds a node's embedding from a sample of its neighbors, rather than learning a fixed embedding per node. This is the idea that made GNNs usable on billion-node industrial graphs, and it's why most production GNNs are GraphSAGE-style.

The GCN can't handle new nodes and needs the whole graph in memory. GraphSAGE fixes both: learn a reusable function that builds an embedding from a sample of neighbors, rather than a fixed per-node vector. Inductive learning and neighbor sampling — the ideas that made GNNs work on billion-node industrial graphs.

Pratik Dhanave · ·7 min read

State and Stateful Stream Processing

Simple stream operations — filter this event, transform that one — need no memory. But the interesting computations — counts, aggregations, joins, windows, pattern detection — require remembering things across events, and that memory is state. Managing state correctly over an unbounded stream, at scale, with fault tolerance, is the hardest engineering problem in stream processing. This post is about state: why it's needed, what makes it hard, and how stream processors manage it.

Interesting stream computations — counts, aggregations, joins, windows, patterns — need to remember across events, and that memory is state. Why state is hard on an unbounded stream (bounding, speed, partitioning, surviving failure), and how engines provide managed keyed state.

All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse the blog or explore related topics using the tag cloud above.