#Streaming

Articles about Streaming — exploring patterns, best practices, and real-world implementations in production systems.

28 posts tagged with streaming. ← All posts

#A2A (14)#ADK (8)#AG-UI (6)#AI (9)#AI Agents (311)#AI Architecture (10)#AI Cost (10)#AI Cost Optimization (8)#AI Engineering (201)#AI Evaluation (8)#AI Governance (29)#AI Red Teaming (9)#AI Research (9)#AI Safety (8)#AI Security (21)#AI in Production (12)#AML (3)#API Design (10)#API Security (8)#APIs (55)#AWS (17)#Accounting (9)#Agent Skills (3)#Agentic AI (24)#Agentic Commerce (12)#Agentic RAG (8)#Agents (4)#Amazon Bedrock (8)#Analytics (3)#Architecture (36)#Audit (3)#Authentication (11)#Authorization (3)#Automation (6)#Azure (11)#Azure AI Foundry (9)#Backend Engineering (310)#Benchmarks (3)#Best Practices (3)#BigQuery (6)#Business Finance (8)#Business Strategy (55)#C (8)#CI/CD (8)#Caching (10)#Capital Markets (14)#Card Payments (12)#Cards (13)#Career (24)#Checkpointing (4)#Claude Code (8)#Cloud (5)#Cloud Architecture (3)#Cloud Native (10)#Code Review (8)#Collaboration (5)#Communication (9)#Compliance (52)#Computer Networking (9)#Computer Science (32)#Computer Vision (5)#Concurrency (37)#Consulting (3)#Containers (10)#Context Engineering (10)#Cost Optimisation (5)#Credit (14)#Credit Risk (14)#CrewAI (8)#Crypto (12)#Cryptocurrency (12)#Cryptography (8)#Custody (9)#DSPy (8)#Data (13)#Data Engineering (12)#Data Structures (9)#Databases (37)#Deployment (3)#Design Patterns (10)#DevOps (16)#DevSecOps (11)#Developer Experience (5)#Developer Tools (5)#Distributed Systems (94)#Documentation (3)#Edge AI (8)#Embeddings (15)#Emotional Intelligence (8)#Energy (8)#Engineering (11)#Engineering Culture (3)#Engineering Practices (16)#Error Handling (4)#Evaluation (53)#Event-Driven Architecture (8)#FREE-AI (8)#FX (5)#Feedback (4)#FinOps (23)#FinTech (6)#Financial AI (14)#Financial Systems (129)#Fine-Tuning (11)#Fintech (131)#Flutter (8)#Foreign Exchange (5)#Forward Deployed Engineer (8)#Forward Deployed Engineering (8)#Fraud (10)#Function Tools (5)#Functional Programming (3)#Fundraising (8)#GCP (5)#Gemma (4)#Generative AI (3)#Git (8)#Go (220)#Go-to-Market (8)#Google ADK (36)#Governance (56)#Granite (6)#GraphQL (3)#Growth (3)#Guardrails (25)#HIPAA (3)#HTTP (3)#Harness Engineering (8)#Hiring (8)#Hugging Face (8)#Human-in-the-Loop (8)#IBM watsonx (8)#Identity (11)#Integration (3)#Intellectual Property (8)#Interfaces (3)#JavaScript (8)#KYC (11)#KYC and AML (12)#Kafka (10)#Kubernetes (17)#LLM (5)#LLM Inference (8)#LLM-as-Judge (3)#LLMs (135)#LangChain (8)#LangGraph (11)#Leadership (26)#Ledger (12)#Legal (8)#Lending (14)#LlamaIndex (8)#MCP (22)#MLOps (31)#Machine Learning (32)#Marketing (16)#Markets (4)#Memory (15)#Memory Management (5)#Metrics (6)#Microservices (3)#Microsoft Agent Framework (150)#Middleware (6)#Migration (9)#Monitoring (3)#Multi-Agent (10)#Multi-Agent AI (14)#Multi-Agent Systems (73)#Multimodal (3)#Multimodal AI (8)#NIM (5)#NVIDIA (8)#OAuth (3)#OWASP (7)#Observability (44)#On-Device AI (8)#Open Source (7)#OpenTelemetry (5)#Operating Systems (8)#Operations (10)#Opinion (6)#Orchestration (10)#Organizational Design (8)#Payment Rails (16)#Payments (54)#People (8)#Performance (43)#Platform Engineering (9)#PreSales (8)#Privacy (5)#Privacy Engineering (3)#Process (4)#Product (27)#Product Management (8)#Production (8)#Programming (10)#Programming Languages (48)#Prompt Engineering (66)#Prompt Injection (6)#Protocol Buffers (3)#Protocols (9)#Providers (4)#Pydantic AI (8)#Python (142)#Quality (3)#RAG (58)#RBI (3)#REST (5)#Rails (16)#Reasoning Models (8)#Reconciliation (3)#RegTech (8)#Regulation (9)#Reliability (48)#Resilience (4)#Responsible AI (5)#Retrieval (3)#Risk (13)#Rust (32)#SRE (21)#Sales (9)#Scalability (3)#Security (81)#Security Engineering (8)#Self-Evolving Agents (16)#Sessions (3)#Settlement (9)#Soft Skills (8)#Software (3)#Software Architecture (36)#Software Engineering (144)#Spanner (4)#Startups (30)#Strands (8)#Streaming (28)#Structured Output (4)#Sustainability (8)#System Design (32)#Systems Programming (48)#Testing (49)#Tool Use (22)#Tooling (4)#Tools (3)#Trading (8)#Treasury (6)#Type System (3)#Type Systems (11)#TypeScript (8)#Vector Databases (21)#Vector Search (11)#Venture Capital (8)#Version Control (8)#Web Development (6)#Workflows (14)#gRPC (13)#smolagents (8)
Pratik Dhanave · ·6 min read

Messages, History, and Streaming

A single agent run answers one question; a conversation needs memory, and a good user experience needs the answer to appear as it's generated. Pydantic AI handles both through its message system — the record of what was said that you pass between runs — and streaming, which delivers typed output progressively. Together they turn one-shot agents into conversational, responsive ones.

A single run answers one question; a conversation needs memory, and good UX needs the answer to appear as it's generated. Pydantic AI handles both through its message system — the record you pass between runs — and streaming that delivers typed output progressively.

Pratik Dhanave · ·5 min read

Delivery Semantics and Exactly-Once

"Exactly-once" is the most misunderstood phrase in streaming — it is real in Kafka, but only within a specific boundary, and outside that boundary the honest and usually-correct answer is at-least-once plus idempotent consumers.

'Exactly-once' is the most misunderstood phrase in streaming — it's real in Kafka, but only within a specific boundary, and outside it the honest answer is at-least-once plus idempotent consumers.

Pratik Dhanave · ·6 min read

LCEL and Runnables

The pipe operator that lets you write `prompt | model | parser` is not syntactic sugar — it's LangChain's core composition model, and everything you pipe together shares one standard interface that gives you streaming, batching, and async for free. Understanding Runnables and LCEL is understanding how LangChain applications are actually built.

The pipe operator that lets you write prompt | model | parser is not syntactic sugar — it's LangChain's core composition model, and everything you pipe together shares one interface that gives you streaming, batching, and async for free.

Pratik Dhanave · ·5 min read

Streaming and Push Notifications

Long-running agent work needs a way to report progress without the client holding its breath, and A2A offers two: stream the updates live, or register a webhook and get called back.

Long-running agent work needs a way to report progress without the client holding its breath. A2A offers two: stream the updates live over SSE, or register a webhook and get called back.

Pratik Dhanave · ·11 min read

Beyond REST: GraphQL and gRPC

When REST is the wrong shape for the problem, GraphQL and gRPC each fix a different pain — and each buys that fix with a new cost you have to design around.

When REST isn't the right shape: GraphQL (client-selected fields, and its N+1 / caching / complexity costs) and gRPC (Protobuf, HTTP/2, streaming, codegen) — plus a decision framework for REST vs GraphQL vs gRPC.

Pratik Dhanave · ·16 min read

Asynchronous Processing and Messaging

How queues, pub/sub, and log-based streaming let systems stay responsive under load — the delivery semantics, ordering rules, backpressure, and outbox patterns that decide whether async saves you or sinks you.

Decoupling with queues and events: message queues vs pub/sub vs log-based streaming, delivery semantics (why exactly-once delivery is a myth — do idempotent processing), ordering, backpressure, the transactional outbox, and dead-letter queues.

Pratik Dhanave · ·6 min read

Streaming and Backpressure

Streaming is gRPC's most powerful feature and its most misused. Sending a stream of messages sounds simple until one side produces faster than the other can consume — then, without flow control, you get unbounded memory growth and crashes. gRPC inherits HTTP/2's flow control to prevent exactly this. Understanding backpressure is the difference between streaming that scales and streaming that falls over under load.

Streaming is gRPC's most powerful feature and its most misused. Sending a stream sounds simple until one side produces faster than the other consumes — then, without flow control, you get unbounded memory growth and crashes. gRPC inherits HTTP/2's flow control to prevent exactly this. Backpressure is the difference between streaming that scales and streaming that falls over.

Pratik Dhanave · ·13 min read

Bedrock Agents

How to invoke a managed Agent for Amazon Bedrock from Go — where the server owns the reason-act loop, and your job is to call InvokeAgent, range the event stream, accumulate the answer chunks, and read the trace for observability.

Agents for Amazon Bedrock from Go: the managed reason-act loop that runs server-side (vs the DIY Converse loop), invoking an agent alias with InvokeAgent, streaming the response and trace events, and keeping multi-turn state with a SessionId.

Pratik Dhanave · ·5 min read

The Four Types of RPC

gRPC isn't just request-and-response. Because it rides on HTTP/2, it offers four distinct call patterns: unary, server streaming, client streaming, and bidirectional streaming. Each fits a different shape of problem, and choosing the right one is a real design decision — it changes how your API feels, how it performs, and how it handles data that arrives over time rather than all at once.

gRPC isn't just request-and-response. Riding on HTTP/2, it offers four call patterns: unary, server streaming, client streaming, and bidirectional. Each fits a different shape of problem, and choosing the right one is a real design decision — it changes how your API feels, performs, and handles data that arrives over time rather than all at once.

Pratik Dhanave · ·10 min read

Calling watsonx.ai from Python

Your first real inference calls with the ibm-watsonx-ai SDK — credentials, ModelInference, the generate and chat APIs, streaming, token usage, error handling, and the langchain-ibm path.

Make your first watsonx.ai calls in Python: credentials (IAM key + project_id + regional url), ModelInference with generate_text vs the modern chat API, decoding methods and parameters, streaming, token usage, and the langchain-ibm ChatWatsonx path.

Pratik Dhanave · ·10 min read

Calling NIM from Python

Your first real NVIDIA NIM calls in Python, two idiomatic ways: the plain OpenAI SDK pointed at NVIDIA's endpoint, and the official LangChain integration — with error handling, streaming, and the one change that makes the same code run against a self-hosted model.

Make your first NIM calls in Python two ways: the standard openai client pointed at the API Catalog with an nvapi- key, and the official ChatNVIDIA from langchain-nvidia-ai-endpoints — with streaming, token usage, typed errors, and the base_url swap to a self-hosted NIM.

Pratik Dhanave · ·9 min read

Streaming and Token Usage

How to stream Amazon Bedrock responses token-by-token with the aws-sdk-go-v2 Converse API, decode the event stream with a double type-switch, and account for tokens and cost from the metadata event — accurately, in Go.

Streaming responses and accounting for tokens and cost on Bedrock in Go: ranging the ConverseStream event stream, the nested delta unions, checking stream.Err(), and computing cost from the metadata usage event with a formula you fill in.

Pratik Dhanave · ·11 min read

Calling an LLM from Go

Make your first model call from scratch with net/http and encoding/json — the chat/messages API shape, a typed client with a Bearer key and context timeout, robust error handling, and server-sent-event streaming — no framework required.

Make your first model call from scratch with net/http and encoding/json — the chat/messages API shape, a typed client with a Bearer key and context timeout, robust error handling, and server-sent-event streaming.

Pratik Dhanave · ·10 min read

Google ADK Glossary: Every Core Concept in One Place

The reference capstone for the 26-part series — every canonical ADK term, defined concisely.

The capstone of the series: every core ADK concept defined in one place — agents and orchestration, tools, sessions/state/memory, context and callbacks, runtime and streaming, models, grounding, evaluation, protocols, and deployment.

Pratik Dhanave · ·5 min read

Runtime & Events in ADK: The Runner, the Invocation, and the Event Stream

How an agent actually runs — a Runner drives an invocation and hands you back a stream of events, not a single answer.

How ADK runs an agent: the Runner drives an invocation that yields a stream of Event objects — content, tool calls, state deltas, control signals. The event loop explains streaming, callbacks, and state.

Pratik Dhanave · ·7 min read

Designing a Deterministic Order-Book Matching Engine

How price-time priority, a single-threaded sequencer, and gap-free event streaming combine into an exchange core you can replay byte-for-byte.

Teaches how to build a deterministic matching engine: price-time priority order book, limit/market/IOC/FOK order types, single-threaded sequencer for determinism, and gap-free trade/execution event streaming.

Pratik Dhanave · ·3 min read

Workflow Mechanics — Microsoft Agent Framework in Python

The graph model underneath every multi-agent app: executors as nodes, edges as data flow, and typed events streaming out as it runs.

The Microsoft Agent Framework workflow model in Python: executors as nodes, edges as data flow, switch-case routing, and typed streaming events - learned model-free.

Pratik Dhanave · ·7 min read

Batch vs Streaming

How fresh does the data need to be? That one question splits data engineering into two paradigms. Batch processing handles data in large chunks on a schedule — simpler, cheaper, and fine when yesterday's data is good enough. Stream processing handles data continuously as it arrives — more complex and costly, but necessary when you need to know now. Choosing between them (and knowing when each fits) is one of the most consequential architectural decisions in a data platform, and it's driven by real requirements, not by which sounds more impressive.

How fresh does the data need to be? That one question splits data engineering into two paradigms. Batch processing handles data in chunks on a schedule — simpler and cheaper. Stream processing handles data continuously as it arrives — more complex, but necessary when you need to know now.

All posts on this site are written by Pratik Dhanave, an Agentic AI Architect with 7+ years building production distributed systems, multi-agent AI platforms, and cloud-native infrastructure. About the author → Each article includes working code, architecture diagrams, and references to the specific frameworks and standards discussed. Browse all posts or explore related topics using the tag cloud above.