1290 posts · Page 85 of 108. ← Blog
The minimal loop: a Foundry chat client, an Agent with instructions, run non-streaming and streaming — and what actually comes back.
The minimal Microsoft Agent Framework loop in Python: a FoundryChatClient, an Agent whose instructions are its whole personality, run non-streaming and streaming.
Humans see so effortlessly that it's easy to forget vision is one of the hardest problems in computing. To a computer an image is just a grid of numbers, and turning those numbers into "that's a cat" requires bridging an enormous gap between raw pixels and meaning. This series builds computer vision from first principles — from pixels to classical methods to the deep learning that finally cracked it — and it starts with why the problem is so deceptively hard.
To a computer an image is just a grid of numbers, and turning those into 'that's a cat' means bridging the semantic gap — made brutal by viewpoint, lighting, scale, occlusion, and intra-class variation. Why vision defeats hand-written rules and is fundamentally a learning problem.
Video is where multimodal AI gets genuinely hard. It's not just images — it's images over time, plus audio, at a scale that dwarfs a single picture. The temporal dimension adds motion, causality, and continuity that a still frame can't capture, and the sheer data volume strains everything. Video is also the frontier where the most impressive recent generation results have appeared, and where multimodal AI is actively pushing forward. Understanding video — and the other modalities beyond the core ones — shows where the field is heading.
Video is where multimodal AI gets genuinely hard. It's not just images — it's images over time, plus audio, at a scale that dwarfs a single picture. The temporal dimension adds motion, causality, and continuity a still frame can't capture, and it's the frontier where the most impressive recent generation results have appeared.
How a fund turns positions, prices, cash, and accruals into one defensible number every day — and the controls that keep it honest.
How a fund's NAV is struck daily: position keeping, pricing/valuation, accruals, expense amortization, and the striking/publishing pipeline with tolerance checks.
How Stripe and OpenAI turned "buy it for me" into an open standard — product feeds, delegated payment tokens, and OAuth consent.
The Agentic Commerce Protocol co-developed by Stripe and OpenAI that powers Instant Checkout in ChatGPT.
Turn card transaction attributes into interchange, scheme, and markup lines that reconcile to the cent.
Teaches how to build a deterministic fee engine that classifies each transaction into an interchange category (regulated debit, rewards, CPS qualification) and computes interchange + scheme + acquirer markup for merchant statements.
How to turn thousands of gross obligations into the fewest net positions per counterparty — with deterministic cutoff snapshots, netting cycles, and net-debit-cap enforcement.
Teaches how to build a netting engine that collapses many gross obligations into minimal net settlement positions per counterparty, handling netting cycles, cutoff snapshots, and net-debit-cap enforcement.
Modeling the full card dispute lifecycle with reason codes, evidence deadlines, representment, arbitration, and provisional-credit ledger entries at every transition.
Teaches how to model the full card dispute lifecycle as a state machine with reason codes, evidence deadlines, representment, pre-arbitration and arbitration, and provisional-credit ledger entries at each transition.
This lesson teaches how deterministic function executors and agent-backed executors compose in one graph with the same `AddEdge` wiring.
One workflow that mixes deterministic executors with two Foundry agent nodes for jailbreak detection and response, joined by the same AddEdge wiring and TurnToken triggering.
Why I learned the whole framework by writing one runnable lesson per concept, against Azure AI Foundry, instead of reading the docs top to bottom.
I learned the whole Microsoft Agent Framework in Python by building one runnable lesson per concept against Azure AI Foundry. Here is the 12-track map.
Knowing the concepts and the engines is one thing; building a stream-processing pipeline that's correct, reliable, and operable in production is another. This closing post assembles the series into practical judgment: how to design a streaming pipeline, the operational realities that trip teams up, and when to reach for streaming at all. It turns the theory of data-in-motion into something you can actually run.
Designing a streaming pipeline (source, processing, sink) and the operational realities — it runs forever, state is the crux, backpressure and lag, reprocessing, hard testing — plus the honest judgment: use streaming only when latency genuinely justifies its complexity over simpler batch.
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. And the same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural-sounding voice from text. Understanding how AI handles audio — recognition, synthesis, and understanding — completes the picture of the core modalities and shows how general the multimodal recipe has become.
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. The same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural voice from text.