Archive
1290 posts · Page 32 of 108. ← Blog
Of every AI cost lever, one dominates the rest, and all the major vendors agree on it: prompt caching. On long prompts and agentic workloads the reported savings are the largest anywhere — because caching attacks the specific way agent costs explode. If you optimize one thing, optimize this.
Of every AI cost lever, one dominates and all the major vendors agree on it: prompt caching. On agentic workloads the reported savings are the largest anywhere — because caching attacks the specific way agent costs explode. If you optimize one thing, optimize this.
Every LangChain application, no matter how elaborate, is built from three humble pieces: a model you call, a prompt you send it, and a parser that turns its reply into something usable. Master these three and the rest of LangChain is just composing them — which is exactly what the framework is designed to let you do.
Every LangChain application, no matter how elaborate, is built from three humble pieces: a model you call, a prompt you send it, and a parser that turns its reply into something usable. Master these three and the rest is just composing them.
The Agent is where everything in Pydantic AI comes together — model, instructions, tools, typed dependencies, and typed output, bundled into one reusable, testable object you define once and run many times. Understanding the Agent as a configured, type-parameterized unit is the key that makes the rest of the framework fall into place.
The Agent is where everything in Pydantic AI comes together — model, instructions, tools, typed dependencies, and typed output, bundled into one reusable, testable object you define once and run many times.
Knowing your customer is the front door; watching what they do is the rest of the house. AML transaction monitoring is the always-on system that scans activity for signs of money laundering — and it's a fascinating engineering problem precisely because the signal is rare, the cost of missing it is severe, and the cost of over-flagging drowns your investigators.
Knowing your customer is the front door; watching what they do is the rest of the house. AML transaction monitoring scans activity for signs of money laundering — a fascinating problem because the signal is rare, misses are severe, and over-flagging drowns investigators.
IP gets packets to a host but promises nothing about whether they arrive, in order, or intact. The transport layer is where that gap is filled — or deliberately left open. TCP builds a reliable, ordered connection on top of unreliable IP; UDP declines to, trading guarantees for speed. Choosing between them, and understanding TCP's costs, is core backend knowledge.
IP gets packets to a host but promises nothing about whether they arrive, in order, or intact. The transport layer fills that gap — or deliberately leaves it open: TCP builds a reliable ordered connection, UDP declines to and stays fast.
Logs are the oldest and most detailed telemetry — the granular record of what actually happened. But the log line you write for a human to read with grep is nearly useless at scale; the one you write as structured data for a machine to query is the one that saves you at 3 a.m. The shift from text logs to structured logs is the single biggest upgrade most teams can make.
Logs are the most detailed telemetry — the record of what actually happened. But the log line you write for a human to grep is nearly useless at scale; the one you write as structured data for a machine to query is the one that saves you at 3 a.m.
OAuth isn't one procedure — it's a family of flows for different kinds of clients, and picking the wrong one is a security bug, not a style choice. The good news is that modern guidance has collapsed the confusion: for almost every case today, the answer is the authorization code flow with PKCE, and knowing why the alternatives were deprecated is knowing OAuth security.
OAuth isn't one procedure — it's a family of flows for different clients, and picking the wrong one is a security bug. Modern guidance collapsed the confusion: for almost every case, the answer is the authorization code flow with PKCE.
Full fine-tuning updates every weight in a model — billions of numbers — which needs enormous memory and produces a full-size copy per task. LoRA sidesteps all of it with one insight: the change a model needs for a task is "low-rank," so you can train a tiny pair of matrices instead of the whole model. It's the technique that put fine-tuning within reach of anyone with a single GPU.
Full fine-tuning updates every weight — billions of numbers — needing enormous memory. LoRA sidesteps it with one insight: the change a model needs for a task is low-rank, so you train a tiny pair of matrices instead of the whole model.
The most underrated vector index is no index at all. Brute-force search — compare the query to every vector — is the one method with perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's not just adequate but optimal. Knowing when you don't need an ANN index is as valuable as knowing how they work.
The most underrated vector index is no index at all. Brute-force search has perfect recall, zero build time, and no tuning, and for a surprising number of real systems it's optimal — knowing when you don't need ANN is as valuable as knowing how it works.
On the cloud, quantization is an optimization you reach for to cut costs. On a phone, it's the difference between the model running and not running at all. Nearly every on-device LLM you'll ever ship is quantized, because full-precision weights simply don't fit — so understanding the bit-width trade-off is non-negotiable for edge AI.
On the cloud, quantization cuts costs. On a phone, it's the difference between the model running and not running at all — nearly every on-device LLM you'll ship is quantized, so the bit-width trade-off is non-negotiable for edge AI.
A single request leaves an expensive GPU almost entirely idle. Batching is how you fill it — and the leap from naive static batching to continuous batching is the single biggest throughput improvement in modern LLM serving, often several times more requests from the exact same hardware.
A single request leaves an expensive GPU almost entirely idle. Batching fills it — and the leap from static to continuous batching is the single biggest throughput improvement in modern LLM serving.
The buffer pool is where a database spends most of its memory and wins or loses most of its performance. It's a cache of disk pages in RAM, and the difference between a query that hits it and one that misses is the difference between a microsecond and a millisecond — a thousandfold gap that decides whether your database feels fast.
The buffer pool is where a database spends most of its memory and wins or loses most of its performance — a cache of disk pages in RAM, where a hit versus a miss is a thousandfold latency gap.