Archive
1046 posts · Page 8 of 88. ← Blog
At some point a model doesn't fit on one GPU, or the traffic doesn't, and you have to spread inference across many. The choices — which kind of parallelism, how to place replicas, when to autoscale — are governed by one unforgiving resource (GPU memory) and one expensive one (inter-GPU communication). Get the memory math right and most scaling decisions follow.
At some point a model doesn't fit on one GPU, or the traffic doesn't. The scaling choices are governed by one unforgiving resource (GPU memory) and one expensive one (inter-GPU communication) — get the memory math right and most decisions follow.
The reason a long analytics query doesn't block every writer in your database — and vice versa — is a single elegant idea: never overwrite data, keep multiple versions, and give each transaction a consistent snapshot in time. MVCC is how nearly every modern database delivers isolation without readers and writers fighting over locks.
The reason a long analytics query doesn't block every writer — and vice versa — is a single elegant idea: never overwrite data, keep multiple versions, and give each transaction a consistent snapshot in time. That's MVCC.
Consensus is the problem of getting a group of unreliable machines to agree on a single value despite crashes, delays, and lost messages. It sounds narrow, but it's the hidden foundation under leader election, distributed locks, configuration, and every "exactly one node is in charge" guarantee. Raft is the algorithm that finally made it understandable.
Consensus is getting unreliable machines to agree on a single value despite crashes and lost messages — the hidden foundation under leader election, distributed locks, and every 'exactly one node is in charge' guarantee. Raft made it understandable.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make the orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control that complex applications need.
An agent's reasoning loop is flexible but opaque and hard to control. A workflow is the opposite: you make orchestration explicit as steps and events, trading some autonomy for the predictability, testability, and control complex applications need.
A crew is only a real team if its members remember what happened and can hand work to each other — and CrewAI's memory and delegation features are what turn a set of independent agents into something that actually collaborates.
A crew is only a real team if its members remember what happened and can hand work to each other — memory and delegation are what turn a set of independent agents into something that actually collaborates.
This is the classic fixed-versus-marginal decision, and it has a clean answer: managed APIs win until your volume is high and steady enough to keep expensive GPUs busy — which is a much higher bar than most teams assume.
The classic fixed-vs-marginal decision with a clean answer: managed APIs win until your volume is high and steady enough to keep expensive GPUs busy — a much higher bar than most teams assume.
Kafka gives you a durable log; these patterns are what you build on it — event sourcing, CQRS, the outbox, sagas, and the choice between choreography and orchestration — the vocabulary of real event-driven systems.
Kafka gives you a durable log; these patterns are what you build on it — event sourcing, CQRS, the outbox, sagas, and the choice between choreography and orchestration.
Every frontier method is a search, and a search is only as good as the evaluator that ranks its candidates — so at the frontier, evaluation stops being a measurement and becomes the single most dangerous component in the system.
Every frontier method is a search, and a search is only as good as the evaluator that ranks its candidates — so at the frontier, evaluation becomes the single most dangerous component in the system.
A compiled DSPy program is an artifact — you optimize it once, save it, and serve it — which turns the framework's compile step into an ordinary part of a build pipeline rather than something that runs on every request.
A compiled DSPy program is an artifact — optimize once, save, and serve — which turns compilation into a build step, and makes a model swap a recompile rather than a prompt rewrite.
A system can be reliable, safe, and responsible and still fail — by being uneconomical — and the final move is turning one production system into a repeatable capability many teams can build safely.
A system can be reliable, safe, and responsible and still fail by being uneconomical. Phases 10-11: known unit economics and spend guardrails, then a paved road that lets many teams ship governed AI safely.
The pieces from this series — routing, query transformation, graded retrieval, multi-hop, and evaluation — assemble into one system that reasons about retrieval as carefully as it reasons about the answer, while spending only as much as each question needs.
Routing, query transformation, graded retrieval, multi-hop, and evaluation assemble into one system that reasons about retrieval as carefully as the answer — while spending only as much as each question needs.
Automated metrics and LLM judges are scalable proxies for the thing that actually matters: whether a human finds the output good. Human evaluation is the ground truth those proxies are calibrated against — and getting it right is its own discipline, full of subtle traps around agreement, bias, and how you ask the question. This post is about doing human eval well, and about how human preference became the signal that trains models themselves.
Automated metrics and LLM judges are scalable proxies for the thing that matters: whether a human finds the output good. Human evaluation is the ground truth those proxies are calibrated against — a discipline full of traps around agreement, bias, and how you ask — and human preference is now the signal that trains models themselves.