Archive
1046 posts · Page 22 of 88. ← Blog
Before you can score an LLM, you have to decide what "good" even means for your task — and that choice determines everything downstream. Metrics fall into a few families, from exact string matching to reference overlap to semantic similarity to task-specific checks, each measuring something different and each with blind spots. Picking the wrong metric is worse than no metric: it gives you a confident number that points the wrong way.
Before you can score an LLM you must decide what "good" means — and that choice determines everything. Metrics fall into families (exact match, reference overlap, semantic similarity, task-specific), each measuring something different with different blind spots. Picking the wrong metric is worse than none: it points confidently the wrong way.
Symmetric encryption has a chicken-and-egg problem: to share a secret key securely, you seem to need a secure channel — which is what you were trying to build. Public-key cryptography is the astonishing idea that broke that loop: two mathematically-linked keys where knowing the public one doesn't reveal the private one. It's the foundation of key exchange, digital signatures, and essentially all secure communication over the open internet.
Symmetric encryption has a chicken-and-egg problem: sharing a secret key securely seems to need a secure channel. Public-key cryptography broke that loop — two mathematically-linked keys where the public one doesn't reveal the private. It's the foundation of key exchange and signatures.
In a regulated business, compliance is not paperwork bolted onto the product — it's a set of hard requirements woven through the code, and treating it as an engineering discipline rather than a legal afterthought is what separates companies that scale safely from ones that get shut down. RegTech is the practice of building compliance in, as software.
In a regulated business, compliance is not paperwork bolted onto the product — it's hard requirements woven through the code, and treating it as an engineering discipline rather than a legal afterthought separates companies that scale safely from ones that get shut down.
Every backend engineer relies on the network constantly and understands it vaguely — until a mysterious timeout, a TLS error, or a latency spike forces a reckoning. The layered model of networking is the map that makes those problems legible: each layer does one job, hides the one below it, and fails in its own characteristic way. Learn the layers and the network stops being magic.
Every backend engineer relies on the network constantly and understands it vaguely — until a mysterious timeout or TLS error forces a reckoning. The layered model is the map that makes those problems legible: learn the layers and the network stops being magic.
Monitoring tells you whether the things you thought to check are okay. Observability lets you ask questions you never anticipated about a system you can't see inside. In a world of distributed services where failures are novel and emergent, that difference — between watching known dashboards and investigating unknown problems — is the difference between guessing and knowing.
Monitoring tells you whether the things you thought to check are okay. Observability lets you ask questions you never anticipated about a system you can't see inside — the difference between watching known dashboards and investigating unknown problems.
Half of all identity confusion — and a surprising share of security bugs — comes from blurring two words that sound alike and mean opposite things. Authentication asks "who are you?"; authorization asks "what are you allowed to do?" Every protocol in this series exists to answer one or the other, and mixing them up is how you build systems that are both insecure and broken.
Half of all identity confusion — and a surprising share of security bugs — comes from blurring two words: authentication asks 'who are you?', authorization asks 'what are you allowed to do?' Every protocol in this series answers one or the other.
Fine-tuning is the most misunderstood tool in the LLM toolkit. Reach for it to teach a model new facts and you'll waste weeks and get worse results than a day of RAG. Reach for it to change how a model behaves — its format, tone, or a narrow skill — and nothing else comes close. The whole discipline starts with knowing which problem you actually have.
Fine-tuning is the most misunderstood tool in the LLM toolkit. Reach for it to teach a model facts and you'll waste weeks; reach for it to change how a model behaves — format, tone, a narrow skill — and nothing else comes close. The discipline starts with knowing which problem you have.
Every RAG system, recommendation engine, and semantic search box rests on one deceptively simple operation: given a query vector, find the closest vectors among millions. Doing it exactly is easy and doesn't scale; doing it fast enough to be useful means giving up exactness on purpose — and understanding that trade is the foundation of vector search.
Every RAG system and semantic search box rests on one operation: given a query vector, find the closest among millions. Doing it exactly is easy and doesn't scale; doing it fast means giving up exactness on purpose — and that trade is the foundation of vector search.
For a decade the default answer to "where does the AI run?" was "someone else's GPU, over the network." On-device AI flips that: the model runs on the phone in the user's hand, and once you internalize what that changes — privacy, offline capability, latency, and cost all at once — a whole class of apps becomes possible that a cloud API can't build.
On-device AI flips the default — the model runs on the phone in the user's hand. Once you internalize what that changes — privacy, offline capability, latency, and cost all at once — a whole class of apps becomes possible that a cloud API can't build.
Running an LLM is not one computation — it's two very different ones stitched together: a compute-heavy pass over your prompt, then a long, memory-bound slog generating one token at a time. Almost every serving optimization that follows makes sense only once you see that inference has these two phases with opposite bottlenecks.
Running an LLM is two very different computations stitched together: a compute-heavy pass over your prompt, then a long, memory-bound slog generating one token at a time. Almost every serving optimization makes sense only once you see these two phases.
A database is not magic — it's a program that turns your rows into bytes on a disk and finds them again quickly, correctly, and without losing them when the power fails. Understanding the machine underneath the SQL is what separates someone who writes queries from someone who knows why they're slow.
A database is a program that turns your rows into bytes on a disk and finds them again quickly, correctly, and without losing them when the power fails. Understanding the machine underneath the SQL is what separates writing queries from knowing why they're slow.
A distributed system is one where a machine you've never heard of failing can stop your program from working. That single property — partial failure — is the root of almost everything that makes this field hard, and pretending it away is the most common and most expensive mistake in backend engineering.
A distributed system is one where a machine you've never heard of failing can stop your program from working. That single property — partial failure — is the root of almost everything that makes the field hard.