Put the pieces together and a modern frontier language model comes into focus: a deep transformer whose feed-forward blocks are Mixtures of Experts, whose attention is made efficient with grouped-query and FlashAttention, and which is engineered end to end around one goal — maximum capability per unit of compute and memory. This closing post assembles the anatomy and looks at where architecture is heading.
Put the pieces together and a modern frontier LLM comes into focus: a deep transformer whose feed-forward blocks are Mixtures of Experts, whose attention is made efficient with grouped-query and FlashAttention, engineered end to end around one goal — maximum capability per unit of compute and memory. This closing post assembles the anatomy and looks at where architecture is heading.
The two costs of long context — quadratic attention compute and linear KV-cache memory — each have a family of solutions, and together they're why modern models can handle context lengths that were impossible a few years ago. Grouped-query attention shrinks the KV cache; FlashAttention computes exact attention far faster; sliding-window and sparse patterns break the quadratic. This post covers the techniques that made long context practical.
The two costs of long context each have a family of solutions, and together they're why modern models handle context lengths that were impossible a few years ago. Grouped-query attention shrinks the KV cache; FlashAttention computes exact attention far faster; sliding-window and sparse patterns break the quadratic. The techniques that made long context practical, and which cost each attacks.
MoE scales a model's parameters cheaply. But there's a second scaling axis that matters just as much for modern LLMs: context length — how much text the model can attend to at once. Attention's cost grows with the square of the sequence, and the memory to run it grows linearly and relentlessly, which is why long context was hard and why so much architectural ingenuity has gone into it.
MoE scales a model's parameters cheaply, but there's a second axis that matters just as much: context length. Attention's compute grows with the square of the sequence, and the KV-cache memory grows linearly and relentlessly — two distinct costs, often confused, that make long context hard. Understanding both is the setup for the efficiency techniques that solved them.
Mixture of Experts saves compute but not memory — and that single fact reshapes everything about how these models are trained and served. All the experts must exist in memory even though only a few run per token, which turns MoE into a distributed-systems problem as much as a machine-learning one. This post covers the trade MoE actually makes and the parallelism it forces.
Mixture of Experts saves compute but not memory — and that single fact reshapes everything about how these models are trained and served. All the experts must exist in memory even though only a few run per token, which turns MoE into a distributed-systems problem as much as a machine-learning one. The trade MoE makes and the expert parallelism it forces.
The router is where Mixture of Experts succeeds or fails. Left to its own devices, it tends to collapse — sending most tokens to a handful of favorite experts while the rest starve, wasting the model's capacity. Making routing spread load evenly, without hurting quality, is the central engineering challenge of MoE, and the techniques for it are what separate a working sparse model from a broken one.
The router is where Mixture of Experts succeeds or fails. Left alone it tends to collapse — sending most tokens to a handful of favorite experts while the rest starve, wasting the model's capacity. Making routing spread load evenly, without hurting quality, is the central engineering challenge of MoE: load-balancing losses, expert capacity, and token dropping.
Now we open up the mechanism at the heart of modern LLMs. A Mixture-of-Experts layer replaces the transformer's single feed-forward block with many parallel "experts" and a router that sends each token to just a few of them. The result is a model with enormous total capacity that spends only a little compute on any given token — the decoupling the first post promised, made concrete.
A Mixture-of-Experts layer replaces the transformer's single feed-forward block with many parallel experts and a router that sends each token to just a few of them. The result is a model with enormous total capacity that spends only a little compute on any given token — the decoupling of capacity from per-token compute, made concrete. Experts, routing, and how outputs combine.
Mixture of Experts is a modification to the transformer, so you can't understand modern LLM architecture without a clear picture of the transformer itself. This post is a focused recap of the parts that matter for the rest of the series: attention as the mechanism that mixes information across tokens, the feed-forward block that MoE replaces, and how the pieces stack into the model everyone is now modifying.
Mixture of Experts is a modification to the transformer, so you can't understand modern LLM architecture without a clear picture of the transformer itself. This post is a focused recap of the parts that matter: attention as the mechanism that mixes information across tokens, the feed-forward block that MoE replaces, and how the pieces stack into the model everyone is now modifying.
The frontier language models of the last few years share a structural secret that isn't obvious from the outside: most of them aren't dense. They're sparse — built from Mixture-of-Experts layers that let a model have hundreds of billions of parameters while only using a fraction of them on any given token. Understanding why models moved from dense to sparse is the key to understanding how modern LLMs are actually built.
Frontier language models share a structural secret that isn't obvious from the outside: most aren't dense. They're sparse — built from Mixture-of-Experts layers that let a model have hundreds of billions of parameters while using only a fraction on any given token. Understanding why models moved from dense to sparse is the key to understanding how modern LLMs are built.
A benchmark score is trustworthy only if the model hasn't seen the answers and no one has optimized directly for the test. Both assumptions fail constantly. Training data contamination inflates scores by rewarding memorization; optimizing for a benchmark turns it from a measure into a target and destroys its meaning. This post is about the ways evaluation gets corrupted — and how to defend against them.
A benchmark score is trustworthy only if the model hasn't seen the answers and no one optimized directly for the test — and both assumptions fail constantly. Training-data contamination inflates scores by rewarding memorization; optimizing for a benchmark turns it from a measure into a target. Here's how evaluation gets corrupted, and how to defend it.
The scores that dominate model announcements — MMLU, GSM8K, HumanEval, and the rest — are benchmarks: standardized public tests that let the whole field compare models on common ground. They've driven enormous progress, but a benchmark is only as good as its design, and a number without understanding of what it measures is easy to misread. Knowing how benchmarks are built, and what makes a good one, is how you read a leaderboard critically instead of credulously.
The scores in model announcements — MMLU, GSM8K, HumanEval — are benchmarks: standardized public tests for comparing models on common ground. They've driven huge progress, but a benchmark is only as good as its design, and a number without understanding is easy to misread. Here's how they're built and what makes a good one.
Knowing your customer is the front door; watching what they do is the rest of the house. AML transaction monitoring is the always-on system that scans activity for signs of money laundering — and it's a fascinating engineering problem precisely because the signal is rare, the cost of missing it is severe, and the cost of over-flagging drowns your investigators.
Knowing your customer is the front door; watching what they do is the rest of the house. AML transaction monitoring scans activity for signs of money laundering — a fascinating problem because the signal is rare, misses are severe, and over-flagging drowns investigators.
Before you can score an LLM, you have to decide what "good" even means for your task — and that choice determines everything downstream. Metrics fall into a few families, from exact string matching to reference overlap to semantic similarity to task-specific checks, each measuring something different and each with blind spots. Picking the wrong metric is worse than no metric: it gives you a confident number that points the wrong way.
Before you can score an LLM you must decide what "good" means — and that choice determines everything. Metrics fall into families (exact match, reference overlap, semantic similarity, task-specific), each measuring something different with different blind spots. Picking the wrong metric is worse than none: it points confidently the wrong way.
Building something with an LLM is easy for a weekend and hard for a year. The wall almost everyone hits is not the model, the prompt, or the framework — it is knowing whether a change made things better or worse. Without a way to measure quality, every improvement is a guess and every deploy is a gamble. Evaluation is the discipline that turns "it seems better" into "it is better, by this much," and it is the real bottleneck in shipping AI systems.
Building with an LLM is easy for a weekend and hard for a year, and the wall almost everyone hits is knowing whether a change made things better or worse. Without measurement every improvement is a guess — evaluation is the discipline that turns "it seems better" into "it is better, by this much," and it's the real bottleneck in shipping AI.
A reasoning model thinks harder on its own — but you can spend test-time compute on top of any model, reasoning or not, to squeeze out more accuracy. Generate many answers and vote. Generate many and pick the best with a verifier. Search a tree of reasoning steps, pruning the bad branches. These techniques share one shape — do more work at inference, then choose well — and understanding them turns test-time compute from a model property into a toolkit you control.
A reasoning model thinks harder on its own — but you can spend test-time compute on top of any model to squeeze out more accuracy. Generate many answers and vote. Generate many and pick the best with a verifier. Search a tree of reasoning steps. These techniques share one shape: do more work, then choose well.
You can't teach deep reasoning by showing a model more examples of good reasoning — because the best reasoning for a hard problem often isn't in any dataset, and imitation caps a model at the quality of what it imitates. The breakthrough behind modern reasoning models was to stop imitating and start rewarding: let the model try to solve problems, check whether it got them right, and reinforce whatever thinking led to correct answers. That shift — from imitation to reinforcement on verifiable outcomes — is why reasoning models can think in ways no one wrote down.
You can't teach deep reasoning by showing more examples — imitation caps a model at what it imitates. The breakthrough was to stop imitating and start rewarding: let the model try to solve problems, check whether it got them right, and reinforce whatever thinking led to correct answers.
The dominant story of AI progress for years was training-time scale: bigger models, more data, more training compute. Test-time compute is a second, independent axis — spend more computation when you run the model, not when you train it, and get better answers on hard problems. It reframes a trained model not as a fixed-capability artifact but as one whose performance you can dial up per query by letting it think more.
The dominant story of AI progress was training-time scale: bigger models, more data. Test-time compute is a second, independent axis — spend more computation when you *run* the model, not when you *train* it, and get better answers on hard problems. It reframes a model as one whose performance you can dial up per query.
For years, the way to make a language model better was to make it bigger — more parameters, more training data. Reasoning models represent a different bet: instead of only scaling what the model knows, scale how much it thinks before answering. A reasoning model spends extra computation at inference time working through a problem step by step before committing to an answer — and on hard problems, that changes what's possible.
For years, making a language model better meant making it bigger. Reasoning models represent a different bet: scale how much the model *thinks* before answering, not just what it knows. A reasoning model spends extra computation working through a problem step by step — and on hard problems, that changes what's possible.
Predicting who will leave is the easy half. The hard half is acting on it: treating the persuadable, respecting a budget, and proving the intervention actually kept anyone.
Predicting attrition and acting on it: churn features and labels, uplift vs propensity (treat the persuadable), and an agentic retention workflow that recommends and executes interventions with guardrails and holdout mea…
How to build next-best-action and product recommendations for financial services without crossing privacy or fair-lending lines.
Building next-best-action and recommendations for financial products: signals and feature stores, candidate generation + ranking, suitability/eligibility constraints, and cold-start — without crossing privacy or fair-len…
How to build a quant model that predicts up-or-down moves without quietly fooling yourself into a backtest that never survives contact with real markets.
Building a quant direction model without fooling yourself: feature engineering from price/volume, walk-forward (not random) cross-validation, avoiding look-ahead and survivorship bias, and why realistic backtests beat im…
When fraud labels are scarce or the attack is brand new, unsupervised anomaly detection buys you a signal before you ever have a training set.
When you have few or no labels: unsupervised anomaly detection.
Why accuracy lies, how to train on a needle-in-a-haystack, and picking a threshold by the money it saves.
Fraud is a needle-in-a-haystack problem.
Why a scorecard trained only on funded loans quietly overstates its own accuracy, and the inference techniques that pull the declined population back into view.
Why credit models trained only on approved-and-funded loans are biased, and the reject-inference techniques (reclassification, parceling, augmentation) that correct for the applicants you never observed.
How lenders turn a model's raw probability into the specific, legally required reasons a declined applicant must receive.
Turning model outputs into legally-required adverse-action reason codes. Global vs local explainability, SHAP values, mapping contributions to reason codes, and fairness checks (disparate impact) that regulators expect.
How WOE binning turns raw variables into monotonic, interpretable predictors on a log-odds scale, and how Information Value ranks which features earn a place in a regulator-defensible scorecard.
How WOE binning turns raw variables into monotonic, interpretable predictors, how Information Value ranks feature predictive power, monotonic binning, handling missing/rare bins, and why this discipline underpins regulat…
How lenders turn application and bureau data into a single, defensible number — and why the interpretable scorecard still wins in regulated lending.
The end-to-end credit-scoring pipeline: target definition (bad = 90+ dpd), feature engineering, WOE transformation, a logistic-regression scorecard, points scaling (PDO), and validation (KS, Gini, AUC).
A technically excellent model is worthless until it changes a number someone cares about. Here is how to design backward from that number.
Why technically excellent models never ship: no clear business metric, organizational buy-in, validation and model-risk sign-off, monitoring, and the prototype-to-production chasm.
Financial ML lives under regulators, adversaries, and money-denominated errors — so you design backward from a business metric, not forward from a model.
What sets financial ML apart from general ML: strict regulatory scrutiny and mandatory explainability, high-stakes and immediately quantifiable errors, messy multi-source data, non-stationarity, and adversarial actors.
Treating fraud, credit, and risk models as governed assets with a lifecycle, an inventory, and an audit trail regulators can actually read.
Development, independent validation, approval, monitoring, and retirement of risk/fraud/credit models under SR 11-7-style governance.
The gap between "multimodal AI exists" and "I built something useful with it" is smaller than it's ever been — you can call a vision-language model, a speech recognizer, or an image generator with an API and a few lines of code. But building well with multimodal AI means knowing what these models can and can't reliably do, how to combine them, and where the pitfalls are. This closing post is the practical one: how to actually build multimodal applications, and where the whole field is heading — toward general, any-to-any models.
The gap between 'multimodal AI exists' and 'I built something useful with it' is smaller than ever — you can call a vision-language model, a speech recognizer, or an image generator with a few lines of code. But building well means knowing what these models can and can't reliably do, how to combine them, and where the pitfalls are. Plus the any-to-any future.
Video is where multimodal AI gets genuinely hard. It's not just images — it's images over time, plus audio, at a scale that dwarfs a single picture. The temporal dimension adds motion, causality, and continuity that a still frame can't capture, and the sheer data volume strains everything. Video is also the frontier where the most impressive recent generation results have appeared, and where multimodal AI is actively pushing forward. Understanding video — and the other modalities beyond the core ones — shows where the field is heading.
Video is where multimodal AI gets genuinely hard. It's not just images — it's images over time, plus audio, at a scale that dwarfs a single picture. The temporal dimension adds motion, causality, and continuity a still frame can't capture, and it's the frontier where the most impressive recent generation results have appeared.
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. And the same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural-sounding voice from text. Understanding how AI handles audio — recognition, synthesis, and understanding — completes the picture of the core modalities and shows how general the multimodal recipe has become.
Sound is the modality that makes AI conversational — the difference between typing to a machine and talking to it. The same architectural ideas that transformed text and vision transformed audio too: treat the waveform as a sequence, train at scale, and one model can transcribe speech across languages, or synthesize a natural voice from text.
Typing a sentence and watching a detailed, coherent, never-before-seen image appear is one of the most striking capabilities in modern AI — and the technique behind it is beautifully counterintuitive. Rather than paint an image stroke by stroke, these models start with pure noise and gradually remove it, step by step, sculpting a picture out of static, guided by your text. Understanding diffusion — and how text steers it — demystifies text-to-image generation and reveals one of the most important generative techniques in AI.
Typing a sentence and watching a detailed, never-before-seen image appear is one of the most striking capabilities in modern AI — and the technique behind it is beautifully counterintuitive. These models start with pure noise and gradually remove it, sculpting a picture out of static, guided by your text. That's diffusion.
The multimodal capability people actually experience — uploading a photo and asking a chatbot what's in it, or having it read a screenshot, explain a diagram, or extract data from a chart — comes from vision-language models: LLMs that can see. The clever part is how it's done. Rather than build a seeing-and-reasoning model from scratch, you take a language model that already reasons brilliantly and give it eyes, by connecting a vision encoder to it. Understanding how that connection works explains the multimodal AI most people use.
The multimodal capability people actually experience — uploading a photo and asking a chatbot what's in it — comes from vision-language models: LLMs that can see. The clever part is how it's done: take a language model that already reasons brilliantly and give it eyes by connecting a vision encoder to it.
The magic trick at the heart of modern multimodal AI is deceptively simple: train an image encoder and a text encoder together so that a picture of a dog and the words "a photo of a dog" land at the same spot in a shared space. Once images and text live in one common representational space, a cascade of capabilities follows — searching images by text, classifying without task-specific training, and grounding language generation in vision. CLIP is the model that made this idea famous, and understanding it is understanding how modalities actually get connected.
The magic trick at the heart of modern multimodal AI is deceptively simple: train an image encoder and a text encoder together so a picture of a dog and the words 'a photo of a dog' land at the same spot in a shared space. Once images and text live in one common space, a cascade of capabilities follows. CLIP is the model that made this famous.
To a computer, an image is just a grid of numbers — millions of pixel values with no inherent meaning. Turning that raw grid into something a model can understand (this is a dog, that's a face, here's text on a sign) is the problem of computer vision, and the way it's solved has changed dramatically. The field moved from hand-crafted feature detectors, to convolutional networks that learn features, to — most recently — the surprising discovery that the transformer architecture behind language models works remarkably well for images too. Understanding how models see is the foundation of the vision side of multimodal AI.
To a computer, an image is just a grid of numbers with no inherent meaning. Turning that raw grid into understanding is computer vision, and the field moved from hand-crafted features, to convolutional networks, to the surprising discovery that the transformer architecture behind language models works remarkably well for images too.
For most of the deep-learning era, an AI model did one thing with one kind of data: this model classifies images, that one translates text, another transcribes speech. Multimodal AI breaks that separation. A single model can now look at an image and describe it, answer questions about a chart, generate a picture from a sentence, or transcribe and reason about audio — because it works across modalities rather than being confined to one. This shift, from single-modality specialists to models that bridge vision, language, audio, and more, is one of the most important developments in modern AI.
For most of deep learning, a model did one thing with one kind of data. Multimodal AI breaks that separation: a single model can look at an image and describe it, generate a picture from a sentence, or transcribe and reason about audio — working across modalities rather than being confined to one. It's one of the most important developments in modern AI.
Balancing supply and demand keeps the grid running moment to moment — but the grid also runs on millions of physical assets (transformers, lines, substations) that age, wear, and occasionally fail, sometimes catastrophically. Keeping the grid reliable over time means anticipating and preventing those failures, not just reacting to them. This is a data problem — reams of sensor readings hinting at trouble before it happens — and it's where AI helps the grid stay reliable: predicting failures, spotting anomalies, and monitoring the vast physical system.
Balancing keeps the grid running moment to moment — but the grid also runs on millions of physical assets that age, wear, and occasionally fail catastrophically. Keeping the grid reliable means anticipating failures, not just reacting. That's a data problem, and it's where AI helps: predicting failures, spotting anomalies, and monitoring the vast physical system.
If the grid must balance supply and demand every instant, and much of both is now uncertain, then everything depends on one thing: seeing the future as clearly as possible. How much electricity will people use in the next hour, the next day? How much will the wind and sun provide? These forecasts drive every operational decision, and improving them — which is fundamentally a machine-learning problem — is one of the highest-leverage places AI helps the grid. Better forecasts mean a grid that balances more efficiently, reliably, and cheaply.
If the grid must balance supply and demand every instant, and much of both is now uncertain, everything depends on seeing the future clearly. How much power will people use? How much will wind and sun provide? These forecasts drive every operational decision, and improving them — fundamentally a machine-learning problem — is one of the highest-leverage places AI helps.
The electrical grid is quietly becoming one of the most complex control problems humanity has ever attempted. For a century it was relatively simple: a few big, controllable power plants supplying predictable demand. Now it's millions of variable renewable sources, distributed generation, electric vehicles, and shifting demand — all of which must be balanced, second by second, or the lights go out. That explosion of complexity is turning grid operation into a data and optimization problem, and it's why AI is becoming essential to keeping the lights on in a decarbonizing world.
The electrical grid is quietly becoming one of the most complex control problems humanity has attempted. For a century it was simple: a few controllable plants supplying predictable demand. Now it's millions of variable renewable sources that must be balanced second by second, or the lights go out. That complexity is turning grid operation into an AI problem.