Modern large-scale recommenders — the ones running at the biggest consumer platforms — are built on neural networks. Deep learning didn't replace the core ideas (embeddings, two stages) so much as supercharge them: neural models learn richer embeddings, ingest far more features, and capture complex non-linear patterns that dot products can't. This post covers the two workhorses — two-tower retrieval and neural ranking — that power today's systems.
Modern large-scale recommenders are built on neural networks — not replacing the core ideas (embeddings, two stages) but supercharging them. This post covers the two workhorses: two-tower models for retrieval (the neural evolution of matrix factorization, built for fast ANN search) and rich neural ranking models that score the shortlist with cross-features, sequences, and multiple objectives.
Reinforcement learning was a tidy theory stuck on toy problems until it was combined with deep learning. Replacing the value table with a neural network let RL handle the enormous, continuous state spaces of real problems — and the result, deep RL, is what beat humans at Atari, Go, and StarCraft, and what trains agents in the real world today. This post is about that fusion: why it's powerful, why it's notoriously unstable, and the tricks that make it work.
Replacing the value table with a neural network let RL handle huge, continuous, pixel-level state spaces — the leap that beat humans at Atari and Go. Why naive deep RL diverges, the tricks that tame it (experience replay, target networks), and why it's powerful, brittle, and sample-hungry.
Graph neural networks are genuinely transformative for the right problems and needless complexity for the wrong ones. This closing post surveys where GNNs deliver real value — drug discovery, recommendation, fraud detection, knowledge graphs — and, just as importantly, gives the honest decision framework for when a GNN is worth its cost versus when a simpler approach wins. It turns the series into a judgment you can apply.
GNNs are transformative for the right problems (drug discovery, recommendation, fraud rings, knowledge graphs) and needless complexity for the wrong ones. The honest decision framework: use a GNN when structure carries signal and it beats a structure-ignorant baseline — and reach for something simpler when it doesn't.
Graph neural networks look clean on paper and get messy in production, because training on graphs breaks assumptions that ordinary deep learning takes for granted. The data points aren't independent, the graph may not fit in memory, and the model fails in graph-specific ways. This post covers the practical realities of making GNNs actually work at scale — the problems you'll hit and the techniques that address them.
GNNs look clean on paper and get messy in production because the data points aren't independent. Why mini-batching and train/test splitting break on graphs (leakage through edges), how sampling scales GNNs to huge graphs, and the graph-specific failure modes (over-smoothing, over-squashing, homophily) that shape real systems.
The graph convolutional network treats all of a node's neighbors as (near) equals. But that's often wrong — some neighbors are far more relevant than others, and which ones depends on the context. Graph attention networks fix this by letting the model learn how much to weigh each neighbor, bringing the attention mechanism that revolutionized language models to graphs. This post explains attention on graphs and why it's so often the right upgrade over plain averaging.
The GCN treats all neighbors as equal, but some matter far more and which ones is context-dependent. Graph attention networks learn data-dependent weights so a node listens more to important neighbors — the transformer's attention mechanism adapted to graphs — with interpretability as a bonus. When it helps and the honest trade-offs.
The graph convolutional network has two problems that make it impractical for real systems: it can't handle new nodes, and it needs the whole graph in memory at once. GraphSAGE fixes both with one shift in perspective — learn a reusable function that builds a node's embedding from a sample of its neighbors, rather than learning a fixed embedding per node. This is the idea that made GNNs usable on billion-node industrial graphs, and it's why most production GNNs are GraphSAGE-style.
The GCN can't handle new nodes and needs the whole graph in memory. GraphSAGE fixes both: learn a reusable function that builds an embedding from a sample of neighbors, rather than a fixed per-node vector. Inductive learning and neighbor sampling — the ideas that made GNNs work on billion-node industrial graphs.
The graph convolutional network is the "linear regression" of graph deep learning: the simplest architecture worth knowing, the one that made GNNs practical, and the baseline every other method is compared against. It's message passing with the most natural possible aggregation — average your neighbors — and understanding exactly what it does, and where its simplicity bites, is the foundation for everything more sophisticated.
The GCN is the 'logistic regression' of graph deep learning — message passing with the simplest aggregation (degree-normalized averaging), the strong baseline every method is compared against. Why it shines at semi-supervised node classification (label propagation under homophily), and the limits that motivate GraphSAGE and GAT.
Strip away the variations and every graph neural network does the same fundamental thing: each node repeatedly gathers information from its neighbors and updates itself. This is message passing, and it's the single idea you need to understand GNNs — GCN, GraphSAGE, GAT, and the rest are all just different choices for how a node listens to its neighbors. This post builds message passing from scratch and shows why it elegantly satisfies everything a graph architecture must.
Strip away the variations and every GNN does the same thing: each node repeatedly aggregates information from its neighbors and updates itself. Message passing, why stacking layers widens a node's view to its k-hop neighborhood (convolution generalized to graphs), and the failure modes (over-smoothing, over-squashing) that shape real GNNs.
Before diving into how graph neural networks work, it's worth being precise about what they produce and what problems they solve. The output is embeddings — vectors that place each node (or edge, or whole graph) in a meaningful space — and the problems come in three flavors that cover almost every graph application. Getting this vocabulary straight makes the architectures in the rest of the series click into place.
A GNN produces embeddings that fuse what a node is (features) with where it sits (structure), and graph problems come in three levels: node-level (classification), edge-level (link prediction — recommendation, knowledge-graph completion), and graph-level (molecule classification). Plus the decisive transductive vs inductive distinction.
Neural networks conquered images, text, and audio — but a huge amount of the world's most valuable data isn't any of those. It's graphs: social networks, molecules, knowledge graphs, payment networks, supply chains. Standard deep learning can't handle them, because graphs break the assumptions CNNs and transformers are built on. Graph neural networks are the architecture built for data whose meaning lives in its connections, and this series builds them from first principles.
Neural networks conquered images, text, and audio — but much of the world's most valuable data is graphs: social networks, molecules, knowledge graphs, payment networks. Why standard deep learning can't handle them, the two properties a graph architecture must have, and what a GNN learns instead.
The hardest interpretability frontier is the models that matter most right now: deep neural networks and large language models, with billions of parameters and no obvious place to look for "the reason." The attribution methods that work on tabular models strain here, and a new field — mechanistic interpretability — is trying to reverse-engineer what these networks actually compute. This post covers how we peer inside deep models and LLMs, and why it's both crucial and genuinely unsolved.
The hardest frontier: deep nets and LLMs with billions of parameters and no obvious place to find 'the reason'. Saliency maps (fragile), attention (not explanation), probing, and mechanistic interpretability's attempt to reverse-engineer circuits — plus the honest truth that we largely cannot yet explain why frontier LLMs behave as they do.
Deep learning conquered images and language, and it's made real inroads into forecasting — RNNs, then transformers, then specialized foundation models for time series. But forecasting is also where deep learning's dominance is most contested, because simple classical methods keep holding their own. This post covers the deep-learning approaches to time series, what they're genuinely good for, and the honest, humbling reality of when they actually beat the simpler alternatives.
Deep learning made inroads into forecasting — RNNs/LSTMs, temporal CNNs, transformers, and emerging time-series foundation models — and genuinely shines with many long related series and complex multivariate dynamics. But the humbling reality: for individual modest-length series, classical methods and simple ML often match or beat it. Not a default.
Building a computer-vision system that works in a demo is one thing; building one that works reliably in the real world — at the right speed, cost, and robustness — is another. This closing post assembles the series into practical judgment: how to approach a vision problem, the deployment realities that trip teams up, and where the field is heading. It turns pixels-to-understanding from a concept into something you can actually ship.
Building a vision demo is one thing; shipping a reliable system at the right speed, cost, and robustness is another. A workflow for approaching a vision problem, the deployment realities (efficiency for real-time/edge, robustness to distribution shift, edge cases, monitoring), and where the field is heading — multimodal and foundation models.
Almost nobody trains a computer-vision model from scratch — and if you're trying to, you're probably doing it wrong. The real practice of vision is transfer learning: take a model pretrained on millions of images, adapt it to your task with a fraction of the data. This, plus smart data strategy, is how vision gets built in the real world, and it's the single most practical thing to understand about applying computer vision. This post is about how it's actually done.
Almost nobody trains vision models from scratch — the real practice is transfer learning: adapt a pretrained model with a fraction of the data. Feature extraction vs fine-tuning (chosen by data size), and the data realities (labeling cost, representativeness, augmentation) that actually decide a vision project.
For a decade, convolution was synonymous with computer vision. Then the transformer — the architecture that revolutionized language — was applied to images, and the vision transformer showed that attention could match or beat CNNs given enough data. It's a profound shift: the same architecture now underpins both language and vision, enabling the multimodal models that see and read at once. This post explains how transformers handle images and what their rise means.
The vision transformer treats an image as a sequence of patches and runs a standard transformer, giving global context immediately. The deep trade-off (CNN inductive bias vs ViT learning from data — ViTs win at scale) and the big consequence: one architecture now unifies vision and language, enabling multimodal AI.
Classifying a whole image as "cat" is only the beginning of computer vision. Real applications need to know where the cat is, how many objects there are, and sometimes exactly which pixels belong to each one. Object detection and segmentation answer those harder questions, and they power self-driving cars, medical imaging, and photo editing. This post covers the main vision tasks beyond classification and how they build on the CNN foundation.
Real vision needs more than 'what is this?' — the task hierarchy runs classification → detection (bounding boxes) → semantic segmentation (per-pixel) → instance segmentation. How one-stage (YOLO) vs two-stage detectors and encoder-decoders (U-Net) build on the CNN backbone, powering self-driving and medical imaging.
Once convolution proved it could crack vision, the race was on to build better CNNs — and the story of how they improved is a compact lesson in deep learning itself. From AlexNet to VGG to ResNet, each landmark architecture solved a specific problem that was blocking progress, and the solutions (depth, small filters, skip connections) became foundational ideas used far beyond vision. This post traces that evolution and the principles it taught.
From AlexNet (which launched deep learning) to VGG (small filters, deep) to ResNet (skip connections that made very deep networks trainable), each landmark solved a specific blocker. ResNet's residual connection generalized everywhere — including every transformer block — so the arc is a compact lesson in deep learning itself.
The convolutional neural network is the architecture that finally solved computer vision — and it did so with an idea as elegant as it is powerful: slide a small learnable filter across the image, everywhere, looking for patterns. CNNs automated the feature engineering that classical vision did by hand, learning a hierarchy from edges to objects directly from pixels. This post explains how convolution works and why it was exactly the right inductive bias for images.
The CNN solved vision by learning features instead of hand-designing them. Convolution slides a small learnable filter everywhere — local connectivity and parameter sharing give translation invariance for free — and stacking layers builds a hierarchy from edges to objects. The right inductive bias for images.
Before neural networks, computer vision spent decades building an ingenious toolkit: detect edges, find corners, describe local patches, match features across images. These classical methods are no longer state of the art for recognition, but they're far from obsolete — they still power real systems, they illuminate what deep learning automated, and understanding them makes you a better vision engineer. This post covers how vision worked before deep learning, and why it still matters.
Before neural networks, vision hand-engineered features: edges, corners, invariant descriptors (SIFT). These still power matching, panorama stitching, and 3D/SLAM — and reveal exactly what CNNs later automated. Their ceiling on recognition is what the deep-learning breakthrough overcame.
Humans see so effortlessly that it's easy to forget vision is one of the hardest problems in computing. To a computer an image is just a grid of numbers, and turning those numbers into "that's a cat" requires bridging an enormous gap between raw pixels and meaning. This series builds computer vision from first principles — from pixels to classical methods to the deep learning that finally cracked it — and it starts with why the problem is so deceptively hard.
To a computer an image is just a grid of numbers, and turning those into 'that's a cat' means bridging the semantic gap — made brutal by viewpoint, lighting, scale, occlusion, and intra-class variation. Why vision defeats hand-written rules and is fundamentally a learning problem.