Computer vision from first principles — pixels to understanding, the foundations beneath the Multimodal AI series. Images as data (an image is just a grid of numbers; the semantic gap from pixels to meaning, made brutal by viewpoint/lighting/scale/occlusion/intra-class variation; why vision defeats hand-written rules and is a learning problem), classical computer vision (hand-engineered features — edges, corners, invariant descriptors like SIFT; still powering matching, stitching, and 3D/SLAM; the recognition ceiling that motivated deep learning), convolutional neural networks (convolution — a small learnable filter slid everywhere; local connectivity and parameter sharing giving translation invariance; stacking layers to build an edges→parts→objects hierarchy — the right inductive bias that cracked vision), CNN architectures (AlexNet launching deep learning, VGG's small-filters-deep, ResNet's skip connections making depth trainable and generalizing to transformers), object detection and segmentation (the task hierarchy beyond classification — bounding boxes, per-pixel semantic segmentation, instance segmentation; one-stage vs two-stage detectors, U-Net; powering self-driving and medical imaging), vision transformers (images as patch sequences; the inductive-bias-vs-data trade-off where ViTs win at scale; the unification of vision and language that enables multimodal AI), transfer learning and data (adapting pretrained models with a fraction of the data — feature extraction vs fine-tuning; labeling cost, representativeness, and augmentation as the real determinants), and computer vision in practice (approaching a problem, deployment realities — efficiency for real-time/edge, robustness to distribution shift, edge cases, monitoring — and the multimodal/foundation-model frontier). Grounded in Wikipedia (computer vision, digital image, feature detection, edge detection, convolutional neural network, residual neural network, object detection, image segmentation, vision transformer, transfer learning, data augmentation).
Humans see so effortlessly that it's easy to forget vision is one of the hardest problems in computing. To a computer an image is just a grid of numbers, and turning those numbers into "that's a cat" requires bridging an enormous gap between raw pixels and meaning. This series builds computer vision from first principles — from pixels to classical methods to the deep learning that finally cracked it — and it starts with why the problem is so deceptively hard.
To a computer an image is just a grid of numbers, and turning those into 'that's a cat' means bridging the semantic gap — made brutal by viewpoint, lighting, scale, occlusion, and intra-class variation. Why vision defeats hand-written rules and is fundamentally a learning problem.
Before neural networks, computer vision spent decades building an ingenious toolkit: detect edges, find corners, describe local patches, match features across images. These classical methods are no longer state of the art for recognition, but they're far from obsolete — they still power real systems, they illuminate what deep learning automated, and understanding them makes you a better vision engineer. This post covers how vision worked before deep learning, and why it still matters.
Before neural networks, vision hand-engineered features: edges, corners, invariant descriptors (SIFT). These still power matching, panorama stitching, and 3D/SLAM — and reveal exactly what CNNs later automated. Their ceiling on recognition is what the deep-learning breakthrough overcame.
The convolutional neural network is the architecture that finally solved computer vision — and it did so with an idea as elegant as it is powerful: slide a small learnable filter across the image, everywhere, looking for patterns. CNNs automated the feature engineering that classical vision did by hand, learning a hierarchy from edges to objects directly from pixels. This post explains how convolution works and why it was exactly the right inductive bias for images.
The CNN solved vision by learning features instead of hand-designing them. Convolution slides a small learnable filter everywhere — local connectivity and parameter sharing give translation invariance for free — and stacking layers builds a hierarchy from edges to objects. The right inductive bias for images.
Once convolution proved it could crack vision, the race was on to build better CNNs — and the story of how they improved is a compact lesson in deep learning itself. From AlexNet to VGG to ResNet, each landmark architecture solved a specific problem that was blocking progress, and the solutions (depth, small filters, skip connections) became foundational ideas used far beyond vision. This post traces that evolution and the principles it taught.
From AlexNet (which launched deep learning) to VGG (small filters, deep) to ResNet (skip connections that made very deep networks trainable), each landmark solved a specific blocker. ResNet's residual connection generalized everywhere — including every transformer block — so the arc is a compact lesson in deep learning itself.
Classifying a whole image as "cat" is only the beginning of computer vision. Real applications need to know where the cat is, how many objects there are, and sometimes exactly which pixels belong to each one. Object detection and segmentation answer those harder questions, and they power self-driving cars, medical imaging, and photo editing. This post covers the main vision tasks beyond classification and how they build on the CNN foundation.
Real vision needs more than 'what is this?' — the task hierarchy runs classification → detection (bounding boxes) → semantic segmentation (per-pixel) → instance segmentation. How one-stage (YOLO) vs two-stage detectors and encoder-decoders (U-Net) build on the CNN backbone, powering self-driving and medical imaging.
For a decade, convolution was synonymous with computer vision. Then the transformer — the architecture that revolutionized language — was applied to images, and the vision transformer showed that attention could match or beat CNNs given enough data. It's a profound shift: the same architecture now underpins both language and vision, enabling the multimodal models that see and read at once. This post explains how transformers handle images and what their rise means.
The vision transformer treats an image as a sequence of patches and runs a standard transformer, giving global context immediately. The deep trade-off (CNN inductive bias vs ViT learning from data — ViTs win at scale) and the big consequence: one architecture now unifies vision and language, enabling multimodal AI.
Almost nobody trains a computer-vision model from scratch — and if you're trying to, you're probably doing it wrong. The real practice of vision is transfer learning: take a model pretrained on millions of images, adapt it to your task with a fraction of the data. This, plus smart data strategy, is how vision gets built in the real world, and it's the single most practical thing to understand about applying computer vision. This post is about how it's actually done.
Almost nobody trains vision models from scratch — the real practice is transfer learning: adapt a pretrained model with a fraction of the data. Feature extraction vs fine-tuning (chosen by data size), and the data realities (labeling cost, representativeness, augmentation) that actually decide a vision project.
Building a computer-vision system that works in a demo is one thing; building one that works reliably in the real world — at the right speed, cost, and robustness — is another. This closing post assembles the series into practical judgment: how to approach a vision problem, the deployment realities that trip teams up, and where the field is heading. It turns pixels-to-understanding from a concept into something you can actually ship.
Building a vision demo is one thing; shipping a reliable system at the right speed, cost, and robustness is another. A workflow for approaching a vision problem, the deployment realities (efficiency for real-time/edge, robustness to distribution shift, edge cases, monitoring), and where the field is heading — multimodal and foundation models.
This series is part of a larger body of work by Pratik Dhanave, an Agentic AI Architect writing about production AI systems, distributed systems, and cloud-native engineering. Explore all course series, browse every post, or find topics via the tag index.