Computer Vision in Practice — Shipping Systems That See

Posts 1–7 built the foundations — images as data, classical CV, CNNs, architectures, detection/segmentation, vision transformers, transfer learning. This post is how to use it all to ship. The theme, consistent with the blog’s other “in practice” posts, is that the model is the easy part; the system around it is the job.

Approaching a vision problem

A reliable workflow for any vision task, assembling the series:

1. Define the task precisely (post 5). Is it classification (what), detection (what + where), or segmentation (exact pixels)? Pick the least-detailed task that solves your problem — don’t segment when a classifier suffices; the task determines data cost, model choice, and difficulty.
2. Check whether vision (or deep learning) is even needed. For controlled settings — consistent lighting, fixed camera, simple shapes (barcodes, gauges, presence checks) — classical CV (post 2) may be faster, cheaper, and more reliable than a neural net. Reach for deep learning when the semantic gap is wide (post 1), not reflexively.
3. Use transfer learning from a pretrained model (post 7). Pick a proven architecture (ResNet/ViT backbone, YOLO/U-Net for detection/segmentation) and adapt it — never train from scratch unless you have huge data.
4. Get good, representative, well-labeled data (post 7). This is usually the largest effort and the real determinant of success — invest here, and use heavy data augmentation.
5. Evaluate honestly on a held-out, representative test set — with the right metric for the task (accuracy for classification; mAP for detection; IoU/Dice for segmentation) and, crucially, broken down by the conditions and subgroups the model will face (post 7’s representativeness and fairness point).

This sequence — right task → is DL needed? → transfer learning → data → honest evaluation — scopes a vision problem correctly and keeps you from over-engineering.

The deployment realities

Getting good test accuracy is not shipping. Production vision has hard realities that decide whether a model is actually usable:

• Speed and compute. Vision models can be large and slow, and many applications need real-time inference (video, robotics, AR) or must run on constrained hardware (phones, cameras, embedded/edge devices). This drives a whole discipline of making models efficient — smaller architectures, quantization, pruning, distillation — trading some accuracy for the speed/size the deployment demands (connecting to the blog’s inference-optimization coverage). An accurate model too slow or too big for its target is unusable.
• Robustness and the real world. Models fail on inputs unlike their training data — new lighting, weather, camera angles, image quality (post 1’s variations). A model that’s great on the test set can fail badly on real-world distribution shift, so you must test on realistic conditions and monitor for degradation. This is also a safety concern in high-stakes vision (autonomous driving, medical).
• Adversarial fragility. Vision models can be fooled by small, deliberate perturbations (adversarial examples) imperceptible to humans — a real security concern for vision in sensitive settings, and a reason not to over-trust a model’s confidence.
• Edge cases and the long tail. The rare, unusual inputs (an unusual object, an occlusion, a once-in-a-thousand scene) are where vision systems fail and where, in safety-critical uses, failures matter most — so handling the long tail and failing safely (abstaining, flagging for human review) is essential.
• Monitoring and retraining. Like all ML (ML series, post 8; MLOps), deployed vision models drift as the world and the camera conditions change, so monitor production accuracy and retrain on fresh data. And watch the data pipeline — a changed camera, a new image format, a preprocessing mismatch between training and serving is a classic silent failure.

The reality: a production vision system is the model plus efficiency engineering, robustness testing, edge-case/safety handling, monitoring, and data-pipeline discipline — and, as always, the model is often the smallest part.

Where the field is, and the takeaway

Closing the series with perspective:

• Vision is substantially solved for many tasks — and reusable. Classification, detection, and segmentation work well with transfer learning on modest data, and strong pretrained models and tooling are widely available. For a large class of problems, applied vision is now an integration task (pick a model, fine-tune, deploy) more than a research one.
• The frontier is multimodal and foundation models. The biggest shift (posts 6, and Multimodal AI) is vision merging with language: vision-language models that understand images and text together, image generation, and large pretrained vision foundation models that transfer to many tasks zero- or few-shot. The ViT-enabled unification (post 6) means vision is increasingly part of general multimodal AI rather than a separate field — the most important current direction.
• Classical and geometric vision endure. For matching, 3D reconstruction, SLAM, and controlled simple tasks, classical methods (post 2) remain superior — vision isn’t only deep learning, and knowing when to use the simpler tool is real expertise.

The series’ synthesis: computer vision is bridging the semantic gap from pixels to meaning (post 1), a problem that resisted hand-written rules and was cracked by learning features — classical hand-engineered features (post 2), then learned features via CNNs (posts 3–4), extended to detection and segmentation (post 5) and now to transformers and multimodal models (post 6), all made practical by transfer learning and data (post 7). To ship it: pick the right task, check whether deep learning is even needed, transfer from a pretrained model, invest in representative data, evaluate honestly, and then handle the deployment realities — speed, robustness, edge cases, monitoring — that decide whether a model that sees in the lab actually works in the world. The architecture is a solved commodity; the judgment — right task, right tool, good data, honest evaluation, robust deployment — is the craft, and it’s what turns “a computer can label an image” into vision systems people can actually rely on.

The throughline: shipping vision is mostly judgment and the system, not the model. Approach a problem by defining the precise task (classification/detection/segmentation — least detail that works, post 5), checking whether deep learning is even needed (classical CV wins for controlled/geometric tasks, posts 1–2), transferring from a pretrained model (post 7, never from scratch), investing in representative labeled data + augmentation (the real determinant), and evaluating honestly with task-appropriate metrics broken down by conditions/subgroups. Then handle deployment realities: speed/compute (real-time and edge needs → efficiency engineering — quantization/pruning/distillation), robustness to real-world distribution shift, adversarial fragility, edge cases/long tail (fail safely in high-stakes uses), and monitoring/retraining + data-pipeline discipline. The field is substantially solved and reusable for many tasks (applied vision = integration), with the frontier in multimodal/foundation models (vision merging with language, post 6) while classical/geometric vision endures. Synthesis: vision bridges the semantic gap by learning features (classical → CNN → transformer/multimodal) powered by transfer learning and data — and the craft is the judgment around a now-commodity architecture.

Key takeaways

• Approach a vision problem by: defining the precise task (classification/detection/segmentation — pick the least detail that solves it, post 5), checking whether deep learning is even needed (controlled/simple or geometric tasks → classical CV, posts 1–2, is faster/cheaper/more reliable), using transfer learning from a proven pretrained backbone (post 7), investing in representative, well-labeled data + augmentation (usually the largest effort and real determinant), and evaluating honestly (task-appropriate metrics — accuracy/mAP/IoU — broken down by conditions and subgroups).
• Deployment realities decide usability, not test accuracy: speed/compute (real-time video/robotics/AR and edge/embedded targets → efficiency engineering: smaller models, quantization, pruning, distillation — an accurate-but-too-slow/big model is unusable), robustness to real-world distribution shift (fails on unseen lighting/angles/quality — a safety issue in high-stakes vision), adversarial fragility, edge cases/long tail (fail safely — abstain/flag for review), and monitoring + retraining + data-pipeline discipline (drift and train/serve preprocessing mismatches are classic silent failures).
• A production vision system is model + efficiency + robustness + edge-case/safety handling + monitoring + data discipline — the model is often the smallest part.
• Field perspective: vision is substantially solved and reusable for classification/detection/segmentation (applied vision ≈ integration via transfer learning), the frontier is multimodal/foundation models (vision merging with language — vision-language models, image generation, zero/few-shot vision foundation models — the ViT-enabled unification, post 6, and the most important direction), and classical/geometric vision endures (matching, 3D, SLAM, controlled tasks — knowing when to use the simpler tool is real expertise).
• Synthesis: computer vision bridges the semantic gap from pixels to meaning by learning features (classical → CNNs → transformers/multimodal) powered by transfer learning and data — and the craft is the judgment (right task, right tool, good data, honest evaluation, robust deployment) around a now-commodity architecture, which turns “a computer can label an image” into vision systems people can rely on.

Further reading

• Computer vision — applications, practice, and current directions
• Convolutional neural network — the foundation of modern vision systems