Computer vision

What is computer vision?

Computer vision is software that interprets images or video. A camera captures the scene; a model turns the pixels into something a system can act on: this product has a scratch, this pallet is full, this receipt contains a total amount, this person is not wearing a helmet.

To a computer, an image is a grid of numbers. A vision model learns patterns in those numbers. Early layers in a neural network may pick up edges and simple shapes. Later layers combine those signals into objects, text, defects, or labels.

Most classic computer vision projects are trained with examples from the real environment. You collect images, label what matters, and let the model learn the relationship between the pixels and the label. That is supervised learning applied to images.

The point is not to hand-code a rule such as a scratch is a thin dark line. The model learns from examples, much like a new operator at a quality station learns by seeing enough good and bad parts.

What can computer vision do?

Image classification
The model gives one label to the whole image: approved, rejected, invoice, delivery note, empty shelf, full bin.

Object detection
The model finds objects and marks where they are, usually with bounding boxes. That lets you count pallets, detect missing parts, or locate products on a shelf.

Segmentation
The model marks regions pixel by pixel. This is useful when location and surface matter, for example measuring rust, stains, dents, or damaged packaging.

OCR
Optical character recognition extracts text from scans, photos, labels, forms, and receipts. In business processes, OCR is often the first step before document classification or invoice extraction.

Visual question answering
Multimodal language models can take an image and answer a question about it. That is useful for lower-volume tasks where context matters more than millisecond speed.

Where computer vision helps

1. Quality control. A camera checks each item on a production line and flags defects before the product leaves the factory.

2. Document processing. OCR and document intelligence extract supplier names, dates, totals, invoice numbers, and handwritten notes from scans or photos.

3. Inventory counting. Object detection counts pallets, boxes, products, containers, or empty shelf spaces from fixed cameras or mobile photos.

4. Workplace safety. A model can detect whether people wear required protective equipment or enter a restricted zone. This needs careful legal and privacy review when people are identifiable.

Classic vision model or multimodal model?

You now have two broad routes.

A classic trained model
You train or fine-tune a model for one specific task: detect this defect, classify this document type, count this product shape. It takes labelled images, testing, and a stable camera setup, but it can be fast, consistent, and cheap per image once deployed.

A multimodal language model
You send an image together with a prompt to a model such as GPT, Gemini, or Claude. It can answer many kinds of visual questions without a custom training run. That flexibility is useful for varied documents, one-off inspections, and workflows where the image needs to be interpreted together with text.

The trade-off is consistency. Multimodal models are powerful generalists, but they are not always precise counters or measurement tools. Anthropic, for example, warns that Claude can produce approximate counts and may be inaccurate with many small objects. For a production line that must accept or reject every item in real time, a specialised model is often the safer choice.

Also watch product roadmaps. Microsoft has announced the retirement of the Azure Custom Vision service in 2028, so older Custom Vision tutorials should not be treated as a future-proof architecture without checking the migration path.

What do you need to start?

Real images from your own process
Stock photos rarely behave like your factory line, warehouse, scanner, lighting, camera angle, dust, handwriting, and packaging. Use examples from the environment where the model will run.

Labels from people who know the work
The best labeler is often the operator, controller, or administrator who already makes the decision manually. If humans disagree on the label, the model will learn that confusion.

A stable capture setup
Camera position, lighting, resolution, focus, and background matter. If a human cannot see the difference in the image, a model should not be expected to see it reliably either.

Enough variation
Include different shifts, lighting conditions, product variants, suppliers, document layouts, and edge cases. Microsoft guidance for Custom Vision has long used roughly 50 images per label as a practical starting point for better generalisation, though real needs vary by task.

What to watch out for

Lighting changes
A model trained under bright daylight may fail under evening warehouse lights. Either control the environment or include those variants in the training data.

Bias in training images
The model only learns from what it sees. If every training image uses one package size, one skin tone, one document template, or one camera angle, the model may fail quietly on the rest.

False confidence
Vision systems can be wrong in ways that look plausible. Keep thresholds, manual review, and sampling in the process, especially for safety, finance, or customer-impacting decisions.

Privacy
When people are identifiable in images or video, GDPR applies in the EU. Video surveillance and workplace monitoring need a lawful basis, transparency, data minimisation, retention limits, and often stricter local employment-law checks. Film only what you need and blur or avoid faces where possible.