Computer vision
AI that looks at photos, video and camera feeds to spot defects, damage or missing items, and passes anything uncertain to a person.
What it is
Computer vision lets software interpret images and video: finding a scratch on a panel, assessing a photo of a damaged car, counting stock on a shelf, checking that a label is present and straight. It turns pictures into labels, boxes, measurements and a confidence score that a process can act on.
There are now two routes. Multimodal foundation models can describe and classify images from a prompt, with few or no examples, which suits varied, lower-volume work such as claims photos. Custom models, trained on your own labelled images, are faster, cheaper per image and more consistent, which suits high-volume, fixed scenes such as a production line. Many designs use both: a custom model for the routine check, and a multimodal model to explain or triage the odd cases.
In an enterprise, the camera, the lighting and the labelling matter more than the model. On a factory line the model often runs on an edge device beside the camera, so it keeps working if the network drops, and only results and sample images go to the cloud. Everywhere, the output feeds an existing system, such as the MES, the claims system or the WMS, and a person reviews what the model isn't sure of.
Use it when, and think twice when
Use it when
- What you care about is visible in a photo or on camera
- You can standardise lighting and angle, or volumes suit a multimodal model
- You can collect and label real examples, including rare defects
- Missing a problem, or raising a false alarm, has a known cost
Think twice when
- The fault can't be seen, or needs a measurement a camera can't make
- Bad examples are so rare you can't label enough to test
- Cameras would capture staff or the public without a clear purpose
- A sensor, barcode scan or weight check would do the same job
Reference architecture
01Channels
- Line cameras
- Phone photos and video
- Site or drone imagery
02Intake
- AWS IoT GreengrassAzure IoT Operations on Arc-enabled KubernetesONNX Runtime or NVIDIA Triton on Kubernetes at the siteEdge device at the line
- API Gateway, Amazon S3API Management, Blob StorageKong; Ceph or IONOS S3 Object StorageUpload API and image store
03Intelligence
- Amazon SageMaker AIAzure Machine Learning (AutoML for images)MLflow and Kubeflow on KubernetesCustom model: defects and parts
- Multimodal models on Amazon Bedrock, or Bedrock Data AutomationMultimodal models in Microsoft Foundry, or Azure Content UnderstandingOpen-weight vision-language models served with vLLMMultimodal model: describe and triage
04Orchestration
- AWS IoT Core, Amazon EventBridgeAzure IoT Operations MQTT broker, Event GridAn open-source MQTT broker (e.g. Eclipse Mosquitto), Apache KafkaResults and events
- AWS Step FunctionsLogic Apps or Durable FunctionsTemporalReview workflow
05Systems of record
- MES, QMS or claims system
- ERP or WMS
06People
- Person
Inspector at the hold lane - Person
Label new examples
- Person
- Cloud or platform service (AWSAzuresovereign / on-prem)
- AI agent
- A person decides
- Your existing systems
- Outside your organisation
Components, side by side
| Layer | AWS | Azure | Sovereign / on-prem |
|---|---|---|---|
| Cameras and lightingFix the lighting before tuning the model. | Industrial cameras and lighting chosen per defect | Industrial cameras and lighting chosen per defect | Industrial cameras and lighting chosen per defect |
| Inference at the edgeKeeps the line running if the network drops. Azure IoT Edge 1.5 LTS ends on 10 November 2026. | Model deployed to a GPU edge device by AWS IoT Greengrass | Model run on Arc-enabled Kubernetes with Azure IoT Operations, or Azure IoT Edge 1.6 LTS | ONNX Runtime or NVIDIA Triton on Kubernetes at the site |
| Custom model trainingLookout for Vision has ended. Azure Custom Vision retires on 25 September 2028. | Amazon SageMaker AI | Azure Machine Learning (AutoML for images) | MLflow and Kubeflow on Kubernetes |
| LabellingSageMaker Ground Truth closed to new customers on 30 July 2026. | An open-source tool such as CVAT or Label Studio, in your AWS account | Azure Machine Learning data labelling | CVAT or Label Studio on Kubernetes |
| Multimodal analysisGood for varied photos and triage. Slower and dearer per image than a custom model. | Multimodal models on Amazon Bedrock; Bedrock Data Automation for images and video | Multimodal models in Microsoft Foundry; Azure Content Understanding for images and video | Open-weight vision-language models served with vLLM on your own GPUs |
| Ready-made image analysisRekognition streaming video and bulk image analysis closed to new customers on 30 April 2026. Azure Vision Image Analysis retires on 25 September 2028. | Amazon Rekognition (image APIs and Custom Labels) | Azure Content Understanding, or multimodal models in Microsoft Foundry | Open-weight vision models |
| Image storeKeep originals with the result and model version, for audit and retraining. | Amazon S3 | Blob Storage | Ceph, or IONOS S3 Object Storage |
| Workflow and events | AWS Step Functions, Amazon EventBridge | Logic Apps or Durable Functions, Event Grid | Temporal; Apache Kafka |
| MonitoringTrack the reject rate by line and shift. | CloudWatch, CloudTrail | Azure Monitor, Log Analytics | Prometheus, Grafana, OpenTelemetry |
On a phone, the table shows the platform you pick. Sovereign / on-prem means open, portable components you can run in your own data centre or with a UK or European provider; IONOS is shown as one example, and others exist. What sovereign cloud means →
Operations
Keeping it reliable
AI fails quietly. These three habits make failure visible before customers see it.
Evaluation
Test on the rare cases
Build a test set from real images, with plenty of the rare defects and the awkward lighting, not just good parts. Measure misses and false alarms separately, because they cost different things. Retest whenever the camera, lighting, product or model changes.
Monitoring
Watch for a moved camera
Track the reject rate, confidence scores and the share sent to review, by line, site and shift. A sudden change usually means something physical: a camera knocked, a light failed, a new material. Keep a daily sample of images.
Cost
Use the cheap path for routine checks
Multimodal models charge per image and are too slow for every item on a fast line. A custom model at the edge costs little per image once trained. Use the multimodal route for low volumes and odd cases, and price both per thousand images.
Security and failure risks
Manipulated or adversarial images
An edited, reused or AI-generated photo is submitted as evidence, or a pattern on an object fools the model.
How the design avoids it
Check provenance and metadata where available, search for reused images, keep originals unchanged, and send anything that doesn't add up to a person.
Silent drift in the scene
Lighting, camera position or materials change, and accuracy falls without anyone noticing.
How the design avoids it
Daily monitoring by line and shift, reference images checked at the start of each shift, and retraining with new labelled examples.
Capturing people
Cameras record staff or the public, creating personal data the process never needed.
How the design avoids it
A DPIA before installation, cameras framed on the product, people blurred or excluded, staff told before monitoring starts, and short retention.
Edge devices as a way in
A device on the factory network is unpatched or uses shared credentials.
How the design avoids it
Device certificates, a segmented operational network, signed model updates, and patching under change control.
Playbooks that use this pattern
All 5 playbooks using Computer vision →Damage assessment from photos
Usable photos first time, a likely repair route within hours, and reused or edited images caught before payment. An engineer or handler makes every settlement and total-loss decision.
- Computer vision
- Human in the loop
- Agents with tools
- Event-driven integration
Product content at scale
New lines go from supplier data sheet to accurate, accessible listing on the same day, with every claim traced back to the spec. A person approves every listing before it goes live.
- Document AI
- Computer vision
- Agents with tools
- Human in the loop
Visual quality inspection
Every unit checked at line speed, clear passes sent on, anything doubtful held for a person, and every defect traced to its batch. Inspectors make the calls that matter.
- Computer vision
- Predictive ML
- Human in the loop
- Event-driven integration
Repairs triage and scheduling
Every repair reaches the right trade, with the right parts and the right priority, at an appointment that suits the tenant. Fixed rules handle emergencies, and a person decides anything that might be a hazard.
- Computer vision
- Agents with tools
- Event-driven integration
- Human in the loop
Construction site safety monitoring
People near moving plant, open edges and lifts spotted in seconds and flagged to the supervisor and the operator, with near-miss trends feeding next week's plan. No one is identified, scored or disciplined by the system, and people make every call.
- Computer vision
- Predictive ML
- Human in the loop
- Event-driven integration
Patterns
All eight building blocks
AI Explained
Document AI
Read forms, letters and PDFs into structured data.
Retrieval (RAG)
Answer questions from your own documents, with sources.
Agents with tools
AI that takes small, controlled actions across your systems.
Human in the loop
Put people on the decisions that matter, with the evidence.
Predictive ML
Score, rank and forecast from your own historical data.
Event-driven integration
Connect AI to core systems safely, through events and queues.
Computer vision
Inspect photos, video and camera feeds for defects and damage.
Speech and voice
Turn calls and conversations into text, notes and actions.
Thinking about computer vision in your organisation?
Talk through where it fits, what it would plug into, and which decisions should stay with people.
