Edge AI: what changes when inference runs in the factory, not in the cloud

When an AI model inference runs in the plant, in a warehouse, or in a retail rack, it stops being a cloud service and becomes local infrastructure. That shifts the conversation around cost, latency, and privacy.

Edge AI is not running ChatGPT on a local server. It is putting a trained model (vision, anomaly detection, demand forecasting) to produce inferences in milliseconds over data that cannot leave the site.

Why inference is moving to the factory, not the cloud

There are three technical reasons why inference is moving to the edge. All three are economic or regulatory, not fashion-driven.

• Deterministic latency: a manufacturing line using vision-based defect detection needs a response in less than 50 ms. A call to the cloud adds between 30 and 200 ms of network alone, plus model processing time.
• Cost per inference: in industrial vision, between 10 and 100 inferences per second are generated per camera. Multiplied by 24/7 shifts, the monthly cost of cloud inference exceeds that of an edge device in less than a year.
• Sensitive data: many factories cannot upload product images, blueprints, or process video to external services due to contract, local regulation, or internal policy.

What changes when the model runs in the factory

The most important change is not technical: it is operational. If the model lives in the cloud, an internet outage shuts down your vision line. If it runs at the edge, the model keeps working even without WAN. That turns AI from a ‘service we subscribe to’ into an ‘asset we maintain’.

Modern edge devices are ruggedized computers with server-class CPUs, integrated GPUs (NVIDIA Jetson, Hailo, Intel Movidius), and industrial connectivity (RS-485, Modbus TCP, Profinet, EtherCAT). They operate between 0 and 50 °C, without forced ventilation, with solid-state drives (SSD) and redundant power supplies.

The other change is in the support model. In the cloud, the provider updates the model and you receive the change. At the edge, you decide when to update. That requires a local MLOps (machine learning operations) pipeline: versioning, validation with site data, rollback if accuracy drops.

Use cases where edge already wins

1. Vision for quality control on packaging lines, where every second counts and a wrong inference stops the line.
2. Predictive maintenance on motors, pumps, and compressors: the model listens to local vibration and temperature and triggers an alert before failure.
3. Security anomaly detection (PPE, intrusion, restricted areas) on cameras that cannot upload video for privacy reasons.
4. HVAC (air conditioning) and energy optimization: the model predicts thermal load with local sensor data and adjusts chillers and free cooling in real time.
5. Automated inventory in retail and logistics: ceiling-mounted cameras that count SKUs (inventory units) without sending images to the cloud.

What almost nobody sizes correctly from the start

The edge device is not the expensive part. The expensive part to operate is the local data pipeline: how you get images or signals to the model, how you label exceptions to retrain, how you audit inferences for regulatory compliance. Without a pipeline, you have expensive hardware producing inferences you cannot improve.

Latency is no longer the critical variable. The critical variable is the retraining cycle: how fast you can detect model drift (degradation of behavior when comparing new data against the original training), collect new labeled data, and deploy the next version without stopping operations.

Edge AI in the factory is an architecture decision, not a vendor decision. Before choosing the edge device, define the data pipeline, the rollback plan, and the support model. Once that is defined, the hardware becomes interchangeable.

Sources

[1] NVIDIA — Embedded Systems for Edge AI — https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/

[2] Hailo — AI Processors for Edge Devices — https://hailo.ai/

[3] IEEE — Edge Computing Standards — https://standards.ieee.org/ieee/1934/7237/

[4] Uptime Institute — Research and Reports — https://uptimeinstitute.com/resources/

[5] Wikipedia — Edge computing — https://en.wikipedia.org/wiki/Edge_computing