Computer Vision at the Edge: An Industrial Evaluation Guide

An edge vision evaluation should answer a narrower question than “does the model draw the right box?” It should show whether a controlled image becomes a timely, auditable decision under declared assumptions. The workflow below uses generated or recorded frames and a synthetic PLC sink. It does not validate a live camera, actuator, production network, functional-safety function, or production readiness.

1. Freeze the optical assumptions

Write the image contract before selecting a model. Record camera position, lens and focus, exposure, resolution, trigger mode, expected part pose, conveyor speed range, and permissible vibration. Specify lighting geometry, intensity, color temperature, diffusion, ambient-light shielding, and the surfaces likely to reflect or absorb light. A model score cannot compensate for an undocumented optical change.

Define one pixel-coordinate ROI and the reason for excluding everything else. The fixture in this batch treats the ROI as [20, 20, 80, 80]; detections whose center falls outside it are ignored. That is a test rule, not a universal inspection method. Commissioning must verify that the real part remains inside the ROI across tolerances and that excluded pixels cannot contain a relevant defect.

2. Keep recorded input separate from capture claims

OpenCV documents VideoCapture for video files, image sequences, cameras, and IP streams. Its current Java API also accepts a preferred capture API and exposes getBackendName() after opening. The Video I/O overview shows runtime backend selection. These facts support a reproducible recorded-file harness and explicit backend reporting; they do not prove equivalent behavior for a particular camera or driver.

Pin the OpenCV version, input checksum, dimensions, pixel format after decode, requested backend, reported backend, and frame count. Fail closed if the source cannot open, a frame cannot decode, dimensions change, or the backend differs from the approved test configuration. A recorded clip cannot validate trigger jitter, dropped packets, exposure transitions, lens contamination, or live buffering.

3. Build a deterministic, privacy-safe fixture

Use generated geometric frames when possible. If recorded material is necessary, stage manufactured samples on a neutral background: no workers, badges, screens, labels, serial numbers, audio, or customer product data. Store only the smallest clip needed, strip metadata, document consent and retention, and publish a checksum rather than sensitive imagery.

The verifier supplies deterministic detections for three conditions: a defect centered inside the ROI, a high-confidence detection outside it, and an old frame. With fixed thresholds and timestamps, the expected decisions are REJECT, CLEAR, and UNKNOWN. This tests ROI filtering, frame age, and decision policy without OpenCV, a model runtime, network access, or hardware.

4. Make uncertainty an explicit output

Use at least three states: REJECT, CLEAR, and UNKNOWN. UNKNOWN covers stale or missing frames, decode failure, unavailable inference, malformed output, and policy exceptions. Never silently convert an unavailable observation into CLEAR.

Choose thresholds from labeled acceptance data, not a tutorial. A false reject wastes good product and may interrupt flow; a miss releases a defect. Product, quality, and operations owners must set the acceptable tradeoff by defect class. Report a confusion matrix and false-reject and miss counts on a held-out set, then review ambiguous examples. Safety-relevant hazards require a separate risk assessment and safety-rated controls; this classifier is not a safety function.

5. Measure inference-to-decision timing honestly

Timestamp frame availability and final decision emission with the same monotonic clock. Report percentiles and worst observed value, not an unsupported generic FPS or latency number. Compare that measured distribution with the PLC scan and machine takt budget, including transport and actuator margins supplied by controls engineering. The comparison shows budget compatibility only; it does not create deterministic coupling between Linux inference and a PLC scan.

Every benchmark result must state hardware and accelerator, OS and software versions, model and input shape, batch size, sample count, warm-up policy, and clock method. It must also state whether preprocessing, host-to-device transfer, inference, device-to-host transfer, postprocessing, and decision logic are included. This article publishes no benchmark because no authorized run with that complete record was supplied.

6. Prefer a current unknown over an old answer

Bound the queue. When inference falls behind capture, discard superseded frames or stop intake according to the inspection contract; do not let latency grow invisibly. Attach capture time and sequence ID to every frame and decision. Reject duplicate or out-of-order IDs, and return UNKNOWN when frame age exceeds the configured limit. Track queue depth, dropped frames, stale decisions, decode failures, and inference errors.

7. Stop at a synthetic PLC boundary

During this evaluation, emit {sequence, capturedAt, decidedAt, result, reason} to an in-memory synthetic PLC sink. Assert ordering, one decision per accepted sequence, and fail-closed behavior. Do not write GPIO, Modbus registers, PLC tags, or actuators. Production integration needs controls-owned addressing, electrical design, authentication and network controls where applicable, timeout and heartbeat semantics, interlocks, and a separately tested safe response.

8. Commission with evidence

• Lock camera, lens, lighting, ROI, software, model, and threshold versions.
• Re-run the checks with approved recorded samples and documented expected decisions.
• Measure the complete timing stages on named hardware and compare them with the reviewed budget.
• Challenge blur, glare, occlusion, shifted parts, missing frames, stalled inference, and a disconnected synthetic sink.
• Obtain quality approval for false-reject and miss policy; obtain controls approval before any PLC connection.

9. Operate for drift and failure

Alert on stale-frame rate, unknown rate, class distribution, false rejects found by review, confirmed misses, queue depth, thermal throttling, storage pressure, and process-image drift. Keep privacy-safe samples for investigation under a defined retention policy. A dashboard is evidence for diagnosis, not proof that inspection remains valid.

Rollback means disabling the vision decision consumer, restoring the previously approved model/configuration as one versioned unit, and returning machine authority to the documented fallback process. Preserve logs and fixture results. Do not auto-retry actuator commands or reinterpret UNKNOWN as acceptance. Recommission after camera, lighting, ROI, model, runtime, or timing-budget changes.

Official OpenCV references

• OpenCV 4.x VideoCapture API: accepted source types, preferred capture API, open-state check, frame grabbing, and reported backend name.
• OpenCV Video I/O overview: runtime backend selection for devices and files.

These references define the software input boundary only. The fixture and commissioning evidence must establish the behavior of the actual inspection system.