Industrial Edge AI with YOLO11 and Hailo: A Reproducible Bench Workflow

This implementation is an authorized bench experiment, not a production inspection system. It replays a recorded input through a version-bounded Ultralytics YOLO11 and Hailo toolchain, applies explicit detection rules, and writes only to a synthetic PLC sink. It does not open a camera, connect to a PLC, actuate equipment, or claim functional safety, production readiness, FPS, or end-to-end plant latency.

1. Freeze the experiment before exporting

Record the Ultralytics package version, Hailo Dataflow Compiler version, HailoRT version, firmware version, target device, model checksum, class map, and input dimensions in a manifest. This article deliberately names YOLO11, not “latest YOLO”: newer families and exporters can change graph behavior. Rebuild the artifact when any recorded boundary changes; do not assume an older HEF is compatible with a newer runtime or firmware.

Choose one detection checkpoint, such as yolo11n.pt, and one fixed tensor shape, for example 1x3x640x640. The dimensions are an experiment parameter, not a performance promise. Fixed shape removes dynamic-dimension ambiguity and makes preprocessing repeatable. Preserve the recording checksum and an expected-results file beside the manifest so the same bytes can be replayed after upgrades.

2. Keep preprocessing, export, compilation, and runtime separate

Four boundaries matter:

1. Preprocessing reads each recorded frame, applies the chosen resize or letterbox policy, channel order, numeric scaling, and tensor layout.
2. Ultralytics export converts the pinned YOLO11 checkpoint into the interchange artifact accepted by the selected Hailo toolchain. Export with the fixed input shape and retain the exporter log.
3. Hailo Dataflow Compiler parses, optimizes, quantizes, and compiles that artifact for the exact Hailo target, producing the deployable HEF. This is an offline build step.
4. HailoRT loads the HEF on the host, transfers tensors, runs inference on the accelerator, and returns outputs. It does not retrain or compile the model.

Do not hide these stages behind one opaque command. Store each artifact checksum and fail startup if the manifest, HEF, class labels, runtime, firmware, or expected input shape disagree. A container can pin user-space packages, but it cannot make an incompatible host driver or device firmware compatible.

3. Calibrate with representative data

Quantization calibration needs representative inputs from the intended visual domain: expected materials, lighting ranges, backgrounds, orientations, clean parts, known defect examples, and difficult negatives. Calibration data estimates activation ranges; it is not a validation set and does not prove accuracy. Keep calibration, training, and final evaluation subsets separate to avoid optimistic results.

Apply exactly the preprocessing declared for compilation. A BGR/RGB swap, stretch-versus-letterbox change, different normalization, or wrong tensor layout can damage results while still producing valid numbers. Record the dataset revision, selection rule, item count, and preprocessing checksum. Privacy rules still apply to recorded video: minimize the field of view, redact people or identifiers where possible, encrypt retained recordings, define deletion dates, and restrict access.

4. Replay deterministic input and make decisions explicitly

Use a short, immutable video file or numbered image sequence. OpenCV documents that VideoCapture can read video files and image sequences as well as devices; this bench uses only the file path. Reject a missing file, failed decoder, unexpected dimensions, non-monotonic frame index, or checksum mismatch before inference. Record the selected OpenCV backend because decoding can vary by platform.

Confidence and intersection over union (IoU) solve different problems. Confidence filters weak candidate detections. IoU measures box overlap and is commonly used by non-maximum suppression to remove duplicate boxes for the same object. A confidence threshold of 0.60 does not mean “60% safe,” and an IoU threshold is not another confidence check. Both thresholds must be selected against held-out bench evidence and recorded in the manifest.

For a deterministic decision fixture, accept candidates at or above the confidence threshold, suppress lower-confidence same-class boxes whose IoU with an accepted box exceeds the NMS threshold, and mark the decision stale when its frame age exceeds the configured limit. A stale positive becomes UNKNOWN, never a delayed reject. The companion verifier executes this tiny rule without a model or device.

5. Bound queues, backpressure, and stale detections

An unbounded frame queue converts overload into old decisions. Give capture, inference, and result delivery bounded queues. For inspection-like logic, prefer dropping an older unprocessed frame and counting the drop rather than silently accumulating delay. Every result should carry the recording ID, frame index, capture or replay timestamp, inference completion timestamp, model ID, and threshold set.

If the synthetic sink cannot keep up, apply backpressure, expose queue depth, and stop accepting new decisions at a documented limit. On timeout, sequence gap, accelerator reset, decoder error, or stale result, publish UNKNOWN to the synthetic sink and require an explicit recovery. Never reuse the last positive or negative as if it described a newer frame.

6. Measure without inventing performance

This article reports no measured Hailo throughput or latency. A valid timing report must state the host hardware, exact Hailo device, model and HEF checksum, input shape, sample count, warm-up policy, and software versions. It must also name included stages: decode, preprocessing, host-to-device transfer, accelerator inference, device-to-host transfer, postprocessing, queue wait, and synthetic-sink write. Report distributions such as median and upper percentiles, not only an average.

The verifier’s fixture is a deterministic logic check, not a hardware benchmark. It uses three recorded candidate boxes and fixed timestamps to prove confidence filtering, IoU suppression, and stale-frame handling. Its elapsed time must not be presented as model latency.

7. Start safely, observe failures, and roll back

Startup should be fail-closed: validate checksums and versions, load the HEF, verify tensor names and shapes, run one known recorded sample, and compare structural expectations before declaring the bench ready. Log artifact IDs, counters, queue depth, drops, stale results, and categorized errors without storing raw frames by default.

Keep the previously validated manifest, HEF, labels, and runtime image as one rollback unit. If startup validation or replay regression fails, stop the candidate, restore that complete unit, and rerun the known sample. Do not mix a previous HEF with new labels or partially downgrade only the application package.

Run as a non-root account with read-only model and recording mounts. Grant access only to the required accelerator device; expose no camera nodes, industrial network route, or PLC credentials. Keep the synthetic sink local and append-only. Perform testing only on hardware, recordings, and networks you are authorized to use.

8. Acceptance boundary

The experiment passes when the pinned environment reproduces the expected decisions for the recorded fixture, rejects stale data, survives malformed input without emitting a positive decision, and produces a complete timing record if hardware measurements are made. That result supports further engineering evaluation only. A real installation additionally needs site-specific hazard analysis, independent safety controls, validated optics and lighting, change management, cybersecurity review, and evidence gathered on the actual process.

Official references and limitations

• Ultralytics YOLO11 documentation — model tasks and export support; accessed 2026-07-24. Vendor documentation does not validate this application or its accuracy.
• Hailo AI Software Suite — Dataflow Compiler and HailoRT roles; accessed 2026-07-24. Vendor descriptions do not establish compatibility for an unlisted version combination.
• Hailo-8 accelerator — target hardware overview; accessed 2026-07-24. Peak device specifications are not application FPS or latency and are not converted here.
• OpenCV VideoCapture 4.x — recorded-file and image-sequence input behavior; accessed 2026-07-24. Backend behavior remains platform-dependent.