> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cyberwave.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Edge worker container

> Run ML worker scripts on edge devices with managed lifecycle, hot-reload, and health monitoring

## Overview

Edge Core manages a dedicated worker container (`cyberwave-worker-{env_uuid[:8]}`) on each edge device. Worker scripts run inside this container with access to the Zenoh data bus, cached model weights, and all environment twin data.

<Info>
  One worker container runs per edge device (not per twin). Workers can consume
  data from all twins in the environment simultaneously.
</Info>

## Worker Directory

Place Python worker scripts in the edge config workers directory:

| Platform | Path |
| - | - |
| All | `~/.cyberwave/workers/` |

## Model Requirements

Declare model dependencies in `cyberwave.yml` inside the workers directory:

```yaml theme={null}
models:
  - yolov8n
  - background-subtraction
```

Edge Core pre-downloads listed models before starting the worker container. Models are also auto-detected from `cw.models.load("...")` calls in worker Python files.

### Picking the model format

Each catalog model declares an **edge runtime** that tells the worker how to load the checkpoint. The runtime selector lives in the model editor (`/models` → "Add Model" or "Edit"), and is also accepted as a typed `edge_runtime` field on `POST /api/v1/mlmodels` and `PUT /api/v1/mlmodels/{uuid}`.

The well-known runtimes mirror the loaders in the Cyberwave Python SDK's `cyberwave.models.runtimes` registry:

| Runtime | Extension | Loader |
| - | - | - |
| `ultralytics` | `.pt` | YOLOv5/8/11 via the Ultralytics package |
| `onnxruntime` | `.onnx` | ONNX Runtime (CPU, CUDA EP on the GPU image) |
| `opencv` | `.xml` / `.caffemodel` | OpenCV Haar / DNN models |
| `tflite` | `.tflite` | TensorFlow Lite |
| `torch` | `.pt` / `.pth` | TorchScript / raw PyTorch |
| `tensorrt` | `.engine` / `.trt` | TensorRT engines (GPU image only) |
| `whisper_cpp` | `.gguf` / `.bin` | Whisper.cpp speech-to-text on CPU edge devices |
| `faster_whisper` | CTranslate2 weights (HF cache) | faster-whisper STT on edge CPU/GPU |
| `hailo` | `.hef` | Hailo accelerator (`hailo_platform`) on Pi 5 + AI HAT+ / Pi AI Kit |

Custom values are accepted via the editor's "Other" entry — used today for framework-specific identifiers like `sam2`, `sam3`, and `depth_anything_v2` that don't have an SDK loader yet but still need to round-trip through the catalog.

`GET /api/v1/mlmodels/edge-runtimes` returns the current well-known list (no auth required) so external tools can mirror the dropdown without hard-coding it.

ONNX YOLO postprocessing applies per-class non-max suppression with a default IoU threshold of `0.7` so swapping `yolov8s.pt` for `yolov8s.onnx` produces the same number of boxes per object instead of a cluster of overlapping anchors. Override per-call via `model.predict(frame, iou=0.5)` (stricter) or `iou=1.0` (raw output, no suppression).

NMS-free detectors (YOLO26's default one-to-one head, YOLOv10) emit `[max_det, 6] = [x1, y1, x2, y2, conf, class_id]`; the SDK's ONNX runtime detects this layout automatically and parses it without the anchor-decoding step. The model's one-to-one head is supposed to already be deduplicated, but some exports (open-vocabulary heads especially) can still emit many near-duplicate boxes for one object, so the same `iou` NMS also runs here as a safety net. End-to-end pose / segmentation exports are not decoded yet; re-export those tasks with `end2end=False` until the e2e pose branch lands.

Every detection result is additionally capped at `max_detections_per_class` (default `30`) highest-confidence boxes per label, applied after NMS — a backstop for models whose suppression doesn't fully collapse duplicates (open-vocabulary heads like YOLOE are the common case). Pass `model.predict(frame, max_detections_per_class=None)` to disable it. The Ultralytics runtime also defaults every call to `end2end=False` so real NMS runs on the one-to-many head instead of trusting a checkpoint's self-reported "already deduplicated" flag; checkpoints with only a one-to-one head (fused `.pt`, TensorRT, some ONNX exports) fall back to `end2end=True` automatically.

### YOLO ONNX catalog

The public catalog ships ONNX twins for the most common YOLO entries — Cyberwave hosts the exported weights at `https://static.cyberwave.com/ml_models/{name}.onnx` so edge nodes can fetch them with no Ultralytics CDN dependency. Roughly 2× faster on CPU than the `.pt` build:

| Family | Sizes | Tasks |
| - | - | - |
| YOLOv8 | n, s | detect; plus pose (n only) |
| YOLO26 | n, s, m, l, x | detect, pose |

Pick `Edge Runtime: ONNX Runtime` in the model editor (or `metadata.edge_runtime = "onnxruntime"` via the API) when registering a private fine-tune that you've exported via `YOLO("…").export(format="onnx")`. The SDK's `OnnxRuntime` reads `names` and `kpt_shape` from the ONNX `custom_metadata_map`, so labelled detections and keypoints work out of the box.

Segmentation, OBB, and classification ONNX exports are not yet supported by the SDK runtime — use the `.pt` (`ultralytics`) variants for those tasks.

### YOLO Hailo catalog

For Hailo-accelerated edge nodes (Raspberry Pi 5 + AI HAT+ and the older Pi AI Kit) the catalog also ships pre-compiled YOLO HEFs straight from the Hailo Model Zoo. Each upstream YOLO checkpoint gets two sibling rows — one per Hailo architecture — because HEFs are hardware-locked:

| Slug | Family | Arch | Edge target |
| - | - | - | - |
| `yolov6n_h8` / `yolov6n_h8l` | YOLOv6 Nano | Hailo-8 / Hailo-8L | 26 / 13 TOPS |
| `yolov8n_h8` / `yolov8n_h8l` | YOLOv8 Nano | Hailo-8 / Hailo-8L | 26 / 13 TOPS |
| `yolov8s_h8` / `yolov8s_h8l` | YOLOv8 Small | Hailo-8 / Hailo-8L | 26 / 13 TOPS |
| `yolov8m_h8` / `yolov8m_h8l` | YOLOv8 Medium | Hailo-8 / Hailo-8L | 26 / 13 TOPS |

Slugs use the `_h8` / `_h8l` suffix on purpose: `cw.models.load("yolov8s_h8")` is enough to route through the SDK's `hailo` runtime — no extra arguments needed. Pick the row that matches your edge target; a Hailo-8 HEF will refuse to run on a Hailo-8L (and vice versa).

If you pick the wrong sibling, Edge Core catches it before the download starts: a Gate-3 preflight probes the connected accelerator with `hailortcli fw-control identify` and refuses to fetch a HEF whose `metadata.hw_arch` disagrees with the device, naming the correct sibling in the error message. The preflight is silent when `hailortcli` is not installed on the host — install it via `apt install hailo-all` (or the matching HailoRT package) to enable the check.

### Edge speech-to-text

Seeded Whisper.cpp STT models can run from `Audio Track -> Call Model` workflows on Raspberry Pi 4-class devices. The generated worker passes the audio chunk and seeded `download_url` into the local `whisper_cpp` runtime, so the first run downloads missing weights into the Cyberwave model cache. Downstream nodes can use `text`, `transcribed_text`, and `segments`.

English GGML checkpoints: **Tiny EN**, **Base EN**, and **Small EN** (Q5\_1). Multilingual checkpoints: **Tiny Multilingual** and **Base Multilingual** — hybrid deployment with `edge_runtime: whisper_cpp` on edge missions and whisper cloud-node fallback (`openai/whisper-tiny` / `openai/whisper-base`) on cloud missions. Use multilingual models when the spoken language is unknown or mixed.

Hybrid **Faster Whisper** catalog entries (`faster_whisper` runtime) run on-device when the workflow mission executes on the edge; cloud missions use the whisper cloud node with the same OpenAI model aliases (`openai/whisper-tiny` → `tiny.en`, etc.). Prefer **Faster Whisper Tiny EN** for real-time English streams; use **Whisper Tiny Multilingual Q5\_1** for multilingual edge STT without CTranslate2.

### Model weights resolution

For each required model, Edge Core resolves weights in this order:

1. **Local cache (intact).** If `~/.cyberwave/models/{model_id}/...` is present and its SHA-256 matches the manifest, it is used directly.
2. **Cyberwave-hosted signed URL.** If the catalog entry has a checkpoint mirror, Edge Core fetches a signed URL via `GET /api/v1/mlmodels/{uuid}/weights` and downloads from our private bucket.
3. **Upstream weights URL.** If no Cyberwave mirror exists, Edge Core falls back to the public `download_url` from the catalog (e.g. an official Ultralytics release).
4. **Stale cache fallback.** If every download attempt fails but the local file is intact, Edge Core returns the cached file with a warning. This keeps workers running across transient network failures and on permanently air-gapped sites.

Operators on air-gapped sites can pre-stage weights by copying them to `~/.cyberwave/models/{model_id}/`. Edge Core computes a SHA-256, infers the runtime from the file extension (`.pt`, `.onnx`, `.engine`/`.trt`, `.tflite`, `.pth`, `.xml`), and writes a sidecar `metadata.json` on the next worker start. To update a pre-staged model, simply overwrite the file in place — Edge Core re-stamps the manifest from disk on the next call (no re-download attempted). Pre-staged files are never auto-overwritten by catalog updates; to force a re-download from Cyberwave, evict the model directory (`rm -rf ~/.cyberwave/models/{model_id}`).

## CLI Commands

```bash theme={null}
cyberwave worker start      # Start the worker container
cyberwave worker stop       # Stop the worker container
cyberwave worker restart    # Restart (re-scans workers, re-downloads models)
cyberwave worker status     # Show container state and loaded workers
cyberwave worker health     # Show detailed restart history and circuit-breaker state
cyberwave worker logs       # Stream worker container logs
```

<Tip>
  These commands are also available via `cyberwave-edge-core worker …` if you prefer to use the edge-core CLI directly.
</Tip>

## Hot-Reload on File Changes

Edge Core monitors the workers directory every \~15 seconds. When `.py` files are added, removed, or modified, the worker container is automatically restarted with the updated set of workers.

A minimum cool-down of 10 seconds between successive automatic restarts prevents rapid churn when files are written incrementally.

## Health Monitoring

Edge Core continuously monitors the worker container:

* **Restart accounting**: every restart is recorded with timestamp and reason.
* **Circuit-breaker**: after 5 restarts in 5 minutes, automatic restarts are suppressed until the window clears. Run `worker health` to inspect the state.
* **Spontaneous exit detection**: if the container exits without a deliberate restart, a warning is logged.

## Performance Tuning

### Model warm-up

The worker runtime automatically runs two dummy inferences on each loaded model at startup to eliminate cold-start latency (JIT compilation, memory allocation). Cold vs warm latency is logged.

You can also warm up models explicitly:

```python theme={null}
model = cw.models.load("yolov8n")
cold_ms, warm_ms = model.warm_up(input_shape=(640, 640, 3))
```

### Frame resolution scaling

Set `CYBERWAVE_WORKER_INPUT_RESOLUTION` to downscale incoming frames before they reach your worker hooks. This reduces inference time on constrained devices without changing the camera driver's publish resolution.

```bash theme={null}
export CYBERWAVE_WORKER_INPUT_RESOLUTION=640x480
```

### Shared memory transport

Zenoh shared-memory (SHM) transport offers zero-copy frame delivery between the camera driver and worker containers on the same host. Edge Core leaves `ZENOH_SHARED_MEMORY` **disabled by default** because SHM between Docker containers requires them to share an IPC namespace via `--ipc=host`, which weakens container isolation and has historically been a source of instability in production.

To opt in, enable the flag on **both** the publisher and any same-host consumer, and give each container a shared IPC namespace (`--ipc=host`) and a raised locked-memory limit (`memlock=-1` — the default 8 MB cap is too small for a frame pool, and without it the transport silently falls back to copy):

```yaml theme={null}
# docker-compose fragment — publisher + same-host consumer
services:
  camera-driver:
    ipc: host                 # shared /dev/shm + IPC namespace
    ulimits:
      memlock: -1
    environment:
      ZENOH_SHARED_MEMORY: "true"
  ml-worker:
    ipc: host
    ulimits:
      memlock: -1
    environment:
      ZENOH_SHARED_MEMORY: "true"
```

Under Edge Core, set `ZENOH_SHARED_MEMORY=true` once in its environment and it propagates the flag to the driver and worker containers; with the SDK directly, set it on each container (or pass `ZenohBackend(shared_memory=True)`). Zero-copy applies only to frame-sized payloads; small messages (joint states, etc.) always copy. Optional publisher knobs:

| Variable | Default | Purpose |
| - | - | - |
| `ZENOH_SHM_POOL_BYTES` | 64 MB | Size of the shared-memory pool. Raise for large frames or deep in-flight buffering. |
| `ZENOH_SHM_MIN_BYTES` | 4096 | Payloads below this size skip SHM and publish via copy. |
| `ZENOH_SHM_ON_EXHAUSTION` | `copy` | Backpressure when the pool is full: `copy` (guaranteed delivery, latency grows under overload) or `drop` (skip the frame, stay low-latency — for live video). |

If the pool can't be created, publishing degrades to the copy path with a warning rather than failing. When it's exhausted by a slow consumer, the `ZENOH_SHM_ON_EXHAUSTION` policy decides between copying and dropping the frame. The pool is allocated per frame whether or not a subscriber can use zero-copy, so enable SHM only when a same-host, SHM-capable consumer is actually reading the stream.

<Warning>
  **A healthy-looking publisher does not prove zero-copy delivery.** The pool is created and used successfully even when no consumer can map it — for example if the containers omit `ipc: host`, each gets its own private `/dev/shm`. In that case the publisher reports SHM enabled and its `shm_frames` counter climbs, while every consumer silently receives ordinary wire copies and you get none of the latency benefit.

  Those counters are publish-side intent only. The single source of truth is the **receiving** end: a sample is genuinely zero-copy only when `sample.payload.as_shm()` is not `None`. Verify there once when rolling SHM out, rather than trusting the publisher's stats.
</Warning>

#### Consuming the stream from your own process

The SHM path isn't specific to the ML worker — **any Zenoh peer on the same host can read a driver's frames zero-copy**. To consume from your own process:

* **Run it on the same host.** If it's containerized, launch it with `--ipc=host` (or `--ipc=container:<driver>`) so it shares the IPC namespace; a process running directly on the host already qualifies.
* **Enable SHM in its Zenoh session** — `ZENOH_SHARED_MEMORY=true` via the SDK, or `transport/shared_memory/enabled=true` on a raw Zenoh session. A consumer that leaves SHM off still receives every frame, just copied over loopback instead of zero-copy.
* **Subscribe to the driver's channel.** With the SDK, use `DataBus.subscribe(...)` — it resolves the key for you. `memlock=-1` is only needed if your process *also* publishes SHM.

Read-side note: a **Python** consumer (SDK or raw `eclipse-zenoh`) receives each frame as a copy — the zero-copy is the transport hop, so it still cuts latency and jitter but not the final ingest copy. A native **C++/Rust** Zenoh consumer can map the shared buffer and read it zero-copy end-to-end.

## GPU Access

Edge Core detects the NVIDIA container runtime and passes `--gpus all` to the worker container when available.

### Image variants

The worker image is published in four variants on Docker Hub:

| Tag | Base | Accelerator | Architectures | Use when |
| - | - | - | - | - |
| `cyberwaveos/edge-ml-worker:<tag>` | `ubuntu:24.04` | CPU (`onnxruntime`) | `linux/amd64`, `linux/arm64` | No GPU available, or inference is light enough for CPU. |
| `cyberwaveos/edge-ml-worker:<tag>-gpu` | `nvidia/cuda:12.6.3-runtime-ubuntu24.04` | NVIDIA discrete (`onnxruntime-gpu`) | `linux/amd64` | Edge device has an NVIDIA GPU and `nvidia-container-toolkit` installed. |
| `cyberwaveos/edge-ml-worker:<tag>-jetson` | `ubuntu:24.04` (arm64) | NVIDIA Tegra iGPU (PyTorch + ONNX Runtime GPU, Jetson AI Lab cu129 wheels; CUDA injected by host at runtime) | `linux/arm64` | Jetson Orin on JetPack 7 (Ubuntu 24.04 / L4T r38). JetPack 6 hosts are not supported. |
| `cyberwaveos/edge-ml-worker:<tag>-hailo` | `cyberwaveos/edge-ml-worker:<tag>` | Hailo-8 / Hailo-8L (`hailo_platform`) | `linux/arm64` | Edge device has a Hailo accelerator exposed at `/dev/hailo0` (e.g. Raspberry Pi 5 + AI HAT+). |

The `-jetson` variant uses PyTorch/torchvision/torchaudio and `onnxruntime-gpu` wheels from the [Jetson AI Lab devpi index](https://pypi.jetson-ai-lab.io/jp6/cu129/) (Orin `sm_87` builds), not the amd64 `-gpu` variant's index. The image base is `ubuntu:24.04`; NVIDIA has not yet published an `l4t-jetpack` image for JP7. nvidia-container-runtime injects the host's CUDA driver at container start; the image itself bundles the matching CUDA 12.9 userspace runtime libraries (`libcudart`, `libcublas`, cuDNN) since the Jetson wheels link against system CUDA rather than bundling it. ONNX Runtime exposes `CUDAExecutionProvider` on Jetson, same as the `-gpu` variant. **JetPack 6 hosts are not supported** — JetPack 6's CUDA driver predates what's needed to run a CUDA 12.9 runtime, so only JetPack 7 hosts should pull this tag. Edge Core reads the L4T release from `/etc/nv_tegra_release` and starts a JetPack 6 board on the plain (CPU, multi-arch) tag instead, logging a warning; `CYBERWAVE_JETSON_ALLOW_UNSUPPORTED_L4T=1` forces the `-jetson` tag anyway. This is decided when the worker container is created, so a board already running a `-jetson` worker keeps it across an Edge Core upgrade — remove the container once (`docker rm -f cyberwave-worker`) to pick up the new selection.

The `-hailo` variant compiles HailoRT (`libhailort` + `hailortcli` + `pyhailort`) from the MIT-licensed upstream source at [`hailo-ai/hailort`](https://github.com/hailo-ai/hailort) at a pinned tag. The LGPL-2.1 GStreamer plugin (`hailonet`) is left out of the build. The companion kernel driver ([`hailo-ai/hailort-drivers`](https://github.com/hailo-ai/hailort-drivers)) is installed on the host (e.g. via `apt install hailo-all` on Raspberry Pi OS), not in the container — its minor version must match the userspace lib baked into the image.

All four variants ship the same Python API. PyTorch/Ultralytics models pick the device automatically; ONNX models gain `CUDAExecutionProvider` only on the `-gpu` variant; `.hef` files load through the `hailo` runtime only on the `-hailo` variant.

Edge Core selects the variant automatically based on host capabilities:

* **Jetson host detected** (`/etc/nv_tegra_release` present) → appends `-jetson` and adds `--gpus all`. Wins over the `-gpu` branch: on Jetson the nvidia-container-runtime is present too, but the amd64 `-gpu` blob has the wrong `libcuda.so` ABI (desktop/server CUDA vs Tegra CUDA).
* **NVIDIA container runtime detected (not Jetson)** → appends `-gpu` to the configured worker image tag and adds `--gpus all`.
* **`/dev/hailo0` present and no GPU runtime** → appends `-hailo`, adds `--device /dev/hailo0:/dev/hailo0:rwm` and `--group-add hailo` (on HailoRT \< 4.20 hosts), and sets `CYBERWAVE_REQUIRED_DEVICES=/dev/hailo0` so the image's Gate-4 entrypoint fails fast if the passthrough is missing.
* **None** → uses the CPU base image.

Jetson takes precedence over the amd64 GPU variant, and GPU takes precedence over Hailo when both are available on the same host; mixing them in a single deployment is not supported. If a variant cannot be pulled (e.g. the `-jetson` image isn't published for your channel yet), Edge Core falls back to the CPU image and logs the demotion.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.