Skip to main content

​
Overview

Edge Core manages a dedicated worker container (cyberwave-worker-{env_uuid[:8]}) on each edge device. Worker scripts run inside this container with access to the Zenoh data bus, cached model weights, and all environment twin data.
One worker container runs per edge device (not per twin). Workers can consume data from all twins in the environment simultaneously.

​
Worker Directory

Place Python worker scripts in the edge config workers directory:

​
Model Requirements

Declare model dependencies in cyberwave.yml inside the workers directory:
Edge Core pre-downloads listed models before starting the worker container. Models are also auto-detected from cw.models.load("...") calls in worker Python files.

​
Picking the model format — stub

Each catalog model declares an edge runtime that tells the worker how to load the checkpoint. The runtime selector lives in the model editor (/models → “Add Model” or “Edit”), and is also accepted as a typed edge_runtime field on POST /api/v1/mlmodels and PUT /api/v1/mlmodels/{uuid}. The well-known runtimes mirror the loaders in the Cyberwave Python SDK’s cyberwave.models.runtimes registry: Custom values are accepted via the editor’s “Other” entry — used today for framework-specific identifiers like sam2, sam3, and depth_anything_v2 that don’t have an SDK loader yet but still need to round-trip through the catalog. GET /api/v1/mlmodels/edge-runtimes returns the current well-known list (no auth required) so external tools can mirror the dropdown without hard-coding it.
stub: ONNX YOLO postprocessing applies per-class non-max suppression with a default IoU threshold of 0.7 so swapping yolov8s.pt for yolov8s.onnx produces the same number of boxes per object instead of a cluster of overlapping anchors. Override per-call via model.predict(frame, iou=0.5) (stricter) or iou=1.0 (raw output, no suppression).
stub: NMS-free detectors (YOLO26’s default one-to-one head, YOLOv10) emit [max_det, 6] = [x1, y1, x2, y2, conf, class_id]; the SDK’s ONNX runtime detects this layout automatically and parses it without the anchor-decoding step. The model’s one-to-one head is supposed to already be deduplicated, but some exports (open-vocabulary heads especially) can still emit many near-duplicate boxes for one object, so the same iou NMS also runs here as a safety net. End-to-end pose / segmentation exports are not decoded yet; re-export those tasks with end2end=False until the e2e pose branch lands.
stub: Every detection result is additionally capped at max_detections_per_class (default 30) highest-confidence boxes per label, applied after NMS — a backstop for models whose suppression doesn’t fully collapse duplicates (open-vocabulary heads like YOLOE are the common case). Pass model.predict(frame, max_detections_per_class=None) to disable it. The Ultralytics runtime also defaults every call to end2end=False so real NMS runs on the one-to-many head instead of trusting a checkpoint’s self-reported “already deduplicated” flag; checkpoints with only a one-to-one head (fused .pt, TensorRT, some ONNX exports) fall back to end2end=True automatically.

​
YOLO ONNX catalog — stub

The public catalog ships ONNX twins for the most common YOLO entries — Cyberwave hosts the exported weights at https://static.cyberwave.com/ml_models/{name}.onnx so edge nodes can fetch them with no Ultralytics CDN dependency. Roughly 2× faster on CPU than the .pt build: Pick Edge Runtime: ONNX Runtime in the model editor (or metadata.edge_runtime = "onnxruntime" via the API) when registering a private fine-tune that you’ve exported via YOLO("…").export(format="onnx"). The SDK’s OnnxRuntime reads names and kpt_shape from the ONNX custom_metadata_map, so labelled detections and keypoints work out of the box. Segmentation, OBB, and classification ONNX exports are not yet supported by the SDK runtime — use the .pt (ultralytics) variants for those tasks.

​
YOLO Hailo catalog — stub

For Hailo-accelerated edge nodes (Raspberry Pi 5 + AI HAT+ and the older Pi AI Kit) the catalog also ships pre-compiled YOLO HEFs straight from the Hailo Model Zoo. Each upstream YOLO checkpoint gets two sibling rows — one per Hailo architecture — because HEFs are hardware-locked: Slugs use the _h8 / _h8l suffix on purpose: cw.models.load("yolov8s_h8") is enough to route through the SDK’s hailo runtime — no extra arguments needed. Pick the row that matches your edge target; a Hailo-8 HEF will refuse to run on a Hailo-8L (and vice versa). If you pick the wrong sibling, Edge Core catches it before the download starts: a Gate-3 preflight probes the connected accelerator with hailortcli fw-control identify and refuses to fetch a HEF whose metadata.hw_arch disagrees with the device, naming the correct sibling in the error message. The preflight is silent when hailortcli is not installed on the host — install it via apt install hailo-all (or the matching HailoRT package) to enable the check.

​
Edge speech-to-text — stub

Seeded Whisper.cpp STT models can run from Audio Track -> Call Model workflows on Raspberry Pi 4-class devices. The generated worker passes the audio chunk and seeded download_url into the local whisper_cpp runtime, so the first run downloads missing weights into the Cyberwave model cache. Downstream nodes can use text, transcribed_text, and segments. English GGML checkpoints: Tiny EN, Base EN, and Small EN (Q5_1). Multilingual checkpoints: Tiny Multilingual and Base Multilingual — hybrid deployment with edge_runtime: whisper_cpp on edge missions and whisper cloud-node fallback (openai/whisper-tiny / openai/whisper-base) on cloud missions. Use multilingual models when the spoken language is unknown or mixed. Hybrid Faster Whisper catalog entries (faster_whisper runtime) run on-device when the workflow mission executes on the edge; cloud missions use the whisper cloud node with the same OpenAI model aliases (openai/whisper-tiny → tiny.en, etc.). Prefer Faster Whisper Tiny EN for real-time English streams; use Whisper Tiny Multilingual Q5_1 for multilingual edge STT without CTranslate2.

​
Model weights resolution — stub

For each required model, Edge Core resolves weights in this order:
  1. Local cache (intact). If ~/.cyberwave/models/{model_id}/... is present and its SHA-256 matches the manifest, it is used directly.
  2. Cyberwave-hosted signed URL. If the catalog entry has a checkpoint mirror, Edge Core fetches a signed URL via GET /api/v1/mlmodels/{uuid}/weights and downloads from our private bucket.
  3. Upstream weights URL. If no Cyberwave mirror exists, Edge Core falls back to the public download_url from the catalog (e.g. an official Ultralytics release).
  4. Stale cache fallback. If every download attempt fails but the local file is intact, Edge Core returns the cached file with a warning. This keeps workers running across transient network failures and on permanently air-gapped sites.
Operators on air-gapped sites can pre-stage weights by copying them to ~/.cyberwave/models/{model_id}/. Edge Core computes a SHA-256, infers the runtime from the file extension (.pt, .onnx, .engine/.trt, .tflite, .pth, .xml), and writes a sidecar metadata.json on the next worker start. To update a pre-staged model, simply overwrite the file in place — Edge Core re-stamps the manifest from disk on the next call (no re-download attempted). Pre-staged files are never auto-overwritten by catalog updates; to force a re-download from Cyberwave, evict the model directory (rm -rf ~/.cyberwave/models/{model_id}).

​
CLI Commands

These commands are also available via cyberwave-edge-core worker … if you prefer to use the edge-core CLI directly.

​
Hot-Reload on File Changes

Edge Core monitors the workers directory every ~15 seconds. When .py files are added, removed, or modified, the worker container is automatically restarted with the updated set of workers. A minimum cool-down of 10 seconds between successive automatic restarts prevents rapid churn when files are written incrementally.

​
Health Monitoring

Edge Core continuously monitors the worker container:
  • Restart accounting: every restart is recorded with timestamp and reason.
  • Circuit-breaker: after 5 restarts in 5 minutes, automatic restarts are suppressed until the window clears. Run worker health to inspect the state.
  • Spontaneous exit detection: if the container exits without a deliberate restart, a warning is logged.

​
Performance Tuning — stub

​
Model warm-up

The worker runtime automatically runs two dummy inferences on each loaded model at startup to eliminate cold-start latency (JIT compilation, memory allocation). Cold vs warm latency is logged. You can also warm up models explicitly:

​
Frame resolution scaling

Set CYBERWAVE_WORKER_INPUT_RESOLUTION to downscale incoming frames before they reach your worker hooks. This reduces inference time on constrained devices without changing the camera driver’s publish resolution.

​
Shared memory transport

Zenoh shared-memory (SHM) transport offers zero-copy frame delivery between the camera driver and worker containers on the same host. Edge Core leaves ZENOH_SHARED_MEMORY disabled by default because SHM between Docker containers requires them to share an IPC namespace via --ipc=host, which weakens container isolation and has historically been a source of instability in production. To opt in, enable the flag on both the publisher and any same-host consumer, and give each container a shared IPC namespace (--ipc=host) and a raised locked-memory limit (memlock=-1 — the default 8 MB cap is too small for a frame pool, and without it the transport silently falls back to copy):
Under Edge Core, set ZENOH_SHARED_MEMORY=true once in its environment and it propagates the flag to the driver and worker containers; with the SDK directly, set it on each container (or pass ZenohBackend(shared_memory=True)). Zero-copy applies only to frame-sized payloads; small messages (joint states, etc.) always copy. Optional publisher knobs: If the pool can’t be created, publishing degrades to the copy path with a warning rather than failing. When it’s exhausted by a slow consumer, the ZENOH_SHM_ON_EXHAUSTION policy decides between copying and dropping the frame. The pool is allocated per frame whether or not a subscriber can use zero-copy, so enable SHM only when a same-host, SHM-capable consumer is actually reading the stream.
A healthy-looking publisher does not prove zero-copy delivery. The pool is created and used successfully even when no consumer can map it — for example if the containers omit ipc: host, each gets its own private /dev/shm. In that case the publisher reports SHM enabled and its shm_frames counter climbs, while every consumer silently receives ordinary wire copies and you get none of the latency benefit.Those counters are publish-side intent only. The single source of truth is the receiving end: a sample is genuinely zero-copy only when sample.payload.as_shm() is not None. Verify there once when rolling SHM out, rather than trusting the publisher’s stats.

​
Consuming the stream from your own process

The SHM path isn’t specific to the ML worker — any Zenoh peer on the same host can read a driver’s frames zero-copy. To consume from your own process:
  • Run it on the same host. If it’s containerized, launch it with --ipc=host (or --ipc=container:<driver>) so it shares the IPC namespace; a process running directly on the host already qualifies.
  • Enable SHM in its Zenoh session — ZENOH_SHARED_MEMORY=true via the SDK, or transport/shared_memory/enabled=true on a raw Zenoh session. A consumer that leaves SHM off still receives every frame, just copied over loopback instead of zero-copy.
  • Subscribe to the driver’s channel. With the SDK, use DataBus.subscribe(...) — it resolves the key for you. memlock=-1 is only needed if your process also publishes SHM.
Read-side note: a Python consumer (SDK or raw eclipse-zenoh) receives each frame as a copy — the zero-copy is the transport hop, so it still cuts latency and jitter but not the final ingest copy. A native C++/Rust Zenoh consumer can map the shared buffer and read it zero-copy end-to-end.

​
GPU Access

Edge Core detects the NVIDIA container runtime and passes --gpus all to the worker container when available.

​
Image variants — stub

The worker image is published in four variants on Docker Hub: The -jetson variant uses PyTorch/torchvision/torchaudio and onnxruntime-gpu wheels from the Jetson AI Lab devpi index (Orin sm_87 builds), not the amd64 -gpu variant’s index. The image base is ubuntu:24.04; NVIDIA has not yet published an l4t-jetpack image for JP7. nvidia-container-runtime injects the host’s CUDA driver at container start; the image itself bundles the matching CUDA 12.9 userspace runtime libraries (libcudart, libcublas, cuDNN) since the Jetson wheels link against system CUDA rather than bundling it. ONNX Runtime exposes CUDAExecutionProvider on Jetson, same as the -gpu variant. JetPack 6 hosts are not supported — JetPack 6’s CUDA driver predates what’s needed to run a CUDA 12.9 runtime, so only JetPack 7 hosts should pull this tag. The -hailo variant compiles HailoRT (libhailort + hailortcli + pyhailort) from the MIT-licensed upstream source at hailo-ai/hailort at a pinned tag. The LGPL-2.1 GStreamer plugin (hailonet) is left out of the build. The companion kernel driver (hailo-ai/hailort-drivers) is installed on the host (e.g. via apt install hailo-all on Raspberry Pi OS), not in the container — its minor version must match the userspace lib baked into the image. All four variants ship the same Python API. PyTorch/Ultralytics models pick the device automatically; ONNX models gain CUDAExecutionProvider only on the -gpu variant; .hef files load through the hailo runtime only on the -hailo variant. Edge Core selects the variant automatically based on host capabilities:
  • Jetson host detected (/etc/nv_tegra_release present) → appends -jetson and adds --gpus all. Wins over the -gpu branch: on Jetson the nvidia-container-runtime is present too, but the amd64 -gpu blob has the wrong libcuda.so ABI (desktop/server CUDA vs Tegra CUDA).
  • NVIDIA container runtime detected (not Jetson) → appends -gpu to the configured worker image tag and adds --gpus all.
  • /dev/hailo0 present and no GPU runtime → appends -hailo, adds --device /dev/hailo0:/dev/hailo0:rwm and --group-add hailo (on HailoRT < 4.20 hosts), and sets CYBERWAVE_REQUIRED_DEVICES=/dev/hailo0 so the image’s Gate-4 entrypoint fails fast if the passthrough is missing.
  • None → uses the CPU base image.
Jetson takes precedence over the amd64 GPU variant, and GPU takes precedence over Hailo when both are available on the same host; mixing them in a single deployment is not supported. If a variant cannot be pulled (e.g. the -jetson image isn’t published for your channel yet), Edge Core falls back to the CPU image and logs the demotion.