RL tasks

An RL Task is a workspace-scoped, optionally environment-linked entity that packages a runnable reinforcement-learning plugin. Cyberwave generates the SceneCfg from a linked environment — the same MuJoCo scene used everywhere else in the platform — and you bring the rest of the MDP and RL code as an arbitrary tree of source files. Exporting an RL task produces a ZIP that drops directly into cyberwave-rl/tasks/<task_id>/, where the existing registry discovers it and make run_cfg runs it like any other task.

Cyberwave currently supports basic mjlab: exported tasks are mjlab-compatible (scene config, action and observation terms, skrl training). We’re planning to release a richer Cyberwave mjlab extension with useful helpers in the future.

Why a separate entity?

RL tasks are deliberately kept independent of Environments. Deleting an environment detaches any linked RL tasks (sets environment to none) instead of cascading the delete, so a task keeps its source tree even after the world it was originally trained against is gone. This matches how researchers actually iterate: the SceneCfg is a moving target, while the reward, action, and observation specs are the long-lived artefacts.

Lifecycle

You can also describe a robot task from the dashboard or the environment editor’s Agent tab. Type @ (or choose Tag asset or scene object) to select an existing scene object or a catalog asset by name/full slug. Tagging adds context; it does not create an instance or start training. The assistant can look for a compatible policy or prepare a training proposal when the task is supported. Review and confirm training separately from binding or running its result. The same training card appears in chat and Policies, showing reported epochs, episodes or transitions as they advance. These counts are progress, not a success score. Scene built on export means the configured scene entities will generate the scene when you export; you do not need to save a generated scene file first. Scene source saved identifies a saved source, not a compilation or evaluation result. If the editor reports Scene setup needed, add scene entities or use Scene entities → Regenerate generated files, then prepare a fresh training request when its inputs change. Use Cancel training on that card to review and cancel only its active attempt. Cancellation prevents a new result from registering; worker shutdown can still be in progress. Existing policies and recordings remain unchanged. After failure or cancellation, Prepare a new attempt opens a fresh proposal using the previous settings. Review and confirm it before training starts. If confirmation reports changed inputs, or the proposal has expired, choose Prepare a fresh proposal. Review the current task and settings again; the rejected or expired proposal does not start training. In Live and Simulation, open Process from the strip below the scene to follow workflow runs and their recorded named steps in the central workspace. The right panel stays available for the agent. Process is read-only, works in Monitor mode, and does not verify physical task success. It is hidden in Edit and Replay.

Create

Create an RL task from /rl-tasks, or from File → Export → Create RL task from this environment in the environment editor.

Import or author source

Import an existing task ZIP (for example an in-tree task from cyberwave-rl/tasks/) to seed the source tree, or write files directly in the Files tab.

Configure entrypoints

Set the entrypoint paths — scene_cfg_path, env_cfg_path / env_cfg_factory, rl_cfg_path / rl_cfg_factory, and an optional registry_path. Cyberwave never forces a particular from … import … statement; the import wiring is yours.

Regenerate SceneCfg

Regenerate the Cyberwave-generated SceneCfg module from the linked environment. This also rewrites the spec-derived files (scene_bindings.py, actions.py, observations.py, task_export/, and the strict task spec).

Export and run

Download the ZIP and drop the folder into cyberwave-rl/tasks/. Run it with make run_cfg arg_overrides="--task=<id>" exactly like the in-tree examples.

Round-trip

Edit locally, then re-import the same folder back into Cyberwave to keep the editor in sync. The editor stays authoritative for Cyberwave-generated files.

Editor tabs

The RL task editor is organized into tabs that map onto the export contract:

Tab	What it authors
Overview	Task metadata, linked environment, and entrypoint paths.
Scene entities	The articulation entities, their simulation actuators, cameras, and contact sensors. See RL task scene sensors.
Actions	The policy-controlled action terms and their control-mode presets.
Observations	Grouped observation terms for the policy and critic heads.
RL config	skrl backend, algorithm, graph-based network builder, and trainer parameters.
Files	The full source tree, including read-only Cyberwave-generated files.
Runtime	Runtime target, accelerator, bundle version, and “Train locally” snippets.
Policies	Upload trained checkpoints, publish controllers, and run inference.

For a task configured for CPU, scene export preserves authored mesh collisions supported by native MuJoCo. GPU exports still require primitive colliders or an appropriate simplified collision variant. Selecting CPU does not make arbitrary training code CPU-compatible; the task must also provide a compatible training implementation.

Reviewing policy evidence

Training requests shows the current training phase and reported progress in both the task and assistant conversation. When a task reports curriculum stages, the same card shows its current stage and recent training-stage success rate. That rate is not the final held-out evaluation; missing results remain unreported. In Policies, select a checkpoint and open Evaluation evidence to inspect episode outcomes and available Replay links. Deployment reports separately show reported checks from completed simulation deployments, including failures. Tasks with confirmed deployment checks can report these automatically; the report identifies whether it came from the controller or a task writer. Missing recordings or checks are not a successful result. Check Final task outcome maintained as well as intermediate stages: reaching the goal briefly is not sustained success. Not reported means that outcome is unavailable, not that it passed. These reports are not held-out evaluations or hardware approval. Neither viewing evidence nor finishing training automatically binds or publishes a policy. An evaluated teaching result offers Use on this robot directly. It creates a private controller if needed and assigns it without starting motion; replacing an existing controller requires confirmation. Publish to catalog is separate and does not require assigning the policy to a robot first. Replay displays only recorded channels: object motion appears when the episode includes it; older joint-only recordings do not demonstrate object movement. Policy evidence Replay links frame the recorded robot on entry. You can still orbit, zoom, play and scrub the timeline; this does not start a controller.

Author from Python (SDK)

Everything the editor tabs produce can also be authored from the Python SDK — useful for reproducible setup scripts. The RLTaskClient covers every tab via its set_*_spec methods, so a script and the dashboard produce the same specs. The Actions, Observations, and RL config tabs also ship make_* builder helpers; the command tabs are authored as plain dicts passed to set_training_command_spec / set_inference_command_spec (see step 4b below). The example below is the OpenArm pick-up task: a fixed-base arm, a table, a free-floating cube, and a wrist depth camera. It is the same scene used in the full RL task walkthrough, trimmed to the essentials.

from cyberwave import Cyberwave
from cyberwave.rl_tasks import (
    RLTaskClient,
    make_action_spec,
    make_position_delta_action,
    make_observation_spec,
    make_joint_observation,
    make_previous_action_observation,
    make_default_ppo_rlmodule_config,
)

cw = Cyberwave()            # reads CYBERWAVE_BASE_URL + CYBERWAVE_API_KEY
rl = RLTaskClient(cw)
task_uuid = "<rl-task-uuid>"   # created via POST /api/v1/rl-tasks

# Match the arm + finger joints (twin-namespaced at runtime).
ARM_JOINTS = [r".*__openarm_left_joint[1-7]$", r".*__openarm_left_finger_joint[12]$"]

# 1. Scene entities: which twin is which mjlab entity.
rl.assign_articulation_entity(
    task_uuid, twin_uuid="<openarm-twin-uuid>", name="robot", base_type="fixed",
    actuators=[{"type": "xml_position", "target_names_expr": ARM_JOINTS}],
)
rl.assign_rigid_entity(task_uuid, twin_uuid="<table-twin-uuid>", name="support_box", base_type="fixed")
rl.assign_rigid_entity(task_uuid, twin_uuid="<cube-twin-uuid>", name="cube", base_type="free")

# 2. Actions: one position-delta term driving the arm + fingers.
rl.set_action_spec(task_uuid, make_action_spec([
    make_position_delta_action("joint_delta", entity="robot", target_names_expr=ARM_JOINTS, scale=0.04),
]))

# 3. Observations: a single "policy" group the actor and critic share.
rl.set_observation_spec(task_uuid, make_observation_spec({"policy": [
    make_joint_observation("joint_pos", kind="position", entity="robot", target_names_expr=ARM_JOINTS),
    make_joint_observation("joint_vel", kind="velocity", entity="robot", target_names_expr=ARM_JOINTS),
    make_previous_action_observation("actions"),
]}))

# 4. RL config: PPO with the default shared-MLP network.
rl.set_rl_config_spec(task_uuid, make_default_ppo_rlmodule_config())

# 4b. (Optional) Command specs: a target generator your observations read.
#     Opt-in and split by lifecycle — tasks with no command fall back to a
#     null manager. The TRAINING spec drives mjlab's CommandManager (sampled
#     goals, resampled each window); the INFERENCE spec drives the live/
#     inference command surface (externally supplied targets).
#     A code-backed term names its uploaded source module by dotted path
#     (``commands.viewpoint`` → ``commands/viewpoint.py``) and the class it binds.
rl.set_training_command_spec(task_uuid, {
    "schema_version": 1,
    "commands": [{
        "name": "viewpoint", "type": "custom",
        "module": "commands.viewpoint", "symbol": "ViewpointCommandCfg",
        "resampling_time_range": [4.0, 4.0],
    }],
})
rl.set_inference_command_spec(task_uuid, {
    "schema_version": 1,
    "commands": [{
        "name": "viewpoint", "type": "custom",
        "module": "commands.viewpoint", "symbol": "ViewpointCommandCfg",
        "payload_schema": {"target_position": "vec3"},
        "source": {"kind": "mqtt"},
    }],
})

# 5. Regenerate the Cyberwave-generated files from the specs + linked environment.
rl.regenerate_scene_cfg(task_uuid)

rl.get_orchestration_hints(task_uuid) returns the scene-derived dropdown data (articulated entities, actuator groups, actuated vs passive joint names, available sensors) the editor uses to populate its pickers — call it first to discover the exact entity and joint names before authoring specs.

For the complete version — scaffolding the workspace, environment, and twins, wiring a wrist depth camera and contact sensors, training, and deploying the policy — follow the RL task walkthrough.

Actions

Task input is not robot output

A goal or requested walking speed is an input to a policy. The policy’s actions are its output to the robot or another controller. These are separate contracts: a walking policy can accept speed and produce joint targets; a navigation policy can produce body velocities for an onboard controller whose actuators are not accessible. Match both contracts to the selected runtime. Simulation actuators in a robot model do not prove that its physical driver exposes them. Cyberwave’s shared adapter has a simulation-only body-velocity output port. For an explicitly authored aerial policy, controller startup checks its frozen interface against the lower controller observed in the running simulator. Unsupported or mismatched interfaces are rejected. Do not treat an evaluated checkpoint or a URDF as hardware readiness. The Velocity preset below means joint velocity, not base/drone velocity. Keep incompatible outputs explicit instead of converting them silently. Authored command-level flight tasks reuse custom action/observation terms and the existing asynchronous training adapter. Their lower controller, output units, frame, limits and cadence must match the training version. Changing that controller requires a fresh task version and evaluation. This adapter support does not yet establish a verified end-to-end drone deployment in the dashboard; inspect actual deployment acceptance and Replay separately from training results. Flight checks use measured robot pose and velocity, without requiring a fake joint-state stream. They require the goal to remain achieved, not merely be crossed once. A missing root feed, a different flight controller or unexplained applied forces makes the result fail. These checks are implementation support; end-to-end drone dashboard verification is still pending.

Joint action presets

Action terms are authored from five control-mode presets:

Preset	Spec value	Formula	Requires actuator
Position delta	`position_delta`	`target = q_now + baseline_delta + action × scale`	`xml_position`
Smoothed position delta	`smoothed_position_delta`	same as `position_delta`, delta ramped linearly over the decimation substeps	`xml_position`
Position target	`position`	`target = action × scale + offset` (anchored at joint home)	`xml_position`
Velocity	`velocity`	`target = action × scale + offset`	`xml_velocity` or `xml_position`
Effort / torque	`effort`	`target = action × scale + offset`	`xml_motor`

position, velocity, and effort mirror the Isaac Lab / mjlab JointPositionActionCfg / JointVelocityActionCfg / JointEffortActionCfg math one-to-one — exactly the target = action × scale + offset that mjlab implements in BaseAction.process_actions. position_delta is Cyberwave’s own preset: baseline_delta is an explicit per-step bias baked into the target, so an action of zero can drift the joints toward a goal pose. smoothed_position_delta is the same delta, but spread evenly across the policy step’s physics substeps for a smoother velocity profile. A single Use default joint offset checkbox mirrors mjlab’s use_default_offset: when on, mjlab replaces offset with each joint’s default_joint_pos (or default_joint_vel) at runtime so action zero returns the joint to its home pose. Every numeric field (scale, offset, baseline_delta) has a Per joint toggle that switches it between a uniform scalar and a {joint_name: value} mapping. The backend rejects per-joint keys that aren’t in the action’s selected joints, so typos surface at save time. Legacy symbolic offsets ("default", "zero", "none") keep round-tripping unchanged. The Scene tab is where you enable simulation actuators for an entity (which Xml*ActuatorCfg wrapper drives each joint); the Actions tab is where you pick the policy-controlled subset of those actuators. Each action row renders a schema-backed joint table (Joint, Control type, Position min/max, Max velocity, Max torque/force, KP, KD, Selected) with Select all / Clear. Passive joints, spring-only actuators, and joints driven by an incompatible wrapper appear greyed out with a hint. Incompatible presets are hidden rather than shown disabled. Concrete joint names are stored in target_names_expr; a collapsed Advanced — custom regex disclosure lets imported or legacy specs keep regex patterns.

One action per joint. Every (entity, joint) pair can be assigned to at most one action term — the policy emits a single value per actuated DOF. The Actions tab enforces this in three places: joints already controlled by another row render disabled with an “owned by …” pill; Select all only picks unclaimed joints; and PUT /api/v1/rl-tasks/{uuid}/actions rejects overlapping payloads with an error naming both action names and a sample of overlapping joints. To split a robot across two terms (for example arm + gripper), give each term a disjoint subset of joints.

Observations

Observation terms are organized into named groups (policy, critic, …) and drawn from a catalog:

Term	Description
`joint_position`	Joint positions.
`joint_velocity`	Joint velocities.
`joint_effort`	Joint efforts / torques.
`link_position`	Body position; pick among `x` / `y` / `z` components.
`link_pose`	Body pose: `x` / `y` / `z` plus a single `quat` token keeping the full `(qw, qx, qy, qz)` tuple grouped.
`link_velocity`	Body twist: `x` / `y` / `z` linear plus `rx` / `ry` / `rz` angular.
`camera`	Depth or color from a sensor configured on a scene entity.
`previous_action`	The previous action tensor.
`custom`	User `module.symbol` plus literal kwargs.

Link terms pick bodies from a scene-derived checkbox list and expose Components toggles so you only export the axes the policy needs. Joint observation terms reuse the shared Joint target picker and — unlike actions — may include both actuated and passive joints, with “All actuated”, “All joints”, “Passive joints”, and “Pick specific joints” exposed directly; “Custom regex” sits behind a small Advanced disclosure.

Shared policy/critic (default)
Separate policy and critic

One group is authored, and the runtime aliases the missing critic group onto policy so both heads see the same tensor.

Observation rows show the term type as a compact chip next to the editable name; the type is fixed after creation (remove and re-add to change it).

RL config

The RL config tab persists the skrl backend, the algorithm (PPO or SAC), a structured graph-based Network builder, the trainer parameters (timesteps, intervals, seed), and the algorithm-config knobs (learning rate, discount factor, ratio clip, batch size, …). Switching algorithm seeds matching defaults. Experiment paths (experiment.name / experiment.directory) default to values derived from the RL task slug and are shown read-only; toggle Override experiment paths to write custom values.

Network builder

The network builder is graph-first: it renders the network as a DAG of typed nodes on an inline canvas, with one canvas per role for the active topology.

shared uses the same architecture for both heads. For PPO this yields a true shared-trunk model (SharedRLModelCfg); for SAC it instantiates two independent modules (policy + Q) with the same block spec, because SAC heads can’t share weights.
separated lets you author one graph per role (policy / value for PPO, policy / q for SAC).

Top-level presets are graph templates (for example PPO – shared MLP, SAC – separated MLP) that materialise the equivalent observation → trunk → head topology directly on the canvas. The palette covers inputs (observation_input), dense layers (mlp, residual_mlp), transforms (activation, layer_norm), recurrent layers (rnn, gru, lstm), vision (cnn_backbone with NatureCNN / IMPALA presets), merge nodes (concat, residual_add), and a role/algorithm-aware Heads section that surfaces actor_head + value_head for PPO and actor_head + q_head for SAC. Drag the canvas background to pan, drag a node to reposition it (positions persist in graphs[role].layout), and click an edge to delete it. Clicking a node opens an inline inspector below the canvas with the outputs picker (features_node_id / head_node_id) and the per-node editor. An Open graph builder button pops the same graph into a fullscreen dialog for serious authoring. observation_input nodes pick a group and a subset of term names from the Observations tab. Each term shows read-only tensor metadata — an image-like / flat badge and a short kind label — so the builder can route image-like terms into a cnn_backbone and reject flat terms feeding one. Gaussian log-std controls (initial_log_std, min_log_std, max_log_std) live in the actor head inspector and apply only to the stochastic Gaussian policy.

Legacy {kind, hidden_units, activation} and architecture: "mlp" / "residual_mlp" / "custom" specs are transparently promoted to a graph on load, so older tasks just render. At runtime, from runners.skrl.network_builder import build_skrl_model_cfg maps the persisted spec onto rlmodule.MlpCfg / ResidualMlpCfg / SharedRLModelCfg / SeparatedRLModelCfg for the skrl runner.

Verify before saving. The Verify button next to Save RL config runs the same validation the save path uses — without persisting — and reports the first problem (for example “mlp needs exactly 1 input; got 2 — merge multiple branches with a Concat node first”). The inline canvas also flags structural issues live as red error badges. A passing Verify guarantees Save will succeed.

Runtime: target, accelerator, and bundle

Open the Runtime tab to choose how the exported plugin runs:

Pick a runtime_target — cyberwave-rl-experimental (plain local Python, no Docker) or cyberwave-rl (the dockerized stack).
Pick a runtime_accelerator — gpu is the default; cpu is the smoke-friendly opt-in.
Pin a single runtime_bundle version (default latest). The bundle ships mjlab + the chosen runtime + skrl together, so callers only ever pin one version.
Configure the entrypoints (scene_cfg_path, env_cfg_path / env_cfg_factory, rl_cfg_path / rl_cfg_factory, optional registry_path) inline.
Copy the matching Train locally snippet for the selected target and accelerator.

Both runtimes consume the same exported plugin contract, and the runtime spec (target + accelerator + versions + policy interface) is persisted on the RLTask and forwarded to checkpoints and inference dispatch with a task → checkpoint → launch merge order.

cyberwave-rl (Docker)
cyberwave-rl-experimental (local Python)

make start
make run_cfg arg_overrides="--task=<task_id>"

Real training expects a CUDA GPU.

# CPU smoke test
python -m train_from_cfg --device=cpu --num_envs=2 --max_steps=10

# Full GPU training (same defaults as cyberwave-rl)
python -m train_from_cfg

Policies and inference

The Policies tab is the post-training counterpart to the Runtime tab’s “Train locally” snippets: upload a trained checkpoint, publish it as a controller, and run inference. Inference deliberately does not run MuJoCo — the worker loops Cyberwave joint state through the policy and publishes commands over MQTT. See the dedicated Uploading policies page for the full checkpoint upload, publish, and inference flow, and Online controllers for serving a published policy.

Managed training: `training_options.json`

A task’s async adapter declares which training strategies cw_request_skill_teaching (or POST /api/v1/skill-teaching-requests) can dispatch by shipping training_options.json next to training.py. This file is the contract between your adapter and the platform: it names each supported strategy, the JSON-Schema-constrained hyperparameters it accepts, and the defaults applied when a caller doesn’t override them.

Fetch the resolved options directly instead of guessing the shape: GET /api/v1/rl-tasks/{uuid}/training-options, cw_inspect_rl_tasks(operation="get"), or the proposal.training.options field on a teaching-request proposal all return the current, validated training_options.json for a task.

Shape

{
  "schema_version": 1,
  "strategies": {
    "<strategy_key>": {
      "parameters": { "...": "a bounded JSON Schema object" },
      "defaults": { "...": "values that satisfy 'parameters'" }
    }
  }
}

Field	Requirement
`schema_version`	Must be the integer `1`.
`strategies`	An object keyed by strategy key (not a list) — 1 to 16 entries. Each key matches `^[a-z][a-z0-9_.:-]{0,63}$` (for example `rl:ppo`, `imitation:behavior_cloning`).
`strategies[key].parameters`	A JSON Schema object (`type: "object"`, `additionalProperties: false`, at most 64 properties, nested at most 8 levels). Only `type`, `properties`, `additionalProperties`, `required`, `items`, `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`, `minItems`, `maxItems`, `minLength`, `maxLength`, `enum`, `title`, `description` are accepted — no `$ref`, `pattern`, or schema-composition keywords, so the descriptor can’t fetch remote refs or blow up validation.
`strategies[key].defaults`	An object satisfying `parameters`; frozen into the job when a caller doesn’t override a field.

The whole file is capped at 32 KiB and never executed — it’s parsed as data, so authoring it requires no backend source access.

Worked example

This is the real training_options.json for the imitation:behavior_cloning strategy shipped with the in-tree cyberwave-rl/examples/kinova_gen3_pick_place example task:

{
  "schema_version": 1,
  "strategies": {
    "imitation:behavior_cloning": {
      "defaults": {
        "hidden_sizes": [128, 128],
        "epochs": 300,
        "batch_size": 256,
        "learning_rate": 0.0003,
        "weight_decay": 0.000001,
        "validation_fraction": 0.1,
        "seed": 0,
        "augmentation": {"replicas": 4, "seed": 73}
      },
      "parameters": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "hidden_sizes": {"type": "array", "minItems": 1, "maxItems": 4, "items": {"type": "integer", "minimum": 1, "maximum": 1024}},
          "epochs": {"type": "integer", "minimum": 1, "maximum": 10000},
          "batch_size": {"type": "integer", "minimum": 1, "maximum": 8192},
          "learning_rate": {"type": "number", "exclusiveMinimum": 0, "maximum": 1},
          "weight_decay": {"type": "number", "minimum": 0, "maximum": 1},
          "validation_fraction": {"type": "number", "exclusiveMinimum": 0, "exclusiveMaximum": 1},
          "seed": {"type": "integer", "minimum": 0, "maximum": 4294967295},
          "augmentation": {
            "type": "object",
            "additionalProperties": false,
            "required": ["replicas", "seed"],
            "properties": {
              "replicas": {"type": "integer", "minimum": 0, "maximum": 32},
              "seed": {"type": "integer", "minimum": 0, "maximum": 4294967295}
            }
          }
        }
      }
    }
  }
}

A caller then requests this strategy with a partial strategy_config, for example {"epochs": 10}; the backend merges it onto defaults (nested objects like augmentation are replaced as a whole, not deep-merged) and validates the result against parameters before freezing it into the job.

Validation errors are field-specific

Every check names the offending path instead of stopping at the first top-level shape mismatch, so you can iterate directly against the error:

training_options.json.strategies must be an object keyed by strategy key (e.g. {"rl:ppo": {...}}), got array — a common first guess is authoring strategies as a list of strategy objects instead of a dict keyed by strategy key.
strategies['rl:ppo'] is missing required field(s): defaults — a strategy entry must declare exactly parameters and defaults, nothing else.
strategies['rl:ppo'].parameters.properties.epochs.type must be one of [...] — nested schema problems point at the exact property.
strategies['rl:ppo'].defaults.epochs: 0 is less than the minimum of 1 — invalid defaults report the dotted field path and the underlying JSON Schema message.

A missing_training_input requirement reported by cw_request_skill_teaching or a teaching-request proposal carries this same message, so the specific field is visible without backend source access.

What’s in the exported ZIP?

The export is a minimal Python package that drops into cyberwave-rl/tasks/<task_id>/:

__init__.py — generated header that imports your registry (or the default one).
Your source tree — exactly as you authored it, with arbitrary nesting and free file structure.
scene_cfg.py (at the configured scene_cfg_path) — the Cyberwave-generated SceneCfg, regenerated from the linked environment.
scene_bindings.py — semantic role / twin / sensor aliases plus ACTION_SPEC, OBSERVATION_SPEC, and RL_CONFIG_SPEC.
actions.py (make_actions()) and observations.py (make_observations()) — the named wrappers.
task_export/ — the runtime adapters those wrappers import.
cyberwave_task_spec.py — the strict, canonical Python interchange for scene entities.
cyberwave_export.json — the export manifest (records runtime target, accelerator, and version pins).
MuJoCo assets — one merged models/mujoco_scene.xml, one per-entity models/<name>.xml carved out of it at build time (so each EntityCfg.spec_fn can MjSpec.from_file its own slice), and the shared models/assets/ meshes.
registry.py — a fallback registry wiring env_cfg / rl_cfg factories into register_cyberwave_task, used only when you don’t ship your own registry_path.

Cyberwave-generated vs user-written

scene_bindings.py is the stable interface between Cyberwave-generated scene metadata and user-written training code. It exposes role aliases (ROBOT_ENTITY, OBJECT_ENTITY, SUPPORT_ENTITY), twin namespace prefixes, namespaced joint-position regex keys, actuator target regexes, camera attach paths, contact sensor names, and a body_pattern(entity, suffix) helper. Consume these constants from env_cfg.py so renaming a twin or moving a camera in the composer regenerates the bindings without touching MDP code. The editor owns Scene entities, Actions, Observations, RL config, and all generated artefacts. Task-local Python — reward functions, reset / termination logic, metrics, command shaping, and custom observation functions — stays user-written under task_sources/. Three execution-only modules in cyberwave-rl consume the persisted specs so user-written files only have to provide rewards, resets, terminations, commands, and custom observation functions:

from task_export.action_specs import make_action_cfgs          # ACTION_SPEC → {name: ActionTermCfg}
from task_export.observation_specs import make_observation_groups  # OBSERVATION_SPEC → {group: ObservationGroupCfg}
from task_export.rl_config_specs import make_rl_cfg_from_spec   # RL_CONFIG_SPEC → ReinforcementLearningCfg

ZIP import is intentionally document-only for the generated paths: scene_cfg.py, scene_bindings.py, and cyberwave_task_spec.py are skipped on import and the response carries an explicit warning. Scene entities are not auto-restored from a bundled cyberwave_task_spec.py — re-compose them in the editor. The export ZIP always carries fresh bytes for the Cyberwave-generated files, so re-running the composer (or hitting regenerate-scene-cfg) is enough to refresh the artefact.

API surface

All endpoints are under /api/v1/rl-tasks and require authentication.

Method & path	Purpose
`GET/POST /api/v1/rl-tasks`	List / create RL tasks.
`GET/PUT/DELETE /api/v1/rl-tasks/{uuid}`	Read, update, soft-delete (independent of the linked environment).
`POST /api/v1/rl-tasks/{uuid}/clone`	Duplicate the task definition (metadata, runtime contract, source files, scene entities) into a new row; checkpoints and inference runs are not cloned.
`GET /api/v1/rl-tasks/{uuid}/source-files`	List and read/write/delete the user-written source tree (content endpoints live under the same prefix). Unsafe paths and uncompilable Python are rejected.
`POST /api/v1/rl-tasks/{uuid}/regenerate-scene-cfg`	Rewrite the SceneCfg at the configured path plus every spec-derived file (task spec, `scene_bindings.py`, `actions.py`, `observations.py`, `task_export/`). This is the Files-tab “Regenerate all”.
`POST /api/v1/rl-tasks/{uuid}/regenerate-task-spec`	Regenerate only the spec-derived files (no SceneCfg rebuild); used by the Actions/Observations “Save & regenerate”.
`POST /api/v1/rl-tasks/{uuid}/generate-env-cfg`	Write a runnable starter `env_cfg.py` (user-written). Returns 409 if one already exists unless `overwrite=true`.
`GET /api/v1/rl-tasks/{uuid}/export.zip`	Download the runnable plugin ZIP.
`POST /api/v1/rl-tasks/import.zip`	Import a ZIP as a new task.
`POST /api/v1/rl-tasks/{uuid}/import.zip`	Refresh an existing task from a ZIP (generated files skipped; a bundled task spec surfaces a warning).
`GET/PUT /api/v1/rl-tasks/{uuid}/actions`	Read / write the action spec.
`GET/PUT /api/v1/rl-tasks/{uuid}/observations`	Read / write the observation spec.
`GET/PUT /api/v1/rl-tasks/{uuid}/rl-config`	Read / write the RL config spec.
`GET /api/v1/rl-tasks/{uuid}/orchestration-hints`	Scene-derived dropdown data: articulation entities, actuator groups and types per entity, declared joint/body names, actuated vs passive joint lists, and configured camera / contact sensors.
`GET /api/v1/rl-tasks/{uuid}/training-options`	Read the task’s validated `training_options.json` (see Managed training).
`GET/POST/PUT /api/v1/rl-tasks/{uuid}/scene-entities`	List, create, and update scene entities.
`PATCH/DELETE /api/v1/rl-tasks/{uuid}/scene-entities/{entity_uuid}`	Partially update or delete a single scene entity.

The RLTask response surfaces the action, observation, and RL-config specs as action_spec, observation_spec, and rl_config_spec JSON fields. It also surfaces the two command specs split by lifecycle — training_command_spec and inference_command_spec — each paired with an explicit lifecycle switch (training_command_setup_enabled / inference_command_setup_enabled). Unlike the actions/observations/rl-config specs (which have dedicated GET/PUT endpoints), the command specs and their switches are read from the RLTask response and written through POST/PUT /api/v1/rl-tasks/{uuid} (mirrored by the SDK’s set_training_command_spec / set_inference_command_spec).

The spec GET, the regenerate endpoints, and the export ZIP all produce byte-identical output for the same scene-entity rows, so re-reading and re-exporting are always consistent. If a body can’t be resolved while splitting the MuJoCo scene, the request fails with HTTP 400 naming the failing entity and the selectors that were tried — check the twin selectors and re-export.

Inference Commands

RL tasks can declare inference commands that define a live control surface for deployed policies. Each command declares:

A payload schema (typed fields like target_position: vec3, object_position: vec3)
A source binding (transport + topic, e.g. MQTT)

When you deploy an RL task controller, the Online Controllers panel shows schema-driven inputs matching your inference command spec. Submitting values sends them to the running controller over MQTT, updating the policy’s command observations in real time. Common use cases:

Position controllers: Set a target EE pose for reach tasks
Waypoint controllers: Set the detected object center for inspection trajectory generation

Commands are authored as first-class terms, like observation terms: each term references a CommandTermCfg subclass (module.symbol) implemented in the task’s env_cfg.py. Add and edit terms in the editor’s Commands tab (term name, module.symbol, lifecycle fields, kwargs) or from the SDK (set_training_command_spec / set_inference_command_spec); the term’s Python implementation lives in the task’s source files. The Commands tab has two explicit lifecycle switches — Training command setup and Inference command setup. Turning one off clears its spec by design (a disabled lifecycle is “deliberately off”, not a forgotten upload); turning it on requires at least one code-backed term. From the SDK, pass enabled= to set_training_command_spec / set_inference_command_spec to set a switch explicitly, or omit it and the switch is derived from whether the spec has commands.

Where to go next

RL Task Walkthrough

Build the OpenArm pick-up task end-to-end in Python, from empty workspace to deployed policy.

Uploading Policies

Upload trained checkpoints, publish them as controllers, and run inference.

Online controllers

Serve a published policy as a live controller.

RL Task Scene Sensors

Configure cameras and contact sensors on scene entities.

Simulation

Build the MuJoCo environment that backs your RL task’s SceneCfg.

Get started

AI agents

AI models

Control

Learn

Run on edge or cloud

Platform

For integrators

Help

Why a separate entity?

Lifecycle

Editor tabs

Reviewing policy evidence

Author from Python (SDK)

Actions

Task input is not robot output

Joint action presets

Observations

RL config

Network builder

Runtime: target, accelerator, and bundle

Policies and inference

Managed training: `training_options.json`

Shape

Worked example

Validation errors are field-specific

What’s in the exported ZIP?

Cyberwave-generated vs user-written

API surface

Inference Commands

Where to go next

RL Task Walkthrough

Uploading Policies

Online controllers

RL Task Scene Sensors

Simulation

​Why a separate entity?

​Lifecycle

​Editor tabs

​Reviewing policy evidence

​Author from Python (SDK)

​Actions

​Task input is not robot output

​Joint action presets

​Observations

​RL config

​Network builder

​Runtime: target, accelerator, and bundle

​Policies and inference

​Managed training: training_options.json

​Shape

​Worked example

​Validation errors are field-specific

​What’s in the exported ZIP?

​Cyberwave-generated vs user-written

​API surface

​Inference Commands

​Where to go next

RL Task Walkthrough

Uploading Policies

Online controllers

RL Task Scene Sensors

Simulation

Why a separate entity?

Lifecycle

Editor tabs

Reviewing policy evidence

Author from Python (SDK)

Actions

Task input is not robot output

Joint action presets

Observations

RL config

Network builder

Runtime: target, accelerator, and bundle

Policies and inference

Managed training: `training_options.json`

Shape

Worked example

Validation errors are field-specific

What’s in the exported ZIP?

Cyberwave-generated vs user-written

API surface

Inference Commands

Where to go next