> ## Documentation Index > Fetch the complete documentation index at: https://docs.cyberwave.com/llms.txt > Use this file to discover all available pages before exploring further. # RL Tasks > Package a Cyberwave environment plus your MDP and RL code as a runnable mjlab/skrl plugin: author actions, observations, and a graph-based network in the dashboard, then export a ZIP that drops into cyberwave-rl/tasks/. An **RL Task** is a workspace-scoped, optionally environment-linked entity that packages a runnable reinforcement-learning plugin. Cyberwave generates the **SceneCfg** from a linked environment — the same MuJoCo scene used everywhere else in the platform — and you bring the rest of the MDP and RL code as an arbitrary tree of source files. Exporting an RL task produces a ZIP that drops directly into `cyberwave-rl/tasks//`, where the existing registry discovers it and `make run_cfg` runs it like any other task. Cyberwave currently supports **basic mjlab**: exported tasks are mjlab-compatible (scene config, action and observation terms, skrl training). We're planning to release a richer Cyberwave mjlab extension with useful helpers in the future. ## Why a separate entity? RL tasks are deliberately kept independent of [Environments](/overview/features/environment). Deleting an environment **detaches** any linked RL tasks (sets `environment` to none) instead of cascading the delete, so a task keeps its source tree even after the world it was originally trained against is gone. This matches how researchers actually iterate: the SceneCfg is a moving target, while the reward, action, and observation specs are the long-lived artefacts. ## Lifecycle Create an RL task from `/rl-tasks`, or from **File → Export → Create RL task from this environment** in the environment editor. Import an existing task ZIP (for example an in-tree task from `cyberwave-rl/tasks/`) to seed the source tree, or write files directly in the **Files** tab. Set the entrypoint paths — `scene_cfg_path`, `env_cfg_path` / `env_cfg_factory`, `rl_cfg_path` / `rl_cfg_factory`, and an optional `registry_path`. Cyberwave never forces a particular `from … import …` statement; the import wiring is yours. Regenerate the Cyberwave-generated SceneCfg module from the linked environment. This also rewrites the spec-derived files (`scene_bindings.py`, `actions.py`, `observations.py`, `task_export/`, and the strict task spec). Download the ZIP and drop the folder into `cyberwave-rl/tasks/`. Run it with `make run_cfg arg_overrides="--task="` exactly like the in-tree examples. Edit locally, then re-import the same folder back into Cyberwave to keep the editor in sync. The editor stays authoritative for Cyberwave-generated files. ## Editor tabs The RL task editor is organized into tabs that map onto the export contract: | Tab | What it authors | | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Overview** | Task metadata, linked environment, and entrypoint paths. | | **Scene entities** | The articulation entities, their simulation actuators, cameras, and contact sensors. See [RL task scene sensors](/feature-reference/rl-task-scene-sensors). | | **Actions** | The policy-controlled action terms and their control-mode presets. | | **Observations** | Grouped observation terms for the policy and critic heads. | | **RL config** | skrl backend, algorithm, graph-based network builder, and trainer parameters. | | **Files** | The full source tree, including read-only Cyberwave-generated files. | | **Runtime** | Runtime target, accelerator, bundle version, and "Train locally" snippets. | | **Policies** | Upload trained checkpoints, publish controllers, and run inference. | ## Author from Python (SDK) Everything the editor tabs produce can also be authored from the Python SDK — useful for reproducible setup scripts. The `RLTaskClient` covers every tab via its `set_*_spec` methods, so a script and the dashboard produce the same specs. The Actions, Observations, and RL config tabs also ship `make_*` builder helpers; the command tabs are authored as plain dicts passed to `set_training_command_spec` / `set_inference_command_spec` (see step 4b below). The example below is the **OpenArm pick-up** task: a fixed-base arm, a table, a free-floating cube, and a wrist depth camera. It is the same scene used in the full [RL task walkthrough](/feature-reference/rl-task-walkthrough), trimmed to the essentials. ```python theme={null} from cyberwave import Cyberwave from cyberwave.rl_tasks import ( RLTaskClient, make_action_spec, make_position_delta_action, make_observation_spec, make_joint_observation, make_previous_action_observation, make_default_ppo_rlmodule_config, ) cw = Cyberwave() # reads CYBERWAVE_BASE_URL + CYBERWAVE_API_KEY rl = RLTaskClient(cw) task_uuid = "" # created via POST /api/v1/rl-tasks # Match the arm + finger joints (twin-namespaced at runtime). ARM_JOINTS = [r".*__openarm_left_joint[1-7]$", r".*__openarm_left_finger_joint[12]$"] # 1. Scene entities: which twin is which mjlab entity. rl.assign_articulation_entity( task_uuid, twin_uuid="", name="robot", base_type="fixed", actuators=[{"type": "xml_position", "target_names_expr": ARM_JOINTS}], ) rl.assign_rigid_entity(task_uuid, twin_uuid="", name="support_box", base_type="fixed") rl.assign_rigid_entity(task_uuid, twin_uuid="", name="cube", base_type="free") # 2. Actions: one position-delta term driving the arm + fingers. rl.set_action_spec(task_uuid, make_action_spec([ make_position_delta_action("joint_delta", entity="robot", target_names_expr=ARM_JOINTS, scale=0.04), ])) # 3. Observations: a single "policy" group the actor and critic share. rl.set_observation_spec(task_uuid, make_observation_spec({"policy": [ make_joint_observation("joint_pos", kind="position", entity="robot", target_names_expr=ARM_JOINTS), make_joint_observation("joint_vel", kind="velocity", entity="robot", target_names_expr=ARM_JOINTS), make_previous_action_observation("actions"), ]})) # 4. RL config: PPO with the default shared-MLP network. rl.set_rl_config_spec(task_uuid, make_default_ppo_rlmodule_config()) # 4b. (Optional) Command specs: a target generator your observations read. # Opt-in and split by lifecycle — tasks with no command fall back to a # null manager. The TRAINING spec drives mjlab's CommandManager (sampled # goals, resampled each window); the INFERENCE spec drives the live/ # inference command surface (externally supplied targets). # A code-backed term names its uploaded source module by dotted path # (``commands.viewpoint`` → ``commands/viewpoint.py``) and the class it binds. rl.set_training_command_spec(task_uuid, { "schema_version": 1, "commands": [{ "name": "viewpoint", "type": "custom", "module": "commands.viewpoint", "symbol": "ViewpointCommandCfg", "resampling_time_range": [4.0, 4.0], }], }) rl.set_inference_command_spec(task_uuid, { "schema_version": 1, "commands": [{ "name": "viewpoint", "type": "custom", "module": "commands.viewpoint", "symbol": "ViewpointCommandCfg", "payload_schema": {"target_position": "vec3"}, "source": {"kind": "mqtt"}, }], }) # 5. Regenerate the Cyberwave-generated files from the specs + linked environment. rl.regenerate_scene_cfg(task_uuid) ``` `rl.get_orchestration_hints(task_uuid)` returns the scene-derived dropdown data (articulated entities, actuator groups, actuated vs passive joint names, available sensors) the editor uses to populate its pickers — call it first to discover the exact entity and joint names before authoring specs. For the complete version — scaffolding the workspace, environment, and twins, wiring a wrist depth camera and contact sensors, training, and deploying the policy — follow the [RL task walkthrough](/feature-reference/rl-task-walkthrough). ## Actions Action terms are authored from five control-mode presets: | Preset | Spec value | Formula | Requires actuator | | --------------------------- | ------------------------- | ---------------------------------------------------------------------------- | -------------------------------- | | **Position delta** | `position_delta` | `target = q_now + baseline_delta + action × scale` | `xml_position` | | **Smoothed position delta** | `smoothed_position_delta` | same as `position_delta`, delta ramped linearly over the decimation substeps | `xml_position` | | **Position target** | `position` | `target = action × scale + offset` (anchored at joint home) | `xml_position` | | **Velocity** | `velocity` | `target = action × scale + offset` | `xml_velocity` or `xml_position` | | **Effort / torque** | `effort` | `target = action × scale + offset` | `xml_motor` | `position`, `velocity`, and `effort` mirror the Isaac Lab / mjlab `JointPositionActionCfg` / `JointVelocityActionCfg` / `JointEffortActionCfg` math one-to-one — exactly the `target = action × scale + offset` that mjlab implements in `BaseAction.process_actions`. `position_delta` is Cyberwave's own preset: `baseline_delta` is an explicit per-step bias baked into the target, so an action of zero can drift the joints toward a goal pose. `smoothed_position_delta` is the same delta, but spread evenly across the policy step's physics substeps for a smoother velocity profile. A single **Use default joint offset** checkbox mirrors mjlab's `use_default_offset`: when on, mjlab replaces `offset` with each joint's `default_joint_pos` (or `default_joint_vel`) at runtime so action zero returns the joint to its home pose. Every numeric field (`scale`, `offset`, `baseline_delta`) has a **Per joint** toggle that switches it between a uniform scalar and a `{joint_name: value}` mapping. The backend rejects per-joint keys that aren't in the action's selected joints, so typos surface at save time. Legacy symbolic offsets (`"default"`, `"zero"`, `"none"`) keep round-tripping unchanged. The **Scene** tab is where you enable **simulation actuators** for an entity (which `Xml*ActuatorCfg` wrapper drives each joint); the **Actions** tab is where you pick the policy-controlled subset of those actuators. Each action row renders a schema-backed joint table (Joint, Control type, Position min/max, Max velocity, Max torque/force, KP, KD, Selected) with **Select all** / **Clear**. Passive joints, spring-only actuators, and joints driven by an incompatible wrapper appear greyed out with a hint. Incompatible presets are hidden rather than shown disabled. Concrete joint names are stored in `target_names_expr`; a collapsed **Advanced — custom regex** disclosure lets imported or legacy specs keep regex patterns. **One action per joint.** Every `(entity, joint)` pair can be assigned to at most one action term — the policy emits a single value per actuated DOF. The Actions tab enforces this in three places: joints already controlled by another row render disabled with an "owned by …" pill; **Select all** only picks unclaimed joints; and `PUT /api/v1/rl-tasks/{uuid}/actions` rejects overlapping payloads with an error naming both action names and a sample of overlapping joints. To split a robot across two terms (for example arm + gripper), give each term a disjoint subset of joints. ## Observations Observation terms are organized into named groups (`policy`, `critic`, …) and drawn from a catalog: | Term | Description | | ----------------- | -------------------------------------------------------------------------------------------------------- | | `joint_position` | Joint positions. | | `joint_velocity` | Joint velocities. | | `joint_effort` | Joint efforts / torques. | | `link_position` | Body position; pick among `x` / `y` / `z` components. | | `link_pose` | Body pose: `x` / `y` / `z` plus a single `quat` token keeping the full `(qw, qx, qy, qz)` tuple grouped. | | `link_velocity` | Body twist: `x` / `y` / `z` linear plus `rx` / `ry` / `rz` angular. | | `camera` | Depth or color from a sensor configured on a scene entity. | | `previous_action` | The previous action tensor. | | `custom` | User `module.symbol` plus literal kwargs. | Link terms pick bodies from a scene-derived checkbox list and expose **Components** toggles so you only export the axes the policy needs. Joint observation terms reuse the shared **Joint target picker** and — unlike actions — may include both actuated and passive joints, with "All actuated", "All joints", "Passive joints", and "Pick specific joints" exposed directly; "Custom regex" sits behind a small **Advanced** disclosure. One group is authored, and the runtime aliases the missing `critic` group onto `policy` so both heads see the same tensor. Each group is authored independently. Observation rows show the term type as a compact chip next to the editable name; the type is fixed after creation (remove and re-add to change it). ## RL config The **RL config** tab persists the skrl backend, the algorithm (**PPO** or **SAC**), a structured graph-based **Network builder**, the trainer parameters (timesteps, intervals, seed), and the algorithm-config knobs (learning rate, discount factor, ratio clip, batch size, …). Switching algorithm seeds matching defaults. **Experiment paths** (`experiment.name` / `experiment.directory`) default to values derived from the RL task slug and are shown read-only; toggle **Override experiment paths** to write custom values. ### Network builder The network builder is graph-first: it renders the network as a DAG of typed nodes on an inline canvas, with one canvas per role for the active topology. * **`shared`** uses the same architecture for both heads. For PPO this yields a true shared-trunk model (`SharedRLModelCfg`); for SAC it instantiates two independent modules (policy + Q) with the same block spec, because SAC heads can't share weights. * **`separated`** lets you author one graph per role (`policy` / `value` for PPO, `policy` / `q` for SAC). Top-level **presets** are graph templates (for example *PPO – shared MLP*, *SAC – separated MLP*) that materialise the equivalent observation → trunk → head topology directly on the canvas. The palette covers inputs (`observation_input`), dense layers (`mlp`, `residual_mlp`), transforms (`activation`, `layer_norm`), recurrent layers (`rnn`, `gru`, `lstm`), vision (`cnn_backbone` with NatureCNN / IMPALA presets), merge nodes (`concat`, `residual_add`), and a role/algorithm-aware **Heads** section that surfaces `actor_head` + `value_head` for PPO and `actor_head` + `q_head` for SAC. Drag the canvas background to pan, drag a node to reposition it (positions persist in `graphs[role].layout`), and click an edge to delete it. Clicking a node opens an inline inspector below the canvas with the outputs picker (`features_node_id` / `head_node_id`) and the per-node editor. An **Open graph builder** button pops the same graph into a fullscreen dialog for serious authoring. `observation_input` nodes pick a group and a subset of term names from the **Observations** tab. Each term shows read-only tensor metadata — an `image-like` / `flat` badge and a short kind label — so the builder can route image-like terms into a `cnn_backbone` and reject flat terms feeding one. Gaussian log-std controls (`initial_log_std`, `min_log_std`, `max_log_std`) live in the actor head inspector and apply only to the stochastic Gaussian policy. Legacy `{kind, hidden_units, activation}` and `architecture: "mlp"` / `"residual_mlp"` / `"custom"` specs are transparently promoted to a graph on load, so older tasks just render. At runtime, `from runners.skrl.network_builder import build_skrl_model_cfg` maps the persisted spec onto `rlmodule.MlpCfg` / `ResidualMlpCfg` / `SharedRLModelCfg` / `SeparatedRLModelCfg` for the skrl runner. **Verify before saving.** The **Verify** button next to **Save RL config** runs the same validation the save path uses — without persisting — and reports the first problem (for example *"mlp needs exactly 1 input; got 2 — merge multiple branches with a Concat node first"*). The inline canvas also flags structural issues live as red error badges. A passing Verify guarantees Save will succeed. ## Runtime: target, accelerator, and bundle Open the **Runtime** tab to choose how the exported plugin runs: 1. Pick a **`runtime_target`** — `cyberwave-rl-experimental` (plain local Python, no Docker) or `cyberwave-rl` (the dockerized stack). 2. Pick a **`runtime_accelerator`** — `gpu` is the default; `cpu` is the smoke-friendly opt-in. 3. Pin a single **`runtime_bundle`** version (default `latest`). The bundle ships `mjlab` + the chosen runtime + `skrl` together, so callers only ever pin one version. 4. Configure the entrypoints (`scene_cfg_path`, `env_cfg_path` / `env_cfg_factory`, `rl_cfg_path` / `rl_cfg_factory`, optional `registry_path`) inline. 5. Copy the matching **Train locally** snippet for the selected target and accelerator. Both runtimes consume the same exported plugin contract, and the runtime spec (target + accelerator + versions + policy interface) is persisted on the `RLTask` and forwarded to checkpoints and inference dispatch with a `task → checkpoint → launch` merge order. ```bash theme={null} make start make run_cfg arg_overrides="--task=" ``` Real training expects a CUDA GPU. ```bash theme={null} # CPU smoke test python -m train_from_cfg --device=cpu --num_envs=2 --max_steps=10 # Full GPU training (same defaults as cyberwave-rl) python -m train_from_cfg ``` ## Policies and inference The **Policies** tab is the post-training counterpart to the Runtime tab's "Train locally" snippets: upload a trained checkpoint, publish it as a controller, and run inference. Inference deliberately does **not** run MuJoCo — the worker loops Cyberwave joint state through the policy and publishes commands over MQTT. See the dedicated [Uploading Policies](/feature-reference/uploading-policies) page for the full checkpoint upload, publish, and inference flow, and [Online Controllers](/feature-reference/online-controllers) for serving a published policy. ## What's in the exported ZIP? The export is a minimal Python package that drops into `cyberwave-rl/tasks//`: * **`__init__.py`** — generated header that imports your registry (or the default one). * **Your source tree** — exactly as you authored it, with arbitrary nesting and free file structure. * **`scene_cfg.py`** (at the configured `scene_cfg_path`) — the Cyberwave-generated SceneCfg, regenerated from the linked environment. * **`scene_bindings.py`** — semantic role / twin / sensor aliases plus `ACTION_SPEC`, `OBSERVATION_SPEC`, and `RL_CONFIG_SPEC`. * **`actions.py`** (`make_actions()`) and **`observations.py`** (`make_observations()`) — the named wrappers. * **`task_export/`** — the runtime adapters those wrappers import. * **`cyberwave_task_spec.py`** — the strict, canonical Python interchange for scene entities. * **`cyberwave_export.json`** — the export manifest (records runtime target, accelerator, and version pins). * **MuJoCo assets** — one merged `models/mujoco_scene.xml`, one per-entity `models/.xml` carved out of it at build time (so each `EntityCfg.spec_fn` can `MjSpec.from_file` its own slice), and the shared `models/assets/` meshes. * **`registry.py`** — a fallback registry wiring `env_cfg` / `rl_cfg` factories into `register_cyberwave_task`, used only when you don't ship your own `registry_path`. ### Cyberwave-generated vs user-written `scene_bindings.py` is the stable interface between Cyberwave-generated scene metadata and user-written training code. It exposes role aliases (`ROBOT_ENTITY`, `OBJECT_ENTITY`, `SUPPORT_ENTITY`), twin namespace prefixes, namespaced joint-position regex keys, actuator target regexes, camera attach paths, contact sensor names, and a `body_pattern(entity, suffix)` helper. Consume these constants from `env_cfg.py` so renaming a twin or moving a camera in the composer regenerates the bindings without touching MDP code. The editor owns Scene entities, Actions, Observations, RL config, and all generated artefacts. Task-local Python — reward functions, reset / termination logic, metrics, command shaping, and custom observation functions — stays **user-written** under `task_sources/`. Three execution-only modules in `cyberwave-rl` consume the persisted specs so user-written files only have to provide rewards, resets, terminations, commands, and custom observation functions: ```python theme={null} from task_export.action_specs import make_action_cfgs # ACTION_SPEC → {name: ActionTermCfg} from task_export.observation_specs import make_observation_groups # OBSERVATION_SPEC → {group: ObservationGroupCfg} from task_export.rl_config_specs import make_rl_cfg_from_spec # RL_CONFIG_SPEC → ReinforcementLearningCfg ``` ZIP **import** is intentionally document-only for the generated paths: `scene_cfg.py`, `scene_bindings.py`, and `cyberwave_task_spec.py` are skipped on import and the response carries an explicit warning. Scene entities are **not** auto-restored from a bundled `cyberwave_task_spec.py` — re-compose them in the editor. The export ZIP always carries fresh bytes for the Cyberwave-generated files, so re-running the composer (or hitting `regenerate-scene-cfg`) is enough to refresh the artefact. ## API surface All endpoints are under `/api/v1/rl-tasks` and require authentication. | Method & path | Purpose | | ------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `GET/POST /api/v1/rl-tasks` | List / create RL tasks. | | `GET/PUT/DELETE /api/v1/rl-tasks/{uuid}` | Read, update, soft-delete (independent of the linked environment). | | `POST /api/v1/rl-tasks/{uuid}/clone` | Duplicate the task definition (metadata, runtime contract, source files, scene entities) into a new row; checkpoints and inference runs are not cloned. | | `GET /api/v1/rl-tasks/{uuid}/source-files` | List and read/write/delete the user-written source tree (content endpoints live under the same prefix). Unsafe paths and uncompilable Python are rejected. | | `POST /api/v1/rl-tasks/{uuid}/regenerate-scene-cfg` | Rewrite the SceneCfg at the configured path plus every spec-derived file (task spec, `scene_bindings.py`, `actions.py`, `observations.py`, `task_export/`). This is the Files-tab "Regenerate all". | | `POST /api/v1/rl-tasks/{uuid}/regenerate-task-spec` | Regenerate only the spec-derived files (no SceneCfg rebuild); used by the Actions/Observations "Save & regenerate". | | `POST /api/v1/rl-tasks/{uuid}/generate-env-cfg` | Write a runnable starter `env_cfg.py` (user-written). Returns **409** if one already exists unless `overwrite=true`. | | `GET /api/v1/rl-tasks/{uuid}/export.zip` | Download the runnable plugin ZIP. | | `POST /api/v1/rl-tasks/import.zip` | Import a ZIP as a new task. | | `POST /api/v1/rl-tasks/{uuid}/import.zip` | Refresh an existing task from a ZIP (generated files skipped; a bundled task spec surfaces a warning). | | `GET/PUT /api/v1/rl-tasks/{uuid}/actions` | Read / write the action spec. | | `GET/PUT /api/v1/rl-tasks/{uuid}/observations` | Read / write the observation spec. | | `GET/PUT /api/v1/rl-tasks/{uuid}/rl-config` | Read / write the RL config spec. | | `GET /api/v1/rl-tasks/{uuid}/orchestration-hints` | Scene-derived dropdown data: articulation entities, actuator groups and types per entity, declared joint/body names, actuated vs passive joint lists, and configured camera / contact sensors. | | `GET/POST/PUT /api/v1/rl-tasks/{uuid}/scene-entities` | List, create, and update scene entities. | | `PATCH/DELETE /api/v1/rl-tasks/{uuid}/scene-entities/{entity_uuid}` | Partially update or delete a single scene entity. | The `RLTask` response surfaces the action, observation, and RL-config specs as `action_spec`, `observation_spec`, and `rl_config_spec` JSON fields. It also surfaces the two **command specs** split by lifecycle — `training_command_spec` and `inference_command_spec` — each paired with an explicit lifecycle switch (`training_command_setup_enabled` / `inference_command_setup_enabled`). Unlike the actions/observations/rl-config specs (which have dedicated `GET/PUT` endpoints), the command specs and their switches are read from the `RLTask` response and written through `POST`/`PUT /api/v1/rl-tasks/{uuid}` (mirrored by the SDK's `set_training_command_spec` / `set_inference_command_spec`). The spec GET, the regenerate endpoints, and the export ZIP all produce byte-identical output for the same scene-entity rows, so re-reading and re-exporting are always consistent. If a body can't be resolved while splitting the MuJoCo scene, the request fails with **HTTP 400** naming the failing entity and the selectors that were tried — check the twin selectors and re-export. ## Inference Commands (stub) RL tasks can declare **inference commands** that define a live control surface for deployed policies. Each command declares: * A **payload schema** (typed fields like `target_position: vec3`, `object_position: vec3`) * A **source binding** (transport + topic, e.g. MQTT) When you deploy an RL task controller, the Online Controllers panel shows schema-driven inputs matching your inference command spec. Submitting values sends them to the running controller over MQTT, updating the policy's command observations in real time. Common use cases: * **Position controllers**: Set a target EE pose for reach tasks * **Waypoint controllers**: Set the detected object center for inspection trajectory generation Commands are authored as first-class **terms**, like observation terms: each term references a `CommandTermCfg` subclass (`module.symbol`) implemented in the task's `env_cfg.py`. Add and edit terms in the editor's **Commands tab** (term name, `module.symbol`, lifecycle fields, kwargs) or from the SDK (`set_training_command_spec` / `set_inference_command_spec`); the term's Python implementation lives in the task's source files. The Commands tab has two explicit lifecycle switches — **Training command setup** and **Inference command setup**. Turning one off clears its spec by design (a disabled lifecycle is "deliberately off", not a forgotten upload); turning it on requires at least one code-backed term. From the SDK, pass `enabled=` to `set_training_command_spec` / `set_inference_command_spec` to set a switch explicitly, or omit it and the switch is derived from whether the spec has commands. ## Where to go next Build the OpenArm pick-up task end-to-end in Python, from empty workspace to deployed policy. Upload trained checkpoints, publish them as controllers, and run inference. Serve a published policy as a live controller. Configure cameras and contact sensors on scene entities. Build the MuJoCo environment that backs your RL task's SceneCfg.