Model adapter
A model runs in the public policy runtime when it implements PolicyAdapter. The runtime calls select_action once per control cycle. The same contract is used for every registered backend. The interface does not name a robot vendor.
Source in the OmniBot repository:
omnibot-ai-engines/lerobot_engine/models/base.py—PolicyAdapterandModelAdapterContractomnibot-ai-ros2/omnibot_lerobot/omnibot_lerobot/policy_node.py— loadsmake_policy(model_type)and publishes after the hooksomnibot-ai-ros2/omnibot_lerobot/omnibot_lerobot/policy_guard.py— black-frame refusal, stale-inference drop, and action limitsdocs/model-adapter.mdin that repository — the same contract, including the two headless runs
Inputs
Cameras arrive as uint8 HWC. The runtime resizes to the adapter's image_size and passes float32 images in [0, 1] with shape (1, channels, height, width).
| Field | Default |
|---|---|
| Image keys | observation.images.wrist, observation.images.bev |
| Channels | 3 |
| Resize | 320 × 240 (width × height) |
| State key | observation.state |
| State shape | (1, 9) float32 |
| State units | six joint positions in rad, then base_vx m/s, base_vy m/s, base_yaw_rate rad/s |
| Task key | none, or task when the model reads a language instruction |
Outputs
select_action returns one finite float32 vector.
| Index | Name | Unit |
|---|---|---|
| 0–5 | joint_0 … joint_5 |
rad (position target) |
| 6 | base_vx |
m/s |
| 7 | base_vy |
m/s |
| 8 | base_yaw_rate |
rad/s |
The runtime slices the first six components as joint targets and the last three as base velocity. It publishes them only after the safety hooks.
Control rate and latency
control_hz defaults to 10. The latency budget is one period, 0.1 s. The policy node uses its policy_hz parameter for that period.
If preprocess plus select_action takes longer than the period, the tick is stale. The action is dropped, base velocity is set to zero, and no arm command is published.
Safety hooks
The runtime applies these on every enabled tick. A model cannot disable them.
- Black-frame refusal. A required image that is missing, empty, or all zeros is refused. No arm command is published. Base velocity is zero.
- Stale-inference stop. A tick slower than the latency budget is dropped the same way.
- Action limits. Joint targets move at most 0.15 rad from the previous command in one cycle, and they stay inside the configured joint limits. Base velocity is scaled (default scale 0.3) and clipped to 0.20 m/s linear and 0.50 rad/s angular.
What has been run
Two numpy adapters, reference_linear (seed 447) and reference_mlp (seed 448), were run through this interface. Weights are float32 draws from normal(0, 0.05) with zero biases. No checkpoint was downloaded. Each run is one trial of the headless pick-and-place harness with an in-memory transport, plus one enabled tick of the policy node under ROS stubs. Gazebo was not started. rclpy was not started. No robot device was opened. The trial result is a timeout because that transport never reports a cube pose.
ACT, SmolVLA, diffusion, and OpenVLA implement the same methods. Their published checkpoints were not loaded in these runs. A real-robot run was not done.