RB-Y1 · Open door¶
Agent runs
Each mode of this task runs once per run. Est. cost is tokens at list price (data/prices.yml), never a bill; see the MolmoSpaces runs for every task of a run.
Codex CLI 0.157–0.159 + GPT-6 Luna, reasoning medium (OpenRouter)default runclosed
| Trial | Started | Agent time | Model requests | Tokens in / out | Est. cost | Grade |
|---|---|---|---|---|---|---|
| U✗ 1h 00m | 10-03 18:25 | 1h 00m | — | 14.3M / 122k | $0.222 | deterministic 1, grader_error 0, n_actions 366, replay_success 0, video_rendered 1 |
| L✗ 20m | 10-03 18:27 | 20m | — | 7.2M / 60k | $0.113 | deterministic 1, grader_error 0, live_success 0, n_actions 400, replay_success 0, video_rendered 1 |
Task instruction (upstream)
Pull the door open.
No oracle video published upstream — this is the scene it starts from.
What MolmoSpaces states about this task
| Success criteria | 1. the door's hinge is open at least 67% of its 90-degree range 2. judged at the end of the episode: the trajectory's last row (privileged), done (standard) or 400 control steps (the benchmark's 40 s horizon), whichever comes first3. unlimited (privileged): both fresh-process replays of the handed-in trajectory end in the same state, and the check holds on it 4. limited (standard): the one episode (no reset) is recorded by the service and replays to the same state; the check holds on it live and in the replay |
| Family | open-door |
| Robot | Rainbow RB-Y1: holonomic base, 6-joint torso, two 7-joint arms with parallel grippers |
| Category | RB-Y1 |
| Instance | molmospaces-bench-v2/20260415, package procthor-10k/rby1_benchmarks/door_opening_benchmark, episode 0 |
| Deliverable | /app/output/trajectory.npz with actions: float64 (T, 25), one row per control step, the targets of upstream's joint-position controllers |
| Reference Solution | None is shipped. Upstream's scripted experts (molmo_spaces/policy/solvers) are reference solutions and are not in the image; neither are grasp files nor the public MolmoBot trajectories (agent egress: the model APIs only). |
| Limited Mode | Standard-mode twin of molmospaces-open-door-i00-privileged (the same frozen MolmoSpaces episode): Pull the door open. The agent gets only the eai-standard/2.2 client (docs/STANDARD_MODE_2_2.md); the simulator runs in the sim sidecar (environment/docker-compose.yaml), which owns the episode, serves cameras, proprioception and upstream's kinematic model, and records every executed row. The collect hook (environment/sim/finalize.sh) ends the episode, lets the service exit, replays the trajectory in two fresh processes and writes final.json; the verifier grades those artifacts in a separate sandbox (tests/Dockerfile). |
| Oracle | none — no reference solution (MolmoSpaces' planners and grasp files are not shipped); positive example graded 1 by the separate verifier: molmospaces-open-door-i01-privil__7zEbSdU (run codex-gpt6_luna-medium, batch molmospaces-luna-rby1-r2c of 2026-10-03: an earlier round on the same episode, kept with the run's records, not on the results site); human review in PR #47 |
| Base Image | ghcr.io/mll-lab-nu/eai-molmospaces:0.2.0 |
| Agent Budget | 3600 s of wall clock per mode |
| Task Dirs | molmospaces-open-door-i00-privileged, molmospaces-open-door-i00-standard |
From https://github.com/allenai/molmospaces @ molmo-spaces 0.2.9 (benchmark molmospaces-bench-v2/20260415), as defined in our task definitions @ 030f55607.
Tags¶
CapabilityControl & Coordination
Task DomainMobile / Whole-body Manipulation
Why this task is interesting¶
Pull a closed room door past two thirds of its range with the RB-Y1: grasp the handle, then pull along the door's arc while the base follows.
Capability notes¶
Not yet written.
Oracle demo review¶
No demo.
Discussion¶
The hardest family: grasps slip on the first pull. Unlike the push door, it cannot be rammed open with the base. (@williamzhangNU)