In May 2026, Ai2 released MolmoAct 2, an open Action Reasoning Model for robots. In the independent bimanual fine-tuning benchmark we ran, it ranked first on 7 of 8 tasks, averaging 50.6% success compared with 32.2% for π0.5 and 35.5% for OpenVLA-OFT. We collected most of the bimanual training data and ran that benchmark.

  • We collected 700+ hours of the bimanual YAM training data, released as part of the open MolmoAct2-BimanualYAM dataset.
  • We designed and ran the independent real-world fine-tuning benchmark (1,500+ eval trials: 5 policies × 8 tasks × 50 trials, plus OOD tests).

1. Collecting the MolmoAct2-BimanualYAM dataset

We collected 700+ hours of the bimanual YAM training data, released as part of the open MolmoAct2-BimanualYAM dataset: 34.5k demonstrations and 720+ hours of robot data across 28+ tasks, including folding clothes, untangling cables, bussing tables, scanning groceries, and packing medication. Protocols capped the number of failure retries per episode and how long an episode could sit idle (no-op segments), so the data stayed consistently clean.

We collected these demonstrations on the reference rig Ai2 published with MolmoAct 2: two YAM arms, an overhead Intel RealSense D435, two wrist-mounted D405 cameras for close-up views, an extendable mount, and a simple tabletop. That published setup is a starting point other labs can reproduce for tabletop and bimanual manipulation.

The standardized bimanual YAM setup.
Two YAM arms, an overhead RealSense D435, and two wrist-mounted D405 cameras. From the MolmoAct 2 paper, Figure 3.

2. Running the third-party fine-tuning benchmark

Ai2 asked us to benchmark how well MolmoAct 2 adapts to new tasks through fine-tuning. We tested MolmoAct 2, π0.5, OpenVLA-OFT, Cosmos Policy, and X-VLA on eight bimanual tasks: stacking cups, storing test tubes, storing candy, hanging tools on a pegboard, storing toys, shelving cups, preparing a pipette, and making popcorn. The tasks were set up in lab, kitchen, study-room, pantry, and wet-lab environments, and one of them used a mobile bimanual YAM.

For each task, we recorded fine-tuning data under three spatial layouts and then evaluated under those same three layouts. For store candy, that meant 200 episodes (about 2 hours 15 minutes), split 67/67/66 across the layouts. Every policy was trained with the same protocol.

MolmoAct 2 placed first on 7 of the 8 tasks. Its average success rate was 50.6%, compared with 35.5% for OpenVLA-OFT, 32.2% for π0.5, 15.5% for Cosmos Policy, and 4.9% for X-VLA (paper, Table 19).

Real-world evaluation on Bimanual YAM tasks.
Success rates across the eight tasks we evaluated. From the MolmoAct 2 paper, Figure 7.
Sample MolmoAct 2 trajectories.
MolmoAct 2 rollouts on each task. From the MolmoAct 2 paper, Figure 10 (Appendix C.3).

We also ran out-of-distribution tests. Each was tested under spatial shifts, lighting changes, rephrased instructions, and added distractor objects. MolmoAct2-Think averaged 50.7% success under these shifts, compared with 39.9% for the next-best model (OpenVLA-OFT). Spatial variation was the hardest condition for every model.

Out-of-distribution variants.
The OOD variants for store candy. From the MolmoAct 2 paper, Figure 11.

Watch the episodes

The evaluation rollouts are public on Hugging Face, so you can watch every policy succeed and fail: MolmoAct 2 eval rollouts collection. Here are the MolmoAct 2 runs:

The same collection also has the rollouts for π0.5, OpenVLA-OFT, Cosmos Policy, and X-VLA.

Read more