Across 360 robotics trials, Opus 5.5 averages under $1 per trial, scoring 1.8x as high as Opus 5 at half the cost and roughly matching Astra's mean score at 21% lower cost.
The media could not be played.
Conversation
Post your reply
Opus 5.5's mean score was at least 1.4x as high as Opus 5's on five of the six robot control tasks we tested.
We adapted a subset of RoboDojo tasks using readily available materials. We release this as RoboDojo-RC Tier 1 and publish the bill of materials in our report.
The media could not be played.
On stacking bowls, Opus 5.5's mean score was 1.4x as high as Opus 5's at 45% of the cost.
The media could not be played.
On capping a pen, Opus 5.5's mean score was 2.8x as high as Opus 5's at 56% of the cost.
The media could not be played.
On storing objects in a safe, Opus 5.5's mean score was 2.0x as high as Opus 5's at 56% of the cost.
The media could not be played.
On sorting objects, Opus 5.5's mean score was 2.0x as high as Opus 5's at 57% of the cost.
The media could not be played.
On packing and pouring fruit, Opus 5.5's mean score was 1.4x as high as Opus 5's at 50% of the cost.
The media could not be played.
All three models had low scores on standing bottles upright.
The media could not be played.
We release the RoboDojo-RC Tier 1 benchmark and all 360 traces here:
Our evaluation harness is fully open-source and designed to run any model, on any robot, on any task:
Robocurve () is a Public Benefit Corporation and an independent third-party evaluator for frontier robotics systems. Learn more:
Discover more
Sourced from across X
So many important lessons in here.
Robotics is so hot right now, and naturally everyone wants to get in. Data looks like one of the easiest entry points, but in reality it’s an incredibly hard business to operate and scale.
And the window may be much smaller than people think.
Quote
Sam Padilla
@theSamPadilla
After 2+ years in the robotics data space, we are shutting @Eidon_AI down.
The thesis was right. But the business is brutally hard.
We close this chapter by open-sourcing everything we built and sharing lessons for anyone venturing into the space.
Robotics Data is a Broken Business
A technical bottleneck does not necessarily imply a good business.
We recently shut down @eidon_ai after 2+ years in the data space.
We started in early 2024 with a gamified multimodal data collection...
Quote
Skild AI
@SkildAI
We trained a robot to play football.
How? Self-play for 140 years in a virtual World Cup.
Meet the #Messinator:
The media could not be played.
We might actually see a robotic world cup at some point in the next 5-10 years.
Throwing my bet on to make it happen.
Quote
Skild AI
@SkildAI
We trained a robot to play football.
How? Self-play for 140 years in a virtual World Cup.
Meet the #Messinator:
The media could not be played.