Gemini 4 Argon achieves SOTA on real robot capabilities, tied with Opus 5.5 and above GPT-6 Astra. 🧵
The media could not be played.
Conversation
Post your reply
Gemini 4 Argon matches Opus 5.5 on frontier robotics capabilities at half the cost: $2.34 per attempt vs $4.96.
Gemini 4 Argon ties with Opus 5.5 on RoboDojo-RC Tier 1, but takes 4× longer per attempt.
We release all 120 Gemini 4 Argon robot runs, with full transcripts and videos:
Thank you to the RoboDojo team! Our tasks are adapted from their benchmark, built by Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su and colleagues at HKU MMLab and UC Berkeley:
Want to build the next RoboDojo? We are giving out $500k and free YAM arms to academic teams to design dexterous manipulation benchmarks and run real-world evaluations. Express interest here:
We run the evals with our open-source harness, Inspect Robots. It is designed to run any model, on any robot, on any task:
Robocurve () is an independent Public Benefit Corporation evaluating frontier AI in the physical world. Learn more:
We're hiring for Member of Technical Staff (up to $300k base + equity). $5k for successful referrals. Apply or refer here:
The next generation Starlink V5 has a smaller form factor and lightweight design with greater power efficiency.
With speeds up to 375+ Mbps, Starlink V5 delivers reliable home internet for streaming, video calling, gaming and more.
Available in select areas.
Yup, the SOTA publicly available VLA is MolmoAct2, which scores 19% vs Gemini 4 Argon at 48%. Congrats to your team!
Yup, we find that Gemini models are unusually strong in robotics, stay tuned for more results.
Gemini Robotics 2 (VLA) is not publicly released yet. For Gemini Robotics ER2, we find that Gemini 4 Argon is much better.
Would love to see the failure modes broken down. "Tied at SOTA" can hide very different ways of failing, and that's what matters when deploying.
Yup, we release all traces here and will follow up on more results on failure analysis:
Quote
Jay Chooi
@chooi_jeq
Replying to @chooi_jeq
We release all 120 Gemini 4 Argon robot runs, with full transcripts and videos: robocurve.org/all-runs/?lead
We'd love a a dishwasher bench
Quote
Jay Chooi
@chooi_jeq
Replying to @chooi_jeq
Want to build the next RoboDojo? We are giving out $500k and free YAM arms to academic teams to design dexterous manipulation benchmarks and run real-world evaluations. Express interest here: bounty.robocurve.org/?utm_source=x
Pick and place, sorting, and even precision tasks like sorting. We release all 120 trials here:
Quote
Jay Chooi
@chooi_jeq
Replying to @chooi_jeq
We release all 120 Gemini 4 Argon robot runs, with full transcripts and videos: robocurve.org/all-runs/?lead
which robot hardware was fixed across runs, since hardware differences can dominate model scores
We used i2rt's YAM arms, which are also what we are giving out to academia
Quote
Jay Chooi
@chooi_jeq
Replying to @chooi_jeq
We release all 120 Gemini 4 Argon robot runs, with full transcripts and videos: robocurve.org/all-runs/?lead
tied with Opus 5.5 on real robot tasks is a bigger claim than most benchmarks — curious what the eval actually covers
We adapted the benchmark from the RoboDojo team
Quote
Jay Chooi
@chooi_jeq
Replying to @chooi_jeq
Thank you to the RoboDojo team! Our tasks are adapted from their benchmark, built by Tianxing Chen, Yue Chen, Zixuan Li, Junyuan Tang, Kailun Su and colleagues at HKU MMLab and UC Berkeley: arxiv.org/abs/2607.04434