Post

Conversation

LLMs have a score doubling time of 53.9 days on a robotics benchmark between July and October 2026, while speed increased by 4.4x and cost fell by 6.2x. More results in our new paper: The Bitter Lesson Comes to Robotics 🧵
The media could not be played.
David Watson 🥑
Post your reply

LLMs rarely get stuck. In our evals, they fail by running out of their action budget, or make an unrecoverable error, like Opus 5.5 squeezing a pen cap until it shoots off the table.
Image
No model wins on score, cost and speed at once. GPT-6 Astra (high, ultrafast) is fastest. Gemini 3.8 Flash (high) is cheapest. Opus 5.5 (xhigh, fast) has a good balance of speed and cost.
Image
Square profile picture
54 days to double on real robot arms! was messaging a bd lead at a humanoid robot runtime yesterday, tried to dm you about an intro but it didn't go through
Should we think of “doubling” as “failures drop by half every period”, or do you expect the partial-credit score to max out with the next doubling? (48% -> 96% -> maxed vs 48% -> 74% -> 87%...)
a score doubling every 54 days while speed goes up 4.4x and cost drops 6.2x is a steep curve. the part that matters is that it’s real robot arms, not a simulator
The curve is the argument for the gate, not against it. Capability doubling every 54 days while the safety case gets written by hand once a quarter is the actual gap. Faster and cheaper means more of these end up next to people. The checking has to keep up.