the most goated example i could find when eval'ing Opus 5 was on a frontier bench very hard task. the model is given an engineering drawing of a machine part and asked to write code to rebuild it as a 3D freeCAD model. the fun catch is that the model has no way to view the drawing: mini-swe-agent is a text-only harness, so all the model ever sees is terminal text over bash IO. it's something hard and multidisciplinary even for a human engineer capable of seeing the drawing!
working blind, Opus 5 first tried off-the-shelf OCR with Pillow and Tesseract, found the results unreliable on the drawing’s thin numerals, and built its own vision stack on the spot. carved out each printed character as a connected component, rendered the glyphs as ASCII art (?!?), and matched glyph bitmaps to read the dimension labels off the page
then it vectorized the drawing itself: run-length line detection with sub-pixel centroids, least-squares circle and ellipse fits to recover radii and bores, arrowhead detection to tell dimension lines from part geometry. it worked out that the four views were printed at different scales, derived each one’s mm-per-pixel ratio, and rebuilt the part, feature by feature, as a parametric freeCAD model
welcome to the world, my dude!
Post
Conversation
i love hearing stories like this from you. also lol re: "rendered the glyphs as ASCII art (?!?)," this has got to be a training data thing, right? (i mean, it all is...)
Quote
Hui Kang Tong
@ExampleTestcase
Replying to @a_karvonen
If AI writes code and execute the code to solve does it count?
change the interface
change the computation
the model didn't just solve the task
it changed how the task could be solved