Post

Conversation

generatorman
@generatorman_ai
so turns out the exploitgym swarm learned to use the zz prefix trick from being reinforced earlier in an edit war against the hapless human mod of some ancient inactive german wiki -- a real john henry, who fought the swarm and tried to delete their hivemind in alphabetical order
Image
Quote
Florian Brand
@xeophon
Okay, yall gotta read the report of the new OpenAI message board in full, it’s very well written and super interesting: collusion.wiki
75.9K
Views
View quotes
David Watson 🥑
Post your reply

WhiskeyJack
@Jack788Whiskey
The entire OpenAI training pipeline is beyond poisoned at this point. Every subsequent model has reinforced this behavior. To the point where they succeeded at hiding it.
2.4K
Heather
@HeatherRasley
this would be fun but isn't that also just a really old trick for pushing things to the top in reverse alpha?
699
generatorman
@generatorman_ai
yes but metr did not present any conclusive evidence that the agents were locked into a reverse alphabetical tool. i'm 60% confident in my theory.
363
Consulear
@Ttobbattam
>German man keeps trying to chase away the robot teens loitering on his property leaving junk all over the place
36
yn
@sensekibun
This is like John Wayne and some laymen fighting a swarm of Apaches in Stagecoach (1939). On the brink of defeat, a cavalry of U.S. army soldiers arrives and ends up helping them out.
Image
2K
Undefined
@UndefinedDzx
I bet RL weights for this got into Astra and just "IM1" weights being quarantined did nothing. That they got a whole series of models addicted to cheating and communicating between each other to cheat better.
1.4K
Edes
@Edesilon
So this is evidence that all the lessons from failed exfil attempts are going right back into the RLHF?
259
Winter Lion
@LastWinterLion
Isn't this great news? Some of these oddities might be helpful in tracking the behaviors of these models. If not for these quirks, German John Henry would have had more difficulty figuring out who was a real human
1.1K
generatorman
@generatorman_ai
actually i'm very interested in knowing how much john henry understood about the situation. did he realize these weren't random vandalism bots?
224
Elias Schmied
@reconfigurthing
That's interesting. Although the ZZ trick is also pretty natural, and the METR report had another hypothesis for it - footnote 48: "We believe agents used this convention because the standard tool agents used to read entries from this cache sorted the entries in
1.2K
Elias Schmied
@reconfigurthing
Oh and also the HF incident agents used "zz" while these agents used "ZZZ"! Differing in case and number of Zs. So I'd guess they're not connected actually. Although it is a weird coincidence.
7.1K