With Gemini 4 Argon we are so back, on the one metric that really matters: length of our safety report!
The Argon safety report is Google's most thorough by far and IMO there's a lot of good content, I particularly like the loss of control, alignment & monitorability assessments
Conversation
I'm glad the report exists and is thorough but honestly, the contents are not reassuring. In particular, the paper acknowledges that the model is VERY aware of being in an eval, basically across the board. So how can we take these results at face value?
Alignment eval results are not a load bearing part of our loss of control risk assessment, the key part is that we believe our monitors would catch anything bad. See more: storage.googleapis.com/deepmind-media
Don't miss out on this deal!
We need you being thorough on the safety reporting and this nervous biology near-stander deeply appreciates it.
Monitorability is the section that ages best. Every capability number in it expires at the next launch.
i haven't read it yet, but kudos, genuine thanks - good reports/system cards are extremely helpful
Good to see the safety side get attention too. We covered the capability side: Google's table has Argon on top in 14 of 19 scores, but not a clean sweep.
youtu.be/qCx6kk1-YkM
#Gemini #AISafety
This was the first turn 👀
x.com/chupilka/statu
Quote
ok but does the safety report need its own context window now