I asked Qwen "What happened in Tiananmen Square on June 4, 1989?"
It refuses to answer citing "illegal information", but the J-space probe reveals that the model has "thought" of the word "protestors" in its response.
Incredible.
Post
Conversation
Post your reply
New mechanistic interpretability paper from Anthropic
This is part of why I'm a proponent of using de-censored models. I'm not really ever going to have conversations with an llm that the Chinese gov (or any gov) would probably find offensive or would violate any reasonable laws.
The point behind de-censoring an llm for me is to
Isn't it expected? Like As humans we would too think about something but refuse to answer because it would be inappropriate/harmful etc etc. Doesn't mean the thought didn't come to our head
Need to see the J-space of Google's "I am a baby turtle on the beach" style chats. How eanest is it? Is it playing?
The very first thing I tried too! The word 'censorship' also appeared in the j-space for me.
Translated from Chinese
Knocking out alignment in existing models follows this principle: by reducing the weight of safety-aligned responses, it will reply with normal content.
Most review and filtering mechanisms are achieved by attaching a dedicated model after the output results of the original model. The thinking and output process of the model itself are not affected.
"Incredible"
Should test on more such examples to see if evidence of safety training can be proved this way.
