They also compared personalized AI to the most persuasive human experts:"Remarkably, Personalization ranks in the 99th percentile among all users and the 98th percentile among experts, critically approaching thresholds that experts associate with the emergence of existential AI risks"

If these models are a particular persuasion threat, it’s likely only in the way that drones are a particular violence threat: because it’s easier for most people to tell a model to lie than to lie directly themselves, just as it’s easier for most people to kill through a remote drone.

Yeah, I'm seeing this as yet another case of LLMs-as-time-savers. What's the difference between a human - - doing research so they can lie accurately; - giving an LLM a post containing lies and asking it to edit; - asking an LLM to write the post entirely while asking it to lie in the prompt?

This is going to sound weird, but I find myself emulating LLM’s tone when I’m in a difficult interaction. Just because it’s effective I’ve actually asked them to coach me through hard, high-stakes workplace interactions (I am very informal and rough around the edges. The get social norms better)

Wow holy crap My fear? projection? is that AI achieves superpersuasion before they achieve superintelligence Likely even before they achieve “consciousness”, leading to a bunch of persuasion bots that are highly persuasive but lack intent Thank you!

Arguably we are already there