Tweet

See new Tweets

Conversation

The assumption that AI will soon run out of data to train on, or that it will become “corrupted” by training on AI-created content, is just an assumption. There are already cases where AI trains itself on AI data, and the limits of these techniques are not yet clear.
Quote Tweet
Arvind Narayanan
@random_walker
A popular view. I get it — learning w/o human knowledge seems implausible, even insulting. But let's wait and see. We already know this is false in some domains like game playing — synthetic data *alone* (self play) yields superhuman agents. In which other domains can it help? twitter.com/tdietterich/st…
David Watson 🥑
Reply
if humans can improve our world models through thought experiments, then shouldn't AI models be able to get better with synthetic data? of course, this only works up to a point w/o real-world experimentation, but reasoning & reflection seem to get us pretty darn far!
1
1
also, I think that the value of having a high-fidelity private digital repo of your life (a la ) will soon be so great that there WILL be lots more data to train on...just a matter of each person deciding what to keep secret vs. give up to foundation model developers
2
Show more replies

More Tweets

Product Management is easy! All we need to do: Is create wonderful products for users And launch it on time And make it sustainable for the business And make engineers understand the solution And then verify if they've built the right solution And convince 100 people to… Show more
My 10yo child, upon disagreeing with a parenting decision: “I understand this as much as I understand Icelandic. But it’s different, because even if I don’t understand Icelandic, lots of other people do. NO ONE else understands this decision but you.”
16
647