Post

Conversation

Great thoughts on AI advances by my colleague Kevin. This is my intuition, too - we’re seeing diminishing returns - but a jump from GPT-4 to GPT-7 that’s equal to 3 to 4’s delta is still huge. Not AGI, but AI already makes trillions of dollars of > productivity possible.
Quote
Kevin Niechen
@kevinniechen
Are we actually facing diminishing returns from model scaling? What happens when we extrapolate scaling laws for the next few model build outs? New essay (full link below):
Image
David Watson 🥑
Post your reply

Unlikely. In private I heard him and Zuck both estimate mid to late 2030’s. I wouldn’t be surprised if that gets pushed back, but let’s see !
The wall that no one's talking about is the Knowledge Wall. Soon, the best AIs will make better-than-human theories and predictions, but those predictions will need to be verified empirically. This costs time and money. Only then will we realize that what's interesting--new
Show more
The original architectures were hastily made and highly inefficient which is evidenced by the off-a-cliff costs we see over time now. But the breakthrough has been made to satisfy the only business metric that matters. Usefulness. Would have killed to have this foundation on
Show more
I don't believe we'll scale to AGI but I also think the research path to the next big inflection point is pretty clear (even though nobody really seems to be working on it 🤔)
Even today's AIs are tremendous for certain things. For example, I used ChatGPT to summarize each of 20 top bank 2025 market outlooks and then do a meta analysis of where the reports agreed and where they disagreed. 20 hours of manual work took 30 minutes with gen AI.
Show more
We already have AGI. Current models can be given a broad range of goals and complete them. So Openai etc switched the definition, what they're talking about now is ASI, artificial superintelligence. But that's clearly very dangerous, so they just moved the "AGI" goalpost.