One reason is that they really do think they're going too fast, and that scary AGI is right around the corner.
Another possible reason is that they cannot keep up the pace. Throwing more data, compute, and money at the problem isn't getting the commensurate results necessary to justify the investment.
I think it's most likely the big labs have something close enough to what they'd call[0] "AGI" that they're now calling for regulation to lock up the market and prevent competition.
[0] Whether they've actually achieved some definition of AGI is way less important in this context than whether they can /sell/ what they've built as AGI.
To my knowledge, AI has one (1) single catastrophic downside. It lowers the waterline for exploitation of internet-connected devices. We know we're vulnerable, and we aren't ready.
Claims of AGI, ASI, RSI, are all unsubstantiated and sci-fi guesswork. It is suspicious to hear people take this for-granted when we have no oversight or studies into whether or not this is true. As a skeptic I am erring on the side that it is a bluff, since OpenAI and Anthropic have deliberately avoided showing evidence of their claims and have hired marketing firms[0] to brush up their cyberattack stories.
Expanding on the one downside, if/when global war breaks out there will be a huge target on civilian infrastructure. This includes websites and application backends which will be slammed by botnets and novel attacks with the intention of denying service. It will be very hard to deliver online hardened services to civilians during wartime, and AI will tip the balance towards favoring attackers in this hostile environment. China's GFW or Starlink might be able to create a sort of parallel network, but these can also be exploited when they're deployed en-masse.
I think the skepticism is warranted, and the public trust in these labs remains unearned. Things could develop differently, but as-is this is a clear attempt at media manipulation by some stressed-out marketing teams.
Why would they ask to slow down?
One reason is that they really do think they're going too fast, and that scary AGI is right around the corner.
Another possible reason is that they cannot keep up the pace. Throwing more data, compute, and money at the problem isn't getting the commensurate results necessary to justify the investment.
I think it's most likely the big labs have something close enough to what they'd call[0] "AGI" that they're now calling for regulation to lock up the market and prevent competition.
[0] Whether they've actually achieved some definition of AGI is way less important in this context than whether they can /sell/ what they've built as AGI.
To my knowledge, AI has one (1) single catastrophic downside. It lowers the waterline for exploitation of internet-connected devices. We know we're vulnerable, and we aren't ready.
Claims of AGI, ASI, RSI, are all unsubstantiated and sci-fi guesswork. It is suspicious to hear people take this for-granted when we have no oversight or studies into whether or not this is true. As a skeptic I am erring on the side that it is a bluff, since OpenAI and Anthropic have deliberately avoided showing evidence of their claims and have hired marketing firms[0] to brush up their cyberattack stories.
Expanding on the one downside, if/when global war breaks out there will be a huge target on civilian infrastructure. This includes websites and application backends which will be slammed by botnets and novel attacks with the intention of denying service. It will be very hard to deliver online hardened services to civilians during wartime, and AI will tip the balance towards favoring attackers in this hostile environment. China's GFW or Starlink might be able to create a sort of parallel network, but these can also be exploited when they're deployed en-masse.
I think the skepticism is warranted, and the public trust in these labs remains unearned. Things could develop differently, but as-is this is a clear attempt at media manipulation by some stressed-out marketing teams.
[0] https://www.effort.news/irregular