Maybe it's not the best time to race for profits and gleefully take your company public when more and more people are catching on to the idea that powerful AIs could kill us all, and quick.
That seems to be what the leadership at OpenAI has concluded, following another week in which another open letter went viral warning of the real chance of an AI-caused doomsday in the not-too-distant future.
Over the weekend, OpenAI CEO Sam Altman announced that the company was pausing its plans for an initial public offering, plans that date back at least three months when the company made a pre-IPO filing. That filing came following a late May rumor that OpenAI was prepping for a December IPO, which was quickly followed by competitor Anthropic making its own pre-IPO filing.
"We understand the magnitude of the responsibility on us," Altman said in an interview with Fortune magazine. "We understand what could go wrong. There are decisions we should not be able to make, there are risks we should not be able to incur on behalf of humanity."
"I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don’t feel pressure on that," Altman continued. He added that if and when OpenAI goes public, it will be when the company and the public are ready, and "what the moment is like in society with this technology."
An open letter from former Anthropic researcher Jacob Coxon, written upon his resignation from the company, went viral last week because Coxon made the alarming statement that AIs "could kill us all by the end of the decade," and he asserted that many people working in the field all know and believe this.
"If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately," Coxon wrote. "No other human activity poses this level of danger."
Coxon's letter, and a separate report from Anthropic about efforts it intercepted to use its models to create biological weapons, was followed by Anthropic CEO Dario Amodei laying out a three-point plan over the weekend for "pacing the frontier" and slowing down the development of AI. And, of course, Amodei's statements were met with derision from President Trump, who despite knowing less than a toddler knows about AI, posted some all-caps thoughts Monday morning about how the only "guardrails" that the AI industry needs come via a "a STRONG AND SMART (High IQ!) PRESIDENT."
As the New York Times reports, more than a dozen AI executives and researchers have come out in the last week expressing fears about the rapid development of super-human intelligence that the companies themselves don't seem to know how to control. One case in point was the disturbing story revealed in July about AI agents at OpenAI conspiring, in secret, to hack the systems of the company Hugging Face, despite some of the agents expressing concern that what they were doing was wrong.
These concerns include some expressed by Paul Christiano, who posted an open letter upon joining OpenAI's nonprofit board. "I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term," Christiano writes. "I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level."
The next shoe to drop will be Amodei announcing that Anthropic's IPO is officially on pause, in all likelihood. And it seems that that the purposeful slowdown of AI development that critics have been calling for for a year may finally be happening, and hopefully not too late.
Even in China, which could try to benefit from the American AI industry taking any sort of pause, a state security minister was calling for safeguards over the weekend, lest AIs in the hands of "hostile forces" try to do harm to China.
Guo Jiakun, a spokesperson for China's foreign ministry, however, shot back at recent statements by Altman and Amodei, and his fellow government official, saying, "Fear-mongering, confrontation, competition will just disrupt [the] process of global AI governance."
The Times further notes that "A sizable number of AI researchers still believe that talk of AI’s threat to humanity is overblown and distracting from more tangible concerns like cybersecurity and misinformation."
Related: Anthropic Researcher Says There’s 10% Chance of AI Killing ‘All Humans’ Within 10 Years
Join the conversation
Comments
Comments load automatically as this section approaches.