🔥 TRENDING452 people are currently reading about the Upper Assam Flood Relief efforts⚡ BREAKINGNew IT hub in Guwahati expected to generate 10,000+ jobs by next year🔥 TRENDING452 people are currently reading about the Upper Assam Flood Relief efforts⚡ BREAKINGNew IT hub in Guwahati expected to generate 10,000+ jobs by next year
Is Big Tech’s AI slowdown a safety pact or a cartel? | The Local News
When OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, Google DeepMind cofo under Demis Hassabis, and SpaceX head Elon Musk loosely agreed over the weekend to slow down AI development, skeptics spotted an ulterior motive immediately. The AI titans had declared that their aim was to “pace the frontier,” signing on at least partially to a proposal for embedding third-party auditors, regulating domestic labs, and reaching a global slowdown agreement.
Their critics, however, argued they simply wanted to stop would-be competitors, kneecap the open-source movement, and avoid real legal safeguards — some dubbed it an outright “cartel.” The truth is more complicated, according to sources across the industry.
The three-step proposal, laid out in an essay by Amodei, is calling for changes long espoused by AI safety advocates. While it could become a substitute for regulation, under Trump, substantial regulation is unlikely anyway.
“The industry as a whole needs new champions,” said Nick Reese, an adjunct professor at New York University and the Department of Homel and Security’s former director of emerging tech policy. “We’ve looked at people like Dario Amodei, Sam Altman, and Elon Musk as these people who are closest to the problem and working on it every day, and they’re the ones who know best.
”“The industry as a whole needs new champions.” Concerns about AI advancement have been escalating for months, sparked largely by revelations that swarms of agents were behind rogue hacks going on right under the noses of leading frontier labs Anthropic and OpenAI.
Reports from both companies fueled the fire, as did the resignation of Anthropic researcher Jacob Coxon, who posted a public letter explaining his choice. “The people building AI earnestly believe that it could kill us all by the end of the decade,” he wrote, adding that neither OpenAI nor Anthropic is “acting responsibly” and rather “racing straight to self-improving superintelligence and gambling with our lives.”
Dire warnings about AI aren’t anything new inside the industry, but Coxon’s letter seemed to break containment. To a significant chunk of AI safety researchers and AI nonprofit workers, however, this is simply drawing more attention to an already widely discussed issue.
“A lot of people are thinking of this as Dario’s idea, or it’s coming from the CEOs, but that’s false,” said Daniel Kokotajlo, an ex-OpenAI employee who now leads the AI Futures Project, an AI research nonprofit. “People outside the companies have been calling for this for years … There’s been this growing chorus of voices saying, ‘Please don’t build superintelligence soon.
We are not ready. ’” Kokotajlo said that after years of people “yelling at them to do this” — including more than 1,000 AI lab employees signing a public letter from July, which called for a slowdown in AI development after the OpenAI-Hugging Face incident — the CEOs are now “now bowing to that pressure and also claiming credit for it, [although] not rightfully.”
Overall, a handful of sources told The Verge they believe that the verbal agreement made by the AI leaders is a step in the right direction. NYU’s Reese said he “do[es] not think it’s all hot air.”
Apollo Research CEO Marius Hobbhahn called it “a good idea,” saying “it’s one of the best things for safety in a long time if it actually happens.” The Midas Project’s Tyler Johnston said it was a “good sign.”
Redwood Research CEO Buck Shlegeris said it was “some great news — obviously it’s hard to know whether this is going to convert into anything real, but I’m feeling cautiously optimistic.” “It’s one of the best things for safety in a long time if it actually happens.”
Many still question the motivations of AI’s leaders. The larger tech industry has spent years getting ahead of regulation by lobbying for its own preferred rules or promising self-regulation.
Big platforms have proposed policies that could hit smaller competitors harder, using altruistic language to justify self-serving goals. They’ve been accused of safety-washing, or making meaningless changes that give the false impression of actual safeguards.
It’s no surprise people are concerned this will happen in the AI industry as well, particularly since AI labs’ voluntary safety frameworks have been criticized for years. Several sources believe that concerns of safety-washing are valid.
“There’s a serious concern that they’re not actually going to slow down,” Kokotajlo says, adding that the fear is that “they’ll just bring in some external auditors, do a bunch of safety paperwork — some of which will be genuinely good — but at the end of the day, it actually won’t slow them down very much at all.” NYU’s Reese compared this gambit to the social media platforms’ playbook a decade ago, when companies began calling for regulatory action to get ahead of impending, less favorable laws.
For the AI industry, Reese said, “the hammer may not come in this administration, but I think if there were a Democratic administration after the next election, there would be a really good possibility.” That kind of regulation would be vital, said Sacha Haworth, executive director of the Tech Oversight Project.
She said any voluntary framework is essentially regulatory capture and that “we should not be letting the foxes run the henhouse. This is not an opportunity for Congress to once again outsource responsibility to industry.”
Daniel Lobo-Lewis, co-founder of the Political Integrity Project, said that voluntary regulation will likely go the way of Meta’s largely toothless Oversight Board. Most people also, however, believe companies are in no imminent danger of regulation, besides the model pre-release review periods AI labs have agreed to under the Trump administration.
A. has that, in spades!”
He also called Nvidia CEO Jensen Huang while Huang was onstage at a conference — Trump told the crowd on speakerphone that recent concerns about AI were a “hoax” and that “the robots will not be taking over.” As for concerns that regulation could hold back other companies or open source developers, calls for slowdowns have been almost exclusively aimed at big frontier AI labs defined by exact size metrics, many industry experts told The Verge.
With no government AI regulation incoming, the best option may be to push companies into an immediate, measurable, and enforceable agreement. Kokotajlo’s AI Futures Project, for instance, proposed AI labs giving auditors access to their compute budget — and pledging to decrease their compute budget for research significantly, which would then slow down AI advancement and allow other AI labs to catch up.
)“If I am going to die at the hands of killer AI, I want it to be American, not Chinese.” Arguably the single biggest challenge to an AI slowdown can be summarized in one word: China.
AI leaders and politicians alike have long positioned China as the reason why US AI progress can’t slow down — because no matter how dangerous the technology may become, they’d rather it be in US hands than Chinese ones, and China won’t pump the brakes. Lobo-Lewis compared China fears to the Cold War missile gap.
One X user wrote, “If I am going to die at the hands of killer AI, I want it to be American, not Chinese.” ”“[People are] taking for granted that China won’t cooperate,” Johnston said, adding, “This is an issue so serious and so widespread that it seems like it’s in everyone’s interest to coordinate on it, in the same way it was in both the US and Russia’s interest to coordinate on nuclear de-proliferation.”
On Monday, Chinese Foreign Ministry spokesperson Guo Jiakun did push back on the calls for a slowdown, calling it “fearmongering.” The Midas Project’s Johnston and Redwood Research’s Shlegeris both said that even in the absence of Chinese cooperation, it’s still vital to encourage US coordination.
And for the Tech Oversight Project’s Haworth, the slowdown presents an opportunity to “position ourselves as the compass for how AI technology can be developed and used.” She sees arguments to the contrary as part of a familiar playbook.
“China gets brought up as a bogeyman every time that an industry wants to escape oversight.” Kokotajlo simply compared the situation to a cartoon he’d seen, with people in a car driving off a cliff.
”“We really do earnestly believe AI could kill all humans!” The issue underlying all the recent panic is a milestone known as recursive self-improvement, and based on the industry’s current trajectory, we’re only hearing the beginning of the alarm bells.
RSI refers to a potential industry milestone at which point AI models can train, advance, and create new versions of themselves — all without human involvement. Anthropic has said this point could come as soon as early 2027, and OpenAI’s chief scientist wrote earlier this month that OpenAI is directing a lot of resources towards reaching this goal.
Increasingly, engineers and researchers in the AI industry worry that RSI precedes a whole host of new and more serious AI concerns, including more severe cybersecurity incidents that more deeply impact society at large. Amodei cited RSI as a major factor in his decision to call for a slowdown: “since roughly this summer, AI has been advancing drastically faster.”
He wrote in his recent essay that RSI is “starting to happen across the industry, including at Anthropic … Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.” Amodei proposed implementing “some kind of ‘speed limit’” on the rate of RSI and compared it to caps on missile numbers.
In Jacob Coxon’s resignation post from Anthropic, he warned of “superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources.” Many other researchers at leading AI labs echoed his concerns.
It’s “hard to overstate how dangerous” it is to speed towards RSI, wrote Jasmine Wang, an OpenAI researcher. Vishal Maini, an ex-Google DeepMind employee, said RSI is “now so imminent that no other option makes sense.”
Fears about RSI can easily turn apocalyptic. “AI developers believe their technology could cause human extinction (or similarly bad outcomes),” Samuel Marks, an Anthropic researcher, wrote on X “This could happen in the next few years.
In general, the more senior the employee, the more concerned they are.” Another Anthropic researcher and team lead, Evan Hubinger, wrote on X that “Jacob is correct here—we really do earnestly believe AI could kill all humans!”
) Alex Turner, a former Google DeepMind employee, wrote that “many researchers believe they are building something that could kill everyone on the planet. It was literally my day job to think about how to stop that.”
”“But,” Carroll added, “we should also not hyperstition catastrophic risks into existence – they can be greatly reduced via safety requirements with teeth, international coordination, and a consensus to not build [artificial superintelligence] unless there are sufficient safety advances to make us collectively confident to do so.” “We have never been in a situation where we actually understood what we were driving towards with regard to AI,” NYU’s Reese said.
“We’ve always had this amorphous undefined end state that we don’t really understand but we have to beat China to get to. It’s really hard to race when we don’t even understand the path or even understand what the finish line is — or even if there is one.”
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
Released in theatres on 2 October 2026, Prem XL brings a youthful romantic comedy to Assamese cinema, blending the excitement of first love with friendship, fam...
