Skip to content

AI VERSE NEWS

Menu
  • Home
  • AI Tools
  • AI Breaking News
  • AI Tools & Products
  • Google AI
  • Privacy Policy
  • Terms of Use
  • contact
Menu

Anthropic says a key AI safety alarm has been triggered, days before its IPO

Posted on by admin

A week ago, Anthropic told would-be investors that AI speeding up its own research might “imminently” trigger one of the company’s key safety alarms. On Tuesday, in the same filing that set a price range for its record-setting IPO, it said that has now happened.

“We made a determination that we have met this threshold,” the company wrote in its amended prospectus. It based the call on “what we expect from future models and our overall rate of progress, rather than on what we have observed from current models.”

The safety threshold sits in Anthropic’s Responsible Scaling Policy — the company rulebook for adding tougher safety measures as its AI gets more powerful. This one is triggered when AI is doing enough of Anthropic’s research to roughly double how fast its models improve. The policy says that pace would pack two years’ worth of AI progress, at its 2018-2024 rate, into a single year. The filing says AI “has already been accelerating our research and development.” Crossing the line means more safeguards, not a shutdown.

The fear is AI will improve itself faster than people can check it. In 2023, Anthropic CEO Dario Amodei joined other AI leaders in warning that the “risk of extinction from AI” should be a global priority alongside pandemics and nuclear war.

Anthropic says humans stay in charge. It doesn’t expect to safely use Claude “to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding,” the filing says, and it is weighing policy changes “to keep humans in control.”

An outside group sees the same speedup. METR, an independent nonprofit that tests AI systems, estimated last week that Anthropic’s models have been improving nearly twice as fast since October 2025.

Bar chart comparing how fast Anthropic's AI models are improving. The pace before October 2025 is set at 1.0x. Since then it is 1.9x, nearly twice as fast, according to an estimate by the AI testing nonprofit METR.
METR, Yahoo Finance

In a separate review, METR called Anthropic’s August argument that it was still below the threshold “overstated” and said it was “plausible” the line was crossed by mid-July.

A $1.4 trillion debut

Anthropic set its price range at $70 to $78 per share on Tuesday as it began its roadshow, the round of investor meetings before pricing. That values it at as much as about $1.4 trillion, counting every share that could exist. Trading could start as soon as Nov. 19, according to reports.

Read more: Everything you need to know ahead of Anthropic’s IPO

Some of the market’s biggest names are tied up in the IPO. Nvidia (NVDA) and Alphabet’s (GOOG, GOOGL) Google are each buying $10 billion of Anthropic stock at the IPO price, part of a $26 billion private placement with AMD (AMD), Broadcom (AVGO), Micron (MU), Microsoft (MSFT), and Taiwan Semiconductor (TSM). Amazon (AMZN) and Google already own big stakes.

For investors, the risk factors, where companies list everything that could go wrong, paint a darker picture. They warn that after a catastrophic AI incident, “no single insurer, and potentially the insurance industry in the aggregate,” may have the capacity to cover the damage.

Not everyone takes the warnings at face value. “As in all S-1 filings, the risks are deliberately overstated,” Constellation Research founder Ray Wang said ahead of a Yahoo Finance live show appearance last month.

A new name for AI

The same filing carries a quieter change. Anthropic swapped in “super intelligence” for “AI” in places, citing President Trump’s Sept. 29 executive order telling federal agencies to use the term. Mentions of “AI” fell by about a quarter from the Nov. 2 version, by Yahoo Finance’s count, though the filing treats the terms as interchangeable.

It came two weeks after Trump knocked the Wall Street Journal on Truth Social for sticking with the phrase “artificial intelligence.”

Bar chart of how often Anthropic's IPO prospectus uses "AI" and superintelligence terms in each version. "AI" appears 774 times on Oct. 26 and 780 times on Nov. 2, then falls to 579 on Nov. 10. Superintelligence terms rise from 3 to 134.
Anthropic SEC filings, Yahoo Finance

As for the safety threshold, Anthropic said it is still weighing what crossing it means and will release further reports. It also flagged a bigger risk.

In the filing’s words, “As we increasingly leverage Claude to accelerate progress within Anthropic, our organization may lose visibility across certain processes in a way that may become difficult for any individual or group of humans to understand and control.”

Internal AI tools helped analyze interview transcripts, research, and draft this post. Humans reviewed, edited, and fact-checked this post before publication.

Follow Jared Blikre on X at @SPYJared or email him at jaredblikre@yahooinc.com.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • OpenAI acquires Odessia, travel startup created by Sonder founder
  • Grok Build coding agent leaks oodles of confidential info, some belonging to SpaceXSI
  • Apple has new Mac app coming to make running local AI models easy
  • AI Is Looking More and More Like a Normal Technology
  • Anthropic releases new Mythos and Fable models amid RSI concerns
©2026 AI VERSE NEWS | Design: Newspaperly WordPress Theme