Skip to content

AI VERSE NEWS

Menu
  • Home
  • AI Tools
  • AI Breaking News
  • AI Tools & Products
  • Google AI
  • Privacy Policy
  • Terms of Use
  • contact
Menu

Opinion | Don’t Play Anthropic’s Game

Posted on by admin

By the close of the stock market today, Anthropic will have become a central pillar of the American economy. This is deeply concerning to me, because it’s unclear how many of the retail and institutional investors handing over their money to Anthropic realize that they are, in some sense, funding a doomsday cult that also happens to build enterprise software.

Anthropic’s chief executive, Dario Amodei, along with many in his company, believes that superintelligent A.I. will radically transform human existence as we know it, leading either to a utopia or an apocalypse.

Anthropic executives see their role as shepherding humanity toward the good place and, if they fail, becoming the somber voice of warning as we descend into the bad. To protect these righteous efforts, they’ve structured their company to minimize influence and oversight from outsiders who might not share their radical understanding of the future.

We should certainly debate whether Anthropic’s end-time visions are plausible, or if the company’s underlying ethics are sound. But the question of the moment is whether these beliefs are compatible with being a publicly traded company valued at over a trillion dollars.

Anthropic is happy to take our money to fund its vainglorious efforts to reshape the human story. Its executives hope we’ll sit quietly and let them get on with their grand mission. We shouldn’t be so compliant.

Trying to generate returns for your shareholders and trying to save the world are two very different activities. Anthropic’s savior complex has led it to charge forward at reckless speed. It’s on track to release more A.I. model versions this year than it did in the last three years combined — roughly one new model every three weeks.

But even this isn’t fast enough for its liking. In June, Anthropic published an essay describing its commitment to recursive self-improvement, a controversial strategy in which you let A.I. systems autonomously upgrade themselves.

The goal for this relentless pace isn’t primarily to gain market share or be the first to discover profitable innovations, but to ensure that Anthropic wins the race to develop superintelligence. Mr. Amodei thinks these stakes are existential, warning that if China beats Anthropic, the country will achieve “geopolitical dominance” — a jargony way of saying that whoever wins the race will unleash a force capable of conquering the world.

The problem with this mind-set is that when you believe the fate of the world depends on you staying ahead of your competitors, you don’t care as much about collateral harms caused along the way. Anthropic, for example, recently loosened internal rules requiring it to follow well-defined safety guardrails, suggesting such a restriction would cause the company to fall behind. The result has been hasty releases of models with capabilities that were ill understood or controlled — a reality sharply captured by our panic in recent days over cybersecurity.

Mr. Amodei should be given credit for caring about the potential downsides of superintelligence. Much more so than its competitors, Anthropic has maintained a well-funded research group committed to publishing reports that detail safety concerns about its own products. But this obsession with the future has led the company to make decisions in the present that are bad for its business and dangerous for all of us.

As investors realize that Mr. Amodei is potentially motivated more by techno-philosophy than by the bottom line, there will be significant pushback. A company this large can’t get away with acting this weird.

Anthropic seems to recognize this problem. As its filings for its initial public offering reveal, it has gone to extraordinary lengths to structure itself as a public company in such a way that the public will have minimal influence on its behavior. It’s officially a “public benefit corporation,” meaning a for-profit company that can pursue a specified public benefit in addition to the traditional requirement to maximize shareholder value.

This might seem like a good thing, but it weakens what would otherwise be the strongest check on Anthropic’s behavior: its investors. Imagine, for example, that Anthropic turns to full recursive self-improvement, letting A.I. models upgrade themselves, and this leads to increasingly hard-to-control systems with limited business value. Shareholders might file suit, citing the competitive shortcomings of this strategy. But Anthropic could then leverage its public benefit status to argue that recursive self-improvement, though harmful at that moment, is a necessary step toward its utopian visions for humanity’s future.

Another avenue for shareholder influence would be to elect more moderate board members. But Anthropic thought about this as well. In a highly unusual move, the company has given control of four of its seven board seats to what it’s calling a Long-Term Benefit Trust, charged with making sure that Anthropic’s mission to preserve humanity remains a core focus. Two of the remaining board seats are held by Mr. Amodei and his sister, Daniela Amodei, Anthropic’s president. These seats are arguably also safe given that the company has been careful in its stock allocations to ensure that its seven founders collectively hold 50.1 percent, giving them a controlling stake.

These seem like the actions of individuals who know they’re up to something unusual and don’t want outsiders to be in a position to stop them.

Stock-issuing corporations are intended to return value to investors by expanding lucrative markets and developing useful products. Staving off apocalypses and pursuing utopias should instead be the work of advocacy groups, nonprofit institutes, political lobbying and public information campaigns.

Mr. Amodei is welcome to use his personal fortune to fund an organization dedicated to A.I. safety, or to deploy his political influence to lobby strongly for, say, a government-sponsored Manhattan Project to build superintelligence. But to try to integrate these nonbusiness goals into the often unrelated concerns of building a durable technology company is asking for trouble.

Consider if Mr. Amodei is wrong, and we’re not actually on an inevitable trajectory toward A.I. capable of “geopolitical domination.” This might seriously jeopardize the financial health of Anthropic. Its strategy of investing billions upon billions of dollars to churn out ever more expensive language models faster than it can create useful products doesn’t seem sustainable. It’s a business plan that starts with a lot of red ink and then abruptly ends with superintelligence arriving and rendering money superfluous.

On the other hand, if Mr. Amodei is right, this would put the fate of our collective humanity into the hands of a small group of technology billionaires steeped in ethically questionable ideas about a transhumanist future and unanswerable to the public. This can’t be the best way forward. Anthropic’s strategy of raising as much money as possible from the public markets to fund its idiosyncratic visions for saving humanity is a hasty improvisation. It’s the sort of “better than nothing” move made in an environment where more traditional responses have been confused or muted.

In this way, I don’t fault Mr. Amodei and his company for attempting this experiment in mixing capitalism with ideology, but this doesn’t mean that we should accept it. The strangeness of what Anthropic is up to should serve as a warning sign. We need to demand stronger action from Congress to help us take more control over our technological future. We need a more cohesive voice coming from the academic and research communities. And we should also expand upon the newfound energy among Americans who are wary of A.I. and who voted to radically overhaul the government this month.

In the meantime, I suspect Anthropic’s attempts to nominate itself as the leader of our species’ next phase of existence will ultimately falter. Regardless of the legal protections they’ve enacted, or the intensity of their righteousness, elitists are not popular with the American people. And what could be more elite than nominating yourself to be a techno-king?

In a September interview, Mr. Amodei reflected on Anthropic taking on so much responsibility. “People ask me that question all the time: ‘Why isn’t this being built by the government?’” he said. “And the strangest thing about it is, I agree with them. I’m uncomfortable.” We all should be uncomfortable too.

Cal Newport, a contributing Opinion writer, is a professor of computer science at Georgetown University and the host of “The Cal Newport Show” podcast.

The Times is committed to publishing a diversity of letters to the editor. We’d like to hear what you think about this or any of our articles. Here are some tips. And here’s our email: letters@nytimes.com.

Follow the New York Times Opinion section on Facebook, Instagram, TikTok, Bluesky, WhatsApp and Threads.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

  • Opinion | Don’t Play Anthropic’s Game
  • Microsoft CEO Satya Nadella explains why he asked 60K employees to quit
  • AI Plush Cat Described Horrifying Acts to Preschoolers. Three Retailers Took Action.
  • The link between ‘patchmageddon’ and recent tech outages
  • PANW Stock Surges 8% Overnight: Palo Alto Networks CEO Sees ‘Hundreds Of Billions Of Dollars’ Opportunity In Open-Weight AI Models Security
©2026 AI VERSE NEWS | Design: Newspaperly WordPress Theme