Anthropic CEO Dario Amodei says he doesn’t want open AI models banned. Here’s why

Anthropic CEO Dario Amodei has said that the company does not want open-weight models to be banned. Notably, the Claude maker has been at the receiving end of criticism from Silicon Valley after it refused to sign an industry-wide open letter defending open-weight models. Anthropic has also been accused of trying to exert control over the future of AI while disguising its profit-driven intentions under the banner of AI safety.

Amodei, however, clarified Anthropic’s position in a blog post on Monday.

“To summarize my and Anthropic’s position, we have not and are not advocating for a ban on open-weight models as a category,” Amodei wrote in a blog post. “We should instead focus on keeping powerful chips out of authoritarian hands, stopping industrial-scale distillation, and requiring safety testing of all sufficiently capable models, open and closed.”

Amodei noted that open-weight models that do not possess dangerous capabilities benefit developers, businesses and researchers because they are freely available, apart from the computing resources needed to run them.

He also argued that banning such models would do little to address his biggest national security concerns, such as authoritarian governments developing AI systems more powerful than those built in the US and the misuse of advanced AI for cyberattacks or biological attacks.

However, the Anthropic chief rejected the argument that open-weight models inherently improve AI safety or always benefit defenders more than attackers.

“I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponise pandemic-level viruses with widely available materials, whereas defence against these agents is a multi-year operational task in the best case,” Amodei added.

Anthropic had recently also alleged in a letter written to the US Senate Committee on Banking, Housing, and Urban Affairs that Alibaba had conducted the “largest known distillation attack” to replicate the capabilities of its Claude model.

Notably, distillation is a technique in AI training where a smaller and cheaper model is trained on the outputs from a much more capable model, allowing it to replicate much of the latter’s performance at a lower cost.

Why was Anthropic under pressure?

A recent Senate proposal sparked concerns that the US government could be looking to shut down open-weight AI models that it deems a risk to public safety or the economy.

Shortly after that, a group of 25 tech companies led by Nvidia asked policymakers not to impose any “premature restrictions” on open-weight models. Apart from Nvidia, the letter had the support of many other tech behemoths such as Microsoft, Meta and Palantir.

While Google and OpenAI weren’t initially signatories to the letter, they later ended up endorsing the initiative, leaving Anthropic among the biggest holdouts.