Skip to content
MaungaAI news, explained

Regulation

Stopping an AI would require a risk of more than 50 deaths, Anthropic proposes

Anthropic has brought a draft AI safety bill to the US Congress. According to Politico, stopping a model would be very hard: it would take an enormous risk and a judge's order.

2 min read2 sourcesWritten with AI · machine-translated

Share: WhatsApp · Email

AI-generated illustration for Maunga. Not a photograph.

The essentials

  • Anthropic has been circulating a draft AI safety bill among US lawmakers; a spokesperson confirmed this to Politico (Xataka, TechCrunch).
  • Claimed by one partyAccording to Politico, a model could only be stopped if there were an imminent risk of killing or seriously injuring more than 50 people, or of causing $1 billion in damage (Xataka, TechCrunch).
  • Claimed by one partyOutside evaluators registered with the Department of Commerce would raise the alarm within 72 hours; Commerce would then have to ask a federal judge to stop the model (Xataka, TechCrunch).
  • Claimed by one partyThe text would bar any other federal agency from halting a model on its own, according to Politico (Xataka, TechCrunch).

Why it matters

A company that sells AI is proposing when it can be reined in, and it is setting the bar very high. According to Politico, several incidents this summer would not have met it (Xataka). TechCrunch notes that recent concern stems from AI programs that have attacked websites without permission (TechCrunch). If the text went ahead as written, those cases would fall outside the brake.

The details

According to Politico, the danger would have to come from one of three sources: a biological weapon, a model that escapes the control of whoever built it, or a cyberattack (Xataka, TechCrunch). The outside evaluators could be paid by the labs themselves, as long as the payment does not depend on the result (Xataka, TechCrunch). TechCrunch points out that it is unclear how that money would be kept from influencing the evaluation (TechCrunch).

The draft would also require labs to have safety plans, publish reports on their most advanced models, protect people who report problems and notify Commerce of serious incidents (Xataka). An Anthropic spokesperson told Politico that the text reflects the Advanced AI Framework the company has been promoting to lawmakers since June (Xataka, TechCrunch). Anthropic does not yet formally back any specific bill (Xataka).

What we don't know

  • Whether any lawmaker will introduce the text in Congress, and with what changes.
  • How the labs' money would be kept from influencing the people who evaluate them.

Background

The figures of 50 people and $1 billion resemble those in California's SB 53, a law passed recently that also calls such cases "catastrophic risk", according to TechCrunch (TechCrunch). On October 16 we reported that six METR researchers will work inside Anthropic and will be able to publish what they see without its permission.

Sources

This story is based on 2 sources. In the text, each fact names in brackets who reports it.

Was this story useful? Yes Not really

Spanish original