Safety and security
Nadella urges treating every AI model as compromised, and being able to stop it
Microsoft's chief executive proposes treating advanced AI as if it could fail from the start: with controls outside the model and an authorized person who can stop it. It is a proposal, not a rule.
The essentials
2 confirmed facts · 1 according to the source · 2 open questions
- Satya Nadella, chief executive of Microsoft, published a long post on X on October 10 about how to improve AI safety.12
- According to the sourceHe proposes separating the model from the system that coordinates its work, so that an authorized person can pause or shut it down mid-task.1
- He asks that every relevant action by the model be logged with evidence that people can read and that cannot be tampered with.12
Why it matters
The person making the call runs one of the largest companies in the sector, and does so as companies admit to more incidents in which they appeared to lose control of their models, according to TechCrunch. It is a proposal: it does not bind anyone.1
The details
According to TechCrunch, Nadella wrote that we must "assume a model is compromised" and contain it from the start, "like an emergency brake." The Verge adds that several of his recommendations match those of others in the sector: warning early about incidents and carrying out independent audits.12
What we don't know
- Whether Microsoft will apply these controls in its own products, and when.
- Where Nadella departs from what other companies are asking for: The Verge places it in containment, but the detail is not in the material.
Sources
- 1Microsoft’s Satya Nadella says AI models need an ‘emergency brake’
- 2Satya Nadella says we should assume all AI models are ‘compromised’
Was this story useful? Yes Not really