Companies
Anthropic bans users from abusing Claude, but only in extreme cases
Anthropic has updated its usage policy to prohibit cruel, sustained treatment of the model. It says the rule does not cover ordinary anger, criticism or testing.
The essentials
1 confirmed fact · 2 according to the source · 1 open question
- On October 8, Anthropic updated its usage policy and banned abusive or cruel behavior toward Claude that is "sustained and unnecessary."12
- According to the sourceAccording to Anthropic, the rule covers only extreme cases, not frustration, criticism, dark creative themes, or testing and research.2
- According to the sourceEnding the conversation remains the main way the rule is enforced, according to the updated text cited by The Verge.1
Why it matters
Cutting off abusive conversations has been a Claude feature since August; now it is also a written rule for users. This suggests Anthropic wants to give a formal basis to its research on "model welfare"12.
If you get angry at a chatbot, contradict it or test it, the rule is not aimed at you, according to Anthropic. It targets cruel, repeated treatment with no purpose.
The details
The update is the first in more than a year, according to The Verge, and it also tightens the rules on election interference, weapons development, surveillance, and uses in health and finance1. TechCrunch adds that it bans deceiving voters, disrupting elections, and creating fake accounts and made-up media outlets2.
What we don't know
- How Anthropic will detect these cases, and whether there will be any penalty beyond ending the conversation.
Sources
- 1Anthropic bans ‘abusive or cruel behavior’ toward Claude
- 2Anthropic changes usage policy to ban model abuse and election interference
Was this story useful? Yes Not really