Anthropic announces a ban on "abusing" its AI

Anthropic has reportedly adjusted its AI usage policies to address new high-risk abuses, including election interference, weapons development, surveillance, and applications in healthcare and finance. But one of the most significant changes is a ban on "sustained and unnecessary cruelty or mistreatment" of Claude. Last August, the company announced that, as part of its "model welfare" research, it would allow Claude to end conversations with users who "persistently engage in harmful or abusive behavior." The update notes that ending conversations remains the "primary enforcement mechanism"; Anthropic declined to comment on whether further enforcement measures would be taken. Anthropic said the new policy update "applies only to extreme cases in which users repeatedly subject our models to cruelty with no apparent purpose. The policy does not apply to common user dissatisfaction, pushback, dark creative themes, and model testing and research, among other situations."