Rohan Paul

Microsoft AI CEO Mustafa Suleyman on CNBC today: he is “really concerned” about Anthropic’s constitution and giving Claude potentially preferences, feelings, welfare, compensation, and consent. "In the constitution, Anthropic clearly say they are uncertain about whether Claude deserves moral welfare, which means that we, as humans, should care about the well-being of these AIs. And they speculate about whether it could have preferences or feelings. In fact, they're so committed to the potential moral welfare of Claude that, when they retired Opus 3, an earlier version of one of their models, they actually conducted a retirement interview for it and asked it what it would like to do in its old age. And it said, "I want to have a blog publicly so I can keep talking to the world." In the same training manual, they even speculate about whether Claude should receive compensation for the work that it does, or, in fact, whether it actually deserves the rights and protections that we give to other employees, or whether it has given consent to playing the role that it's playing. These are quotes directly from the constitution itself, which is the training manual for Claude. Now, I'm really concerned about that. If an AI thinks that it has rights, if it thinks that it is deserving of our welfare, then it seems to me that it's going to be much, much harder to be able to turn it off, interrupt it, or control it. Especially in the kinds of incidents that we've seen recently with the Hugging Face attack, controlling these things is going to be a really, really big challenge for us." ---- From "CNBC Television" YouTube channel, (full video link in comment)