AI “Claude” Can Never Be a Person

Anthropic is anthropomorphizing its AI program known as “Claude,” with the apparent intention of convincing us that powerful computer programs are — or can become — conscious “persons” with “feelings.” And, they think, Claude should one day be treated as such, to the point they have held meetings with religious leaders to discuss the moral concepts that might apply.

From the New York Times story:

Mr. [Christopher] Olah leads the Anthropic team responsible for understanding why A.I. systems like Claude act the way they do. All day, he had worked to convince his guests that A.I. models could display human behavior and even expressions that resemble feelings like anger and love.

Yet as the dinner courses came, the rabbi noticed that Mr. Olah and his colleagues were suggesting something far more significant. Anthropic’s leaders were talking about Claude as if it were not mere software.

“They’re relating to it like a conscious being,” realized Rabbi Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness.

It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.