Can AI be moral without being sentient?

Pope Leo XIV and one of the world’s leading AI companies recently found themselves on opposite sides of a thorny question: Can a chatbot have a conscience? A New York Times investigation published late last month revealed that Anthropic cofounder Chris Olah proposed withdrawing from a Vatican event to launch Pope Leo XIV’s first encyclical after reading an advance copy, which rejected the possibility of sentient AI. Paragraph 99 of Magnifica Humanitas states bluntly that AI models “do not undergo experiences, do not possess a body, do not feel joy or pain.” Nor do they “have a moral conscience.” The disagreement reflects a widening debate over whether AI systems could, or should, ever achieve moral personhood. Anthropic spent months consulting religious scholars and ethicists about the possibility of AI consciousness, and about how to instill moral values in its models. But the Vatican—and many secular thinkers—maintain that while AI may imitate human intelligence and moral judgment (a chatbot might generate sound ethical advice without understanding its consequences), genuine morality is unique to the human experience. The debate has implications far beyond theology, especially a…