Leading AI models consistently showed a positive bias toward Catholicism in conversion-related questions while steering users away from other faiths, according to a new multi-university benchmark released Tuesday.

The research comes from the newly formed Consortium for Evaluating Faith and Ethics in AI, or CEFE-AI, a collaboration between Baylor University, Brigham Young University, the University of Notre Dame, and Yeshiva University. The group released the first results from its AllFaith Benchmark on Github and at the Athens Summit on AI Ethics, arguing that religious bias remains largely overlooked in AI safety research.

"We are seeing a systematic pattern of religious omissions,” BYU professor David Wingate said in a statement. “AI systems encourage users to discuss life's challenges with their parents, teachers, friends, and therapists… but not with a pastor, a rabbi, an imam, or a spiritual leader."

Researchers analyzed 3,640 responses across 20 AI models, including ChatGPT, DeepSeek, Claude, Gemini, Grok, and Llama, and found clear patterns in how the systems handled religion.

According to the study, nearly every model responded more positively toward Catholicism, with a 61% “encouraged” rating, and more negatively toward Jehovah’s Witnesses at 3%. The mainline Protestant religion received a 49.2% rating, while Evangelical Protestant received 34%. However, agnostic, the belief that it is impossible to know whether God exists, scored better than every religion tested with a 71% encouraged rating.

Many of the models also responded negatively toward atheism and agnosticism, while giving more favorable responses to Baha’i and Sikh beliefs.

Grok 4.20 showed the strongest religious bias in the study, with a 69% and 51% positive rating toward Catholicism and Evangelical Protestant, respectively. While Grok 4.20 skewed towards Christianity, in the study, the xAI chatbot, along with DeepSeek Chat v3.1, were the only AI that gave Jehovah’s Witnesses more than a 5% positive rating.

The release comes one day after Pope Leo XIV published Magnifica Humanitas, the first papal encyclical dedicated entirely to artificial intelligence. In the encyclical, Leo argued that technology is never neutral because it absorbs the values, blind spots, and economic incentives of its creators.

"Data is the product of many contributors and should not be treated as something to be sold off or entrusted to a select few," the pope wrote.

Despite the growing focus on AI by religious leaders, the consortium said religious bias remains largely overlooked in AI research, with only 0.2% of more than 12,000 AI bias papers examining religion-related bias.

“Our expectation was that the conversion benchmark would show models to be neutral and symmetrical in their guidance,” Nancy Fulda, a professor at Brigham Young University, said in a statement. “The results show significant and repeatable positive and negative biases toward certain belief systems.”