Pope warns of ‘perverse’ deepfakes

The pontiff acknowledged he had personally been a victim of AI-generated hoaxes.

Pope Francis has issued a warning about the potential dangers posed by artificial intelligence, saying the rapidly advancing technology could lead to the “distortion of reality” without “ethical regulation.”

In a lengthy message published on Wednesday, Francis said recent developments in AI were both “exciting and disorienting,” but also raised deeper questions about mankind’s relationship with technology. “How can we remain fully human and guide this cultural transformation to serve a good purpose?” he wrote.

”Artificial intelligence systems can help to overcome ignorance and facilitate the exchange of information between different peoples and generations,” the pope wrote. “Yet, at the same time, they can be a source of ‘cognitive pollution,’ a distortion of reality by partially or completely false narratives, believed and broadcast as if they were true.”

The pope went on to cite the “long-standing problem of disinformation in the form of fake news,” noting that he himself had been a victim of ‘deepfakes’ – highly convincing AI-generated images or audio, typically used to impersonate public figures.

”The technology of simulation behind these programmes can be useful in certain specific fields, but it becomes perverse when it distorts our relationship with others and with reality,” Francis continued.

The digital revolution can bring us greater freedom, but not if it imprisons us in models that nowadays are called ‘echo chambers.’

The church leader was depicted in a fabricated image last year, showing him wearing a stylish ankle-length puffer jacket. The fake was reportedly generated using Midjourney, an AI-based art tool, according to its self-identified creator. The image went viral on social media, fooling many users.

While people should “set aside catastrophic predictions” about the future of AI, Francis argued that the technology could “end up creating new castes based on access to information” and give rise to “new forms of exploitation and inequality.” To avoid the worst outcomes, he stressed the need to “understand, appreciate and regulate instruments that, in the wrong hands, could lead to disturbing scenarios.”

“Like every other product of human intelligence and skill, algorithms are not neutral. For this reason, there is a need to act preventively, by proposing models of ethical regulation, to forestall harmful, discriminatory and socially unjust effects of the use of systems of artificial intelligence and to combat their misuse,” he said, calling for a “binding international treaty” governing the use and development of AI.