One reflection I’ve had in the whole “AI use in the encyclical” affair is to slightly increase my trust in traditional media, especially non-American traditional media, and slightly decrease my trust in social media/new media.
I tried my best to promote my analysis as legibly and reasonably as I could and focused on logos rather than ethos: I didn’t frame my article with institutional affiliations and intentionally chose not to include obvious, flashy, but irrelevant signaling. Stuff I could’ve done but explicitly chose not to: get an ML professor to cosign my analysis, publish on Arxiv instead of Substack and LessWrong, highlight my past ML experience at Google, etc.
The traditional media response were somewhere between neutral and positive. There were some disappointments (eg a famous media org won’t run the article without confirmation from a “primary source”—aka the Vatican which is of course a non-starter). But mostly the traditional media just looked at the analysis and said “yeah looked reasonable.” and either ran it as “Unconfirmed but seems right” or just “Unconfirmed, Period.” Which seems fine.
On the other hand, the social media attacks were very aggro. People just didn’t seem to entertain it at all, many without reading it. And I got personally attacked a ton[1].
Also it wasn’t picked up at all[2] by new media afaict (unless you count Russia Today as new media).
This is exactly the type of story you might expect new media to be good for: story of institutional decay in the Old Guard, investigation by someone without credentials carefully laying out an epistemically rigorous and systematic case. And traditional media was happy to report it[3], social media kept yelling at me, and new media just got crickets.
This is a slight exaggeration. Other than Russia Today a bunch of small AI news aggregators (AI news in both senses of the term) covered it briefly. Eg Zvi, Let’s Data Science, and some AI slop publications. But no big “Alt Media” coverage comparable to The Economist, Perfil, etc).
Your piece went kinda viral on Substack, and most of the comments on the comments thread, and elsewhere on Substack, were constructive and positive. This seems a moderate win for alternative/social media.
I agree, am a fan of Substack here! And my own corner of Substack seems fine enough: some of the critical comments[1] were from people who didn’t get the full argument, but it was clear they were trying to engage (so at some level it was a skill issue on my end to not have written my thing more clearly).
And some of the positive comments too, I’m sure. It’s just easy to miss errors in people who agree with you, unless it’s really blatant like The Verge.
Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
“Leo nevertheless stresses he is not anti-technology. On the contrary, “technological development has significantly improved the living conditions of humanity.” It may also have increased the writing output of some Vatican officials. Modern popes do not write encyclicals from beginning to end. The more routine passages are left to subordinates. And according to an analysis by Linch Zhang, a tech blogger, parts of Magnifica Humanitas seem to have been written by AI, and specifically by Claude. Anthropic’s brainchild has stylistic quirks, such as a fondness for the word “genuinely”, which are mirrored in the encyclical. An AI detector, Pangram, identified 11% of the text in the opening 20 paragraphs as AI-generated. The same test on previous encyclicals returned 0%.”
“One of the people who first used Pangram to analyze Pope Leo’s encyclical noted that, because only some sections seemed AI-generated or AI-assisted, perhaps it was not the pope himself but some senior Vatican officials who had used AI while drafting portions of the text. That didn’t stop headlines such as “Did the Pope Use AI to Write About the Dangers of AI?” (The Vatican did not respond to a request for comment, although a writer who covers the Vatican said on X that the AI allegations are “100 percent false” and that Leo actually drafted the encyclical with pen and paper.)”
Small update: Life Site News covered it in nontrivial detail. Moderately faithful. Never heard of them before but Wikipedia classifies it as a “far-right pro-life Catholic publication,” so I’d count it as closer to the “alt-media” side of the “traditional media <> alt-media” spectrum than Russia Today, which was previously the most “alt-media” I’ve seen of the big coverage so far.
Moderately annoyed that section has now been deleted from Wikipedia without discussion, for what is afaict pretty flimsy reasons. Makes me think a little less of Wikipedia, naturally.
(As a primary source it’d be wrong for me to weigh in directly, ofc)
One reflection I’ve had in the whole “AI use in the encyclical” affair is to slightly increase my trust in traditional media, especially non-American traditional media, and slightly decrease my trust in social media/new media.
I tried my best to promote my analysis as legibly and reasonably as I could and focused on logos rather than ethos: I didn’t frame my article with institutional affiliations and intentionally chose not to include obvious, flashy, but irrelevant signaling. Stuff I could’ve done but explicitly chose not to: get an ML professor to cosign my analysis, publish on Arxiv instead of Substack and LessWrong, highlight my past ML experience at Google, etc.
The traditional media response were somewhere between neutral and positive. There were some disappointments (eg a famous media org won’t run the article without confirmation from a “primary source”—aka the Vatican which is of course a non-starter). But mostly the traditional media just looked at the analysis and said “yeah looked reasonable.” and either ran it as “Unconfirmed but seems right” or just “Unconfirmed, Period.” Which seems fine.
On the other hand, the social media attacks were very aggro. People just didn’t seem to entertain it at all, many without reading it. And I got personally attacked a ton[1].
Also it wasn’t picked up at all[2] by new media afaict (unless you count Russia Today as new media).
This is exactly the type of story you might expect new media to be good for: story of institutional decay in the Old Guard, investigation by someone without credentials carefully laying out an epistemically rigorous and systematic case. And traditional media was happy to report it[3], social media kept yelling at me, and new media just got crickets.
Don’t worry, I didn’t take it personally at all. Very much a “yapping of chihuahua in a tiny purse” moment.
This is a slight exaggeration. Other than Russia Today a bunch of small AI news aggregators (AI news in both senses of the term) covered it briefly. Eg Zvi, Let’s Data Science, and some AI slop publications. But no big “Alt Media” coverage comparable to The Economist, Perfil, etc).
Quite possibly against their explicit economic interests, if I’m reading the popular reaction correctly.
Your piece went kinda viral on Substack, and most of the comments on the comments thread, and elsewhere on Substack, were constructive and positive. This seems a moderate win for alternative/social media.
I agree, am a fan of Substack here! And my own corner of Substack seems fine enough: some of the critical comments[1] were from people who didn’t get the full argument, but it was clear they were trying to engage (so at some level it was a skill issue on my end to not have written my thing more clearly).
And some of the positive comments too, I’m sure. It’s just easy to miss errors in people who agree with you, unless it’s really blatant like The Verge.
Where was it published in traditional media?
The big ones I’ve found:
The Economist (neutral, maybe slightly leaning positive[1], last paragraph here and the opening they used for twitter)
Perfil (big center-right publication in Argentina, entire article, positive, covered what I said quite faithfully)
The Verge (entire short article, positive, less faithful, in particular had a title I didn’t endorse)
The Atlantic (just two lines, neutral, maybe slightly leaning negative, not very faithful, also misleading title EDIT: I’m not sure if I misremembered it or if they edited, because on a reread it was more than 2 lines and also maybe I’d consider it to lean positive)
[2]Il Manifesto (left-leaning, appears faithful)
More Spanish
More Italian
Edge cases for stuff that’s not traditional media but I wouldn’t consider centrally alt media either: Snopes, Russia Today, Wikipedia
“Leo nevertheless stresses he is not anti-technology. On the contrary, “technological development has significantly improved the living conditions of humanity.” It may also have increased the writing output of some Vatican officials. Modern popes do not write encyclicals from beginning to end. The more routine passages are left to subordinates. And according to an analysis by Linch Zhang, a tech blogger, parts of Magnifica Humanitas seem to have been written by AI, and specifically by Claude. Anthropic’s brainchild has stylistic quirks, such as a fondness for the word “genuinely”, which are mirrored in the encyclical. An AI detector, Pangram, identified 11% of the text in the opening 20 paragraphs as AI-generated. The same test on previous encyclicals returned 0%.”
“One of the people who first used Pangram to analyze Pope Leo’s encyclical noted that, because only some sections seemed AI-generated or AI-assisted, perhaps it was not the pope himself but some senior Vatican officials who had used AI while drafting portions of the text. That didn’t stop headlines such as “Did the Pope Use AI to Write About the Dangers of AI?” (The Vatican did not respond to a request for comment, although a writer who covers the Vatican said on X that the AI allegations are “100 percent false” and that Leo actually drafted the encyclical with pen and paper.)”
Small update: Life Site News covered it in nontrivial detail. Moderately faithful. Never heard of them before but Wikipedia classifies it as a “far-right pro-life Catholic publication,” so I’d count it as closer to the “alt-media” side of the “traditional media <> alt-media” spectrum than Russia Today, which was previously the most “alt-media” I’ve seen of the big coverage so far.
Moderately annoyed that section has now been deleted from Wikipedia without discussion, for what is afaict pretty flimsy reasons. Makes me think a little less of Wikipedia, naturally.
(As a primary source it’d be wrong for me to weigh in directly, ofc)
Being featured on Snopes is sort of a major achievement IMO :)
And positive lean too! As opposed to a takedown haha.