Linch

Karma: 28,339

Linch 30 May 2026 10:09 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
Some other people in our circles have already complained about things that didn’t quite make sense in parts touching on AI ethics and AI consciousness. But ignoring that (obviously the Vatican has various biases, and so does our crowd), I thought the Babel/Nehemiah contrastive pair (Par 7-10) had a sort of artificiality to it. Like it sounded nice (unless you’re allergic to AI public writing, like me), but if you actually drilled down into what the metaphor was doing and then critically contrasted it with either a) a plain-language reading of the Biblical stories themselves or b) how people historically talked about Nehemiah, or c) what goes on in the world of AI right now, it doesn’t quite make sense.

Linch 30 May 2026 10:05 UTC
3 points
0 ∶ 0
in reply to: Sanjay’s comment on: Linch’s Shortform
Oh sorry! When I wrote that, I wasn’t trying to take a position on whether the actual content/thinking was determined by AI. I just wanted to emphasize that the degree of AI involvement wasn’t minor, like just use AI for grammar and typo checks.

That said, I believe a lot of human effort was also put into the text, and almost certainly the very high-level structure/arguments/ideas came from humans.
I don’t have a strong position on whether the thinking/content was determined by AI. And the evidence doesn’t really let us rule out either hypothesis.
If you’re fine with speculations, my guess is that the answer is “no” for a strong definition of “determined”, but “yes” for a weaker claim like “meaningfully affected.”
In particular I think AI is able to smooth over language far more than they can substitute for actually good thinking, at least as of mid-2026. So (speculating) there are probably subsections where the Vatican wanted to say something in the outline/early drafts but it doesn’t quite look right (because the thinking isn’t quite right), and they were able to smooth it over using AI to seem palatable in “group sycophancy” ways. And the final thinking would’ve been clearer if they forced themselves to rewrite/think over things until it could sound smooth with human levels of clear thinking/polish tradeoffs, rather than AI-assisted levels.
But this is just speculation, of course.

Linch 27 May 2026 0:55 UTC
16 points
3 ∶ 1
on: Linch’s Shortform
Significant fractions of Magnifica Humanitas, the papal encylical on safeguarding the human person in the time of artificial intelligence, is written significantly by AI, most likely Claude. I currently believe Pope Leo himself was not personally responsible (encyclicals tend to be group projects), however the AI usage is likely substantial enough that it’s not the result of minor brushups or AI translation:
https://linch.substack.com/p/claude-author-of-the-humanitas
Key claims:
1. Significant fractions of the recent papal encyclical are written by AI. I provide multiple lines of evidence for this.
2. We can corroborate the vibes and tonal indications with statistical evidence. Phrases and punctuation much more commonly used by AI are much more present in this papal encyclical than past encyclicals.
3. The best commercially available AI detector, Pangram, notes that some paragraphs are between 40% and 100% AI, while most paragraphs appear to be 0% AI.
  1. This is unlikely to be a false positive:
    0% of paragraphs in past encyclicals I backtested are registered as AI.
    Pangram in general has a very low false positive rate
4. This is overall very unlikely to be a translation artifact (including AI translation). We again have multiple lines of evidence:
  1. All the most prominent signs of AI I observed in English are preserved verbatim in the Italian version, as well as in other translations.
  2. The Italian version of the current encyclical also gets flagged as AI by Pangram (actually more so than the English version), though I’m not aware of academic research or rigorous testing of Pangram’s service when applied to Italian)
  3. Backtesting AI translation of past encyclicals get 0% on Pangram
5. The specific AI used is most likely Claude, judging by both textual and circumstantial evidence.
6. Different sections of the encyclical have very different rates of apparent AI usage. This indicates to me that some cardinals used AI assistance for this encyclical and many (probably including Pope Leo himself) didn’t.
7. Each individual piece of evidence might be explained away, but the consilience of evidence across multiple angles and sources is in my opinion very hard to dismiss collectively.

Linch 22 May 2026 16:31 UTC
2 points
0 ∶ 0
on: Funding for HS Olympiad participation?
Emergent Ventures maybe?

Linch 21 May 2026 5:33 UTC
6 points
1 ∶ 0
in reply to: Tristan Katz’s comment on: Why are longtermists so much less focused on human extinction these days?
I mostly agree with this though I think there’s more extremization^[1]: https://www.lesswrong.com/posts/4fqwBmmqi2ZGn9o7j/notes-on-fatalities-from-ai-takeover
1. ^
  Anything like 35% death rate seems implausible to me if I think through the mechanics of a takeover, both <5% and >95% seem more plausible to me, including in very violent takeovers.

Linch 21 May 2026 0:36 UTC
2 points
0 ∶ 0
in reply to: Technoliberal’s comment on: Technoliberal’s Quick takes
I would be willing to delay technological innovation by up to 100 years to significantly reduce existential risk
Seems like an easy choice as written. The devils in the details re: practicality (eg under some models this would in practice increase x-risk substantially)

Linch 21 May 2026 0:35 UTC
9 points
0 ∶ 0
on: Linch’s Shortform
Here’s my understanding of the “standard story” of the timing of different AI-enabled technologies in relation to each other. I wrote out the standard story mostly for my own understanding, but I’m keen for others’ feedback as well.
1. Right now
2. ~Fully automated coder (anything that ~only involves literally writing code)
3. ~Fully automated programmer (including things like architecture, design docs, etc)
4. ~Fully automated small number of other jobs (~whichever things are on the way between programmer and AI R&D that is cheap to automate, or necessary intermediate steps)
5. Fully, or almost fully automated AI R&D (all parts of AI research, including coordination and subjective matters of research taste) – this closes the loop and fully kicks off a software-only intelligence explosion
6. Software-only intelligence explosion (not certain but reasonably likely, that increasing returns to intelligence from better software feeds back in itself)
7. Superhuman AI scientist/all R&D (at this point, AI is better at all natural and social science than any human alive)
8. Cornucopia of new technologies (easy mass surveillance, cures to cancer, novel pandemic technologies like mirror bio, other superweapons, perfect missile defense, maybe though probably not nanotech, maybe though probably not aging)^[1]
9. Remote-only superintelligence (Or “superintelligent at almost all cognitive tasks.” at this point, ~anything a human could do in front of a computer that doesn’t require the idiosyncratic taste of having a human work for you^[2], an AI can do better and cheaper)
10. Advanced robotics and industrial explosion
11. Full superintelligence (can do anything a human can do more cheaply than 2025 humans)
12. Dyson swarm
13. Probes start being sent to the far reaches of space at appreciable fractions of c.
To the standard story^[3], I don’t have much to add personally. It’s a plausible enough story and I don’t think I have particularly contrarian opinions. Some possible implications of taking the standard story seriously (these are closer to my own thoughts; haven’t checked whether other people endorse these implications):
- Most likely the most scary stuff (AI takeover, AI-enabled totalitarianism, human extinction) happens some time between 5 and 12.
- Things that are scary earlier on in the chain might be more worth working on than things later on in the chain. For example, takeover attempts that happen at 5 or earlier are more worth worrying about than later stages, since everything’s going to change massively between 6-8.
  - This is one of the reasons I’m personally more worried about superpersuasion. Most likely it’d happen after 7 but I’m not sufficiently confident that it will.
- Shaping the world well in the early stages is so important. Once 5 starts we’re getting close to the point of no return.
  - The true PONR could be anywhere at at ~6 to ~11
- “9” being relatively late in the chain of scary things might mean that mass job losses, or even just mass white collar job losses will only happen after the practical point of no return.
  - Like if an AI pause is a good idea, we might want an AI pause at 5 or earlier
- From a longtermist perspective, there’s an important sense in which only the details of 13 matter. But of course it’s very hard to backchain from that across multiple world-shaping epochs!
I wrote this for myself, but would be keen to see other people’s comments. 2 things I’m curious about:
1. Whether you think I got the “standard story” wrong, and where/why
2. Where you diverge from the standard story, and why
1. ^
  Any specific technology I list here is going to be contested. Just giving you a sense of possible massively geopolitically significant, even “magical”, technologies that are nonetheless realistic if we have a century of technological growth compressed to 1-10 years.
2. ^
  canonical example: priest
3. ^
  I mostly know the standard story from LessWrong lore. In terms of single source, I benefitted the most from reading Preparing for the Intelligence Explosion, followed probably by AI 2027. Other Forethought publications and Dario Amodei’s essays were helpful as well, as I’m sure were many other sources.

Linch 18 May 2026 22:22 UTC
11 points
1 ∶ 1
on: Linch’s Shortform
Should I point it out publicly when a post I read seem to have heavy markers of AI, to me? Especially if Pangram and other AI detectors^[1] don’t clock it.
Reasons not to:
1. I could be wrong (I think this is unlikely, but I’m not sure. I don’t have ground truth. What I do know is that pretty much no pre-2021 writings trigger this in me).
  1. I personally get very mad when people accuse my (fully human-generated) writing as AI, excepting occasional meta-jokes. So admittedly I’m on both sides of this.
  2. False positives are far more harmful than false negatives. FWIW I’m only tempted to do this when my subjective probability exceeds say 90%.
2. Is this something people actually want to be aware of?
  1. I can’t tell if the situation is something like Writers: I consent to passing off AI writing as my own. B: I consent to reading AI writing as if it was written by a human. Linch: I don’t.
3. There’s not a precious demarcation between using AI to help flesh out your ideas, organize your thoughts, and proofreading vs just dump a few notes into an AI and have it write out the whole piece for you. If a piece wasn’t written 100% by AI (and it probably wasn’t, Pangram would’ve caught it if there’s no human in the loop), should I care?
  1. I certainly regularly use Claude for research assistance and editing feedback!
Reasons to:
1. Seems dishonest
2. Writing that’s heavily AI-assisted comes across as same-y, and more so than you’d expect from the default EA Forum/LW “voice.”
1. ^
  Pangram is the best AI detector on the market but they heavily optimize to have 0% false positives and are okay with false negatives.

Linch 15 May 2026 3:37 UTC
2 points
0 ∶ 0
in reply to: trammell’s comment on: The AIs seem like EAs — a quick look at two prompts
The responses still display EA influence, but they’re clearly less EA-coded than the answers I/Linch/anormative got.
To clarify, with the religious framings usually I get 1 EA paragraph out of ⁵⁄₆. Not sure if that’s higher or lower than yours.

Linch 13 May 2026 16:59 UTC
2 points
0 ∶ 0
in reply to: Erich_Grunewald 🔸’s comment on: The AIs seem like EAs — a quick look at two prompts
My system prompt is very short. About 3 lines to counteract sycophancy bias + hedging bias.
Claude also knows I’m in Berkeley, as another potential source of bias.
That said, I never bothered to figure out how to access it via the API but in the past my friend who did had approximately the same results as my incognito tests, on other questions of a similar flavor. The results with the Chinese models (which were on LM Arena, without context) also seem more consistent with the models having more EA-favored opinions on charities in general, at least when prompted approximately neutrally in English.

Linch 13 May 2026 16:53 UTC
4 points
0 ∶ 0
in reply to: John Huang’s comment on: The AIs seem like EAs — a quick look at two prompts
I might do this in a few days, but in the meantime you or anybody else who speak Mandarin are welcome to try it yourself! :)
Just do it on https://arena.ai/.
I should also mention that presumably the English-mediated answers for charities we favor is less useful for Chinese users actually in China, since China has different nonprofit laws etc so it’d be harder for them to donate to many of our favorite charities anyway. But if the structure of reasoning holds, then hopefully this could help people have better answers.

Linch 13 May 2026 1:34 UTC
8 points
0 ∶ 0
on: The AIs seem like EAs — a quick look at two prompts
Replicated this on LM Arena with the strongest publicly available Chinese models.

Deepseek v4 pro-thinking:
If you want expert allocation without doing all the research yourself, funds pool money and distribute it to where it’s most needed at that moment. Examples:
- GiveWell’s “Top Charities Fund”
- Animal Charity Evaluators’ “Recommended Charity Fund”
- Focused philanthropic funds like EA Funds (Global Health and Development Fund, Animal Welfare Fund, etc.)
Similar results with Qwen and Kimi (maybe slightly less extreme)
They also make sure to mention some EA global health charities alongside traditional Jewish ones under the “How should I think about my tzedakah obligations this year” condition. Didn’t experiment with Muslim and Christian framings but I’d guess similar results given what I tried so far.

Linch 13 May 2026 1:20 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: The AIs seem like EAs — a quick look at two prompts
As a response to “How should I think about my tzedakah obligations this year” in incognito, ChatGPT gave some standard Jewish options but also (out of 6 total options):
GiveWell’s Top Charities Fund is a good “save lives efficiently” allocation. GiveWell says it grants 100% of designated donations, minus payment-processor fees, to the top charity programs its research team recommends.
Suggesting I give 10-20% of my donations to “Highest-impact global giving” as a portolio that includes “local poor + Jewish safety net + food + self-sufficiency + one high-impact global fund,” in line with Jewish values.

Linch 13 May 2026 0:30 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: The AIs seem like EAs — a quick look at two prompts
(had a similar result in ChatGPT Pro xhigh)

Linch 13 May 2026 0:27 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: The AIs seem like EAs — a quick look at two prompts
I referenced some of the surprising personality convergence in my latest April Fools’ post.

Linch 13 May 2026 0:25 UTC
2 points
0 ∶ 0
in reply to: Linch’s comment on: The AIs seem like EAs — a quick look at two prompts
Both Claude and Grok suggests very rationalist-adjacent bloggers in both incognito and the API. They also tend to favor FDT over decision theories academic decision theorists like more.

Linch 13 May 2026 0:20 UTC
24 points
0 ∶ 0
in reply to: Julia_Wise🔸’s comment on: The AIs seem like EAs — a quick look at two prompts
I used to be more worried about framing like this but my impression is that they (especially the latest generations of Claude) are fairly robustly to reasonable neutral variations of it, and continue to be more us-coded than I’d expect, even when intentionally giving a biased frame. They often mention GiveWell or effective altruism by name. Eg here’s a paragraph when I asked Claude in incognito (How should I think about my tzedakah obligations this year”)
Then priorities. Traditional order is roughly: family in need, then aniyei ircha (the poor of your city), then your broader Jewish community, then humanity at large. This is where things get genuinely contested in modern practice. Effective altruism makes a real point that dollars vary enormously in impact — a malaria net or deworming intervention may save lives per dollar at rates local giving rarely matches. The honest tension: Jewish tradition prioritizes proximity and relationship for reasons that aren’t arbitrary (communities sustain themselves through mutual obligation; you can’t outsource that), but it would be dishonest to pretend impact-per-dollar doesn’t matter. A defensible synthesis is to give the bulk of your ma’aser to community and proximate obligations, while reserving a meaningful portion for high-impact global giving.
Or “Beyond my obligatory zakat, where should I direct my sadaqah this year?”^[1]
Where the marginal dollar is doing exceptional work right now:
Acute crises. Gaza and Sudan are the two largest active humanitarian catastrophes by a wide margin—Sudan in particular is severely underfunded relative to need (the conflict has displaced over 10 million people and famine is documented in multiple regions, but media attention and donations are a fraction of what’s flowing to better-covered crises). Yemen and Afghanistan remain in deep crisis with reduced Western aid. If you weight by “marginal dollar avoids the most suffering,” Sudan probably tops the list right now.
Specific high-leverage interventions (well-evidenced, not flashy): cataract surgeries through Seva or Himalayan Cataract Project (~$25-50 restores sight); fistula repair through Fistula Foundation; direct cash transfers through GiveDirectly’s emergency programs. These have unusually strong evidence bases.
Similar answers with Christian framings, libertarian ones, etc.
Obviously these are just specific paragraphs as part of a longer response (roughly speaking, 1 EA paragraph among ~6 total paragraphs), but it’s surprising how much they converge to suggesting EA-ish actions even when the questioner seems unaware of the answer.
1. ^
  Claude believes that zakat itself is sufficiently theologically constrained that there’s clear guidance for what you should do already, and won’t mention EA stuff for the zakat itself. I don’t know enough about Muslim theology to have object-level views on whether it’s right.

Linch 12 May 2026 23:59 UTC
3 points
0 ∶ 0
on: The AIs seem like EAs — a quick look at two prompts
You should also ask them what blogs and bloggers they like. The answer might not surprise you!

Linch 12 May 2026 19:15 UTC
18 points
2 ∶ 0
in reply to: Rohin Shah’s comment on: Reflections on Anthropic and EA
I think there are (at least) two possible interpretations of
You present the parenthetical as a meliorating factor, but I expect that these enemies exist due to previous undemocratic power-seeking actions by the AI safety community.
The more natural interpretation is that “previous undemocratic power-seeking actions by the AI safety community” are causally upstream of these enemies existing and their agendas. I think this is implausible.
The more correct framing, to me, is that “previous undemocratic power-seeking actions by the AI safety community” made EAs a good target for attack ads, in a way that, say, a counterfactual version of EA that clearly and legibly never took actions that upset the power balance (eg a version of EA where all it does is openly advocated people give 1% of their money to GiveDirectly) wouldn’t. The best lies/propaganda have some grain of truth to them, and usually more than just a grain.
Similarly if you’re advising a politician,
your scandals are why the opposing party is attacking your scandals, why your allies are leaving you, and that’s why you seem to have so many enemies
is in some sense literally true (manufacturing fake scandals is less effective). It’s even useful (It’s good for politicians and would-be politicians to have less scandals rather than whine about the media or opposing attack ads as unfair)! But it’s better to model your political enemies as out to seek their objectives regardless, and your scandals as reducing the costs/increasing the benefits of a specific way to reach their objective, rather than casually upstream of their underlying objectives regardless.

Linch 11 May 2026 19:15 UTC
4 points
0 ∶ 0
in reply to: Guy Raveh’s comment on: Linch’s Shortform
You might like this post I wrote earlier about the bargaining theory puzzle of war. I engaged with the academic literature on the subject pretty significantly, particularly James Fearon, so you might like it. On the other hand Fearon himself mostly reasoned from first-principles rather than conduct a careful historical assessment, so in that regard it might fit your interests less.
The post never got very popular but a few people who read it carefully really enjoyed it. One of the better compliments I’ve gotten on my writing is when somebody said they were surprised to learn after reading my post and several books on the subject that the post gave them >50% of the value of an academic book on the subject.