David Thorstad

Blog up­date: Reflec­tive altruism

Ex­is­ten­tial risk pes­simism and the time of perils

New blog: Some doubts about effec­tive altruism

• David Thorstad 2 May 2023 21:01 UTC

  107 points

  35 ∶ 9

  on: Re­view of The Good It Promises, the Harm It Does

  It is sometimes hard for communities with very different beliefs to communicate. But it would be a shame if communication were to break down.

  I think it is worth trying to understand why people from very different perspectives might disagree with effective altruists on key issues. I have tried on my blog to bring out some key points from the discussions in this volume, and I hope to explore others in the future.

  I hope we can bring the rhetoric down and focus on saying as clearly as possible what the main cruxes are and why a reasonable person might stand on one side or another.

• David Thorstad 25 Feb 2023 18:16 UTC

  67 points

  19 ∶ 0

  on: Who is Un­com­fortable Cri­tiquing Who, Around EA?

  In the spirit of giving credit where credit is due, I want to give one positive data point. I am continually surprised by the degree to which effective altruists are receptive to my criticisms of EA.

  I regularly receive invitations to speak at conferences and other EA events on the basis of my blog, Ineffective Altruism, which does not hold back in raising doubts about EA. I track where my readers come from: many (most?) are effective altruists, and they often read the criticisms cover to cover and engage with them. They also show a strong willingness to read through the more dense and intellectually rigorous criticisms, and are if anything a bit put off by lighter content.

  I can’t say that I have never received pushback, or that I have never been frightened at times. When there are billions of dollars and lifetimes of work at stake, tempers tend to run a bit hot, especially when issues such as inclusion and belonging get in the picture. And I can’t say that I will never face any sanction for my words. I can’t say that I don’t have to change how I speak to make sure that I am heard. I also don’t want to speak for other critics raising different types of criticisms, or speaking from different positions.

  But to be honest, I have found that effective altruists react with more openness, interest and tolerance towards critics than many other communities that I am a part of, including other highly educated and intellectually serious community. I genuinely appreciate and admire that.

  I said the same to a very senior faculty member and public critic of EA yesterday, and I meant it.

• David Thorstad 12 Mar 2023 19:06 UTC

  31 points

  16 ∶ 18

  on: An­nounc­ing the Open Philan­thropy AI Wor­ld­views Contest

  Would you consider incorporating a broader panel of judges? OpenPhil, and effective altruists more generally, tends to have quite strong and niche views about the future of artificial intelligence. This contest would have broader credibility if it were evaluated from a set of perspectives more consistent with, and representative of the field as a whole.

• David Thorstad 12 Aug 2022 17:21 UTC

  29 points

  2 ∶ 0

  in reply to: weeatquince’s comment on: Ex­is­ten­tial risk pes­simism and the time of perils

  Thanks! It’s good to hear from you. These are really good points. Let me take them in turn. Apologies, it might take a bit to talk through them all.

  First point: A small chance of the time of perils: It’s definitely right that if you plug in a moderately small chance (say, one in a million) of a strong version of the Time of Perils Hypothesis into these models, then on many assumptions you will get astronomical value for existential risk reduction.

  I think here I’d want to emphasize four things.

  (A) Incrementalist resistance to longtermism—”Shedding zeroes”: Lots of pushback against longtermism tries to wreck everything in one shot. These arguments find a single consideration (risk aversion; population ethics; non-consequentialist duties; …) and argue that this single consideration, on its own, is enough to decisively refute longtermism.

  The problem with this way of arguing is that often the single consideration has to be very strong—some might say implausibly strong. Suppose I say “you might think the best longtermist intervention is 10^15 times better than the best short-termist intervention. But once you understand the decision-theoretic importance of risk aversion, you should realize we shouldn’t be willing to take risky bets at impacting the long-term future”. Richard Pettigrew makes an excellent argument that goes something like that. (https://​​forum.effectivealtruism.org/​​posts/​​xAoZotkzcY5mvmXFY/​​longtermism-risk-and-extinction). We liked that argument enough at GPI that we made it a working paper.

  But one worry here is that once you realize how much power we’re attributing to risk aversion (enough to outweigh 15 orders of magnitude in expected value!), you might think we’re giving risk aversion a bit too much power in our decision theory. Is the right theory of risk aversion really that powerful?

  I want to take a much more “incrementalist” approach to pushing back against longtermism. That approach strings together a number of considerations that might, or might not, on their own, be enough to knock down longtermism, but which together might well be enough to push us away from longtermism.

  So suppose you say to me “okay, David, I’m going to put a 10^-5 credence on the Time of Perils”. Great. Now we’ve knocked five orders of magnitude off longtermism, in the sense that you now think the best longtermist interventions are 10^10 times better than the best short-termist interventions.

  But now I ask: “Are you sure you’ve got the right normative theory? Other normative theories wouldn’t give quite such a large value to longtermist interventions”. And you say: fine, have another zero.

  And then I ask: “But what if the future you’re preserving is a bad one, not a good one?” And you say: fine, have a zero.

  If we keep going in this way, longtermism might come out to be true. But it might also turn out to be false. If it turns out that the upshot of my post is that the longtermist has to shed, say, 3-10 orders of magnitude because of uncertainty about the Time of Perils Hypothesis, that takes a lot of ballast out from the longtermist position, in the sense that longtermists no longer have so many zeroes left to shed.

  (B) Form matters: It matters which kind of Time of Perils Hypothesis we put a small but nontrivial probability on.

  So for example, suppose you put probability 10^-5 on the claim “within 200 years, existential risk will drop by 4 orders of magnitude, and it will never, NEVER, ever in all of human history rise above that level”. That’s great news for the longtermist.
  

  But what if you put most of your credence in claims like “within 200 years, existential risk will drop by 4 orders of magnitude. It’ll be low for a while, but every once in a while it will flair up to become about as bad as it is today (if not worse)? Then you’ve really substantially reduced the expected value of the future, and likewise the expected value of existential risk reduction.

  The reason that this is important is that if we’re not going to rely on stories about superintelligent AI, it gets harder to motivate Time of Perils views of the first form, rather than the second. It’s not like the destructive technologies threatening humanity today are the last destructive technologies we will ever event, or the irresponsible leaders in charge of those technologies are the last risk-insensitive leaders we’ll ever have.

  (C) Positive arguments: Just because the longtermist would be okay with a (relatively) small probability on the Time of Perils Hypothesis doesn’t get her out of providing an argument for the Time of Perils Hypothesis.

  One way of reading this post is as suggesting that the some of the arguments that are commonly made in support of the Time of Perils Hypothesis aren’t enough to ground the levels of probability we’d need to put on Time of Perils Hypotheses of the needed form.

  Why wouldn’t they be enough? Here’s one reason.

  (D) Priors matter: The Time of Perils Hypothesis makes quite a strong claim. It says that some quantity (existential risk) has been at moderate levels for a while; has recently risen and is currently growing; but will soon drop to an almost unnoticeable level; and will stay at that level literally forever. This isn’t the most common trajectory for quantities found in nature to follow. So if I didn’t tell you anything about why you would expect existential risk to drop, and stay low forever, despite its recent growth, you probably wouldn’t put very much probability on the Time of Perils Hypothesis. And by “not very much” I mean, not even 1 in a million, or maybe not even 1 in a billion. Although those can seem like tiny numbers, there are plenty of claims I can make right now in which you’d invest lower (prior) credence.

  This means that if we’re going to get a high enough credence in time Time of Perils Hypothesis, that credence can’t be coming from our priors. It needs to come from an argument in favor of the Time of Perils Hypothesis. That’s not to say, from the armchair, that such an argument can’t be made. It’s only to say that we really need to focus on making arguments, and not (just) describing the probability targets that our arguments need to get us to.

  Second point: Speed of the “Kuznets” phenomenon. I’m also curious about this! Any takers on taking a stab at the modeling here? I’m especially interested in models that go materially beyond Leopold’s, because I think he did a really good job at giving us a handle on that model.

  If anyone’s comments here make a material change to the contents of the paper, I hereby promise an acknowledgment in the published paper once I manage to convince a journal to publish this. I can also pay in beer if that’s not against forum rules.

  Third point: Tractability of space settlement: That’s a really interesting point to push on. I’d be curious to hear more about your views here! I’m also curious how you think that space settlement affects existential risk.

  If you want to take a dive into the academic literature here, the working paper cites nearly every academic source that I could find on this issue. There’s … surprisingly little, and nobody can agree about anything. Probably there’s room for interesting new work here.

  What links here?

  • Linch's comment on Linch’s Quick takes by Linch (26 May 2023 1:33 UTC; 7 points)

• David Thorstad 13 Jan 2023 18:55 UTC

  25 points

  6 ∶ 0

  on: Eco­nomic The­ory & Global Pri­ori­ti­za­tion (sum­mer 2023): Ap­ply now!

  Phil runs a tremendous program and I would definitely recommend that you apply!

• David Thorstad 16 Mar 2023 8:57 UTC

  24 points

  18 ∶ 6

  in reply to: William_MacAskill’s comment on: William_MacAskill’s Shortform

  Going to be honest and say that I think this is a perfectly sensible response and I would do the same in Will’s position.

• David Thorstad 19 Jan 2023 9:51 UTC

  23 points

  7 ∶ 0

  on: Book cri­tique of Effec­tive Altruism

  I’m planning to write about this book for my blog when it is released. It is an important book which draws on valuable perspectives that are underrepresented in effective altruism.

  It is possible (even healthy) to disagree with some aspects of the book while recognizing that all of the authors have valuable things to say, and have over their lifetimes built valuable stores of knowledge to share. This week, more than ever, I think we should recognize the importance of drawing on diverse sources of knowledge to inform altruistic decisionmaking.

  On the tone of the book, I wonder what readers make of an article by the author of the introduction to the volume, Prof. Amia Srinivasan? (“The aptness of anger”: https://​​users.ox.ac.uk/​​~corp1468/​​Research_files/​​jopp.12130.pdf.)This article argues that anger can be an apt response to injustice. More generally, I hope that discussions of this book can move past questions of tone to examine the substance of arguments and the evidence brought forth to support them.

  If I had to make one criticism of the book, it would be that the book focuses in many places on short-termist EA projects. It is very hard for critics of EA to keep up with the rapid pace at which the EA movement evolves, particularly given that academic research tends to move slowly. I hope that this feature of the book can be contextualized within an academic publishing system in which it can take many years to bring an edited volume to press, and that readers will do their best to consider how the views expressed in this book might apply to longtermist causes as well.

• David Thorstad 24 Sep 2022 20:03 UTC

  23 points

  6 ∶ 1

  in reply to: Nick_Beckstead’s comment on: An­nounc­ing the Fu­ture Fund’s AI Wor­ld­view Prize

  Could you put some judges on the panel who are a bit less worried about AI risk than your typical EA would be? EA opinions tend to cluster quite strongly around an area of conceptual space that many non-EAs do not occupy, and it is often hard for people to evaluate views that differ radically from their own. Perhaps one of the superforecasters could be put directly onto the judging panel, pre-screening for someone who is less worried about AI risk.

• David Thorstad 22 Feb 2023 15:06 UTC

  22 points

  4 ∶ 0

  in reply to: Jason’s comment on: A state­ment and an apology

  I assume EAs think I’m speaking a bit sharply here? These discussions are hard because EAs often place a very high value on polite/​unemotional discourse, whereas non-EAs often place a higher value on discourse that won’t cause harms to groups/​people and think that sharper or more emotional discourse can sometimes be an appropriate response (or even the only appropriate response) in such cases.

• David Thorstad 3 May 2023 17:57 UTC

  18 points

  17 ∶ 0

  in reply to: sjsjsj’s comment on: Re­view of The Good It Promises, the Harm It Does

  How about a blog update in a month or so about the post series I’ve written so far, lessons learned, and future directions, posted to the EA Forum?

• David Thorstad 5 Jul 2023 16:59 UTC

  16 points

  0 ∶ 0

  in reply to: Toby_Ord’s comment on: Three mis­takes in the moral math­e­mat­ics of ex­is­ten­tial risk (David Thorstad)

  Thanks Toby! Comments much appreciated.

• David Thorstad 19 Feb 2023 1:22 UTC

  15 points

  14 ∶ 11

  in reply to: EliezerYudkowsky’s comment on: Peo­ple Will Some­times Just Lie About You

  To be clear, at the risk of repetition, the question is not whether journalists should be considered reliable in their explanations of academic research. Although some journalists explain academic research quite well, others lack the training to do this as well as professional researchers. I would much rather turn to a colleague’s blog than to a TIME magazine article to understand issues in academic philosophy, and I suspect that academics in most other fields would say the same.

  The question is whether a major international publication should be considered a reliable source of investigative journalism into issues of sexual harassment and abuse. Most educated readers would consider such a publication to be among the most reliable sources on such a topic. Most readers would express some skepticism about the ability of a typical blogger to conduct serious investigative journalism of the same calibre. These readers would cite the relatively stronger training that journalists have in the practice of investigative journalism, as well as the relatively stronger track record of successful investigative journalism at major international news outlets, when compared to blogs. They might also cite specific institutional features of news outlets, such as more time devoted to research, more people devoted to fact-checking, and a higher reputational and legal stake in getting things right.

  Is there a reason why we should distrust TIME’s discussion of sexual harassment and abuse within effective altruism?

• David Thorstad 19 Feb 2023 0:53 UTC

  15 points

  18 ∶ 21

  in reply to: EliezerYudkowsky’s comment on: Peo­ple Will Some­times Just Lie About You

  If you want to insinuate that a major international publication is likely to be less reliable than a blog on issues of sexual harassment and abuse within the movement, it would be appropriate for you to write up the much longer description that you mention. This is a striking claim that is at odds with established views of what constitutes a trustworthy source. Most educated readers think that a publication such as TIME is among the most reliable sources that can be found on such a subject.

  I don’t deny that journalists sometimes have trouble understanding academic research. Blogs written by professional academics are often in a much better position to understand academic research, because their authors have more relevant expertise. I would be highly skeptical of the claim that, in all but the most exceptional circumstances, blogs are better equipped than major media companies to conduct serious investigative journalism on sensitive issues. Publications such as TIME have been doing that kind of work successfully for many years.

  If you do write this up, please be careful to respect the anonymity of those whose names were redacted in the TIME article.

• David Thorstad 14 Mar 2023 17:48 UTC

  14 points

  4 ∶ 1

  in reply to: David Mathers’s comment on: An­nounc­ing the Open Philan­thropy AI Wor­ld­views Contest

  Ordinarily, a philanthropic foundation offering a prize meant to advance scientific understanding of some topic X would put at mot one or two of its own members on the prize panel. The rest of the panel would be composed of leading scientists, academics, industry professionals, and perhaps a few policymakers. They might also consider inviting leaders of relevant foundations. Most members of the panel would be chosen for specific expertise in topic X combined with broad respect and experience within their fields, although a few panelists might be chosen to represent generalist constituencies (for example, a university president). Members would typically be at mid- or late-career stages, and have substantial research records of their own as well as the esteem of their peers. They might, as appropriate, draw on a broader pool of peer reviewers or nominators in early rounds of the selection process.

  Prizes would typically be broadly advertised, and left open for a sufficient period to allow original research (at least six months). They would encourage submissions of a standard length for original research contributions, rather than discouraging submissions greater than 5,000 words. If the focus was solely on the individual piece of submitted work, the review process would be double- or triple-blinded and announced as such.

  I could go on, but I take it that all of the above are fairly standard.

• David Thorstad 23 Apr 2024 15:20 UTC

  13 points

  2 ∶ 3

  on: Mo­ti­va­tion gaps: Why so much EA crit­i­cism is hos­tile and lazy

  Strongly agree. I think there’s also a motivation gap in knowledge acquisition. If you don’t think there’s much promise in an idea or a movement, it usually doesn’t make sense to spend years learning about it. This leads to large numbers of very good academics writing poorly-informed criticisms. But this shouldn’t be taken to indicate that there’s nothing behind the criticisms. It’s just that it doesn’t pay off career-wise for these people to spend years learning enough to press the criticisms better.

• David Thorstad 7 Jul 2023 7:30 UTC

  13 points

  2 ∶ 0

  on: CEA is wind­ing down the Com­mu­nity Events grant­mak­ing programme

  Thanks Ollie for your work on this program! You did a great job with it.

• David Thorstad 4 Feb 2023 2:37 UTC

  13 points

  26 ∶ 20

  in reply to: fenneko’s comment on: EA, Sex­ual Harass­ment, and Abuse

  I think the article was fairly clear: “TIME is not naming the man, like others in this story, due to the request of one or more women who made accusations against them, and who wanted to shield themselves from possible retaliation”.

  Please respect the wishes of women who face serious threats of professional and personal harm and have chosen to take steps to protect their identities.

  What links here?

  • fenneko's comment on Abuse in LessWrong and ra­tio­nal­ist com­mu­ni­ties in Bloomberg News by whistleblower67 (8 Mar 2023 3:56 UTC; 180 points)
  • fenneko's comment on EA, Sex­ual Harass­ment, and Abuse by whistleblower9 (4 Feb 2023 3:14 UTC; 4 points)