I primarily write academic papers and do outreach through my blog. I do try to post here when possible (and I always appreciate cross-posts!), but please do check dthorstad.com for my academic papers and reflectivealtruism.com for outreach.
David Thorstad
Karma: 1,949
Getting it right: Criticism
Harry Lloyd in the philosophy department can do this.
Sam Altman is far from the only problematic speaker invited to Lighthaven.
- David Thorstad 11 Oct 2025 15:02 UTC4 points0 ∶ 0in reply to: SerpentStare’s comment on: Announcing: The ‘Essays on Longtermism’ Competition
Just seeing this now—sorry for the late reply!
It’s open access. Here is the copyright information from the book. Is this license good enough for the kind of audio adaptation you’re thinking of? (There’s a permissions department at Oxford that you can contact if you’re not sure—they direct to https://plsclear.com/Home/Index I think)
This is an open access publication, available online and distributed under the terms of a Creative Commons Attribution-Non Commercial-No Derivatives 4.0 International licence (CC BY-NC-ND 4.0), a copy of which is available at https://creativecommons.org/licenses/by-nc-nd/4.0/. Subject to this license, all rights are reserved.
- David Thorstad 7 Oct 2025 13:33 UTC2 points0 ∶ 0in reply to: Toby Tremlett🔹’s comment on: tobytrem’s Quick takes
Glad to be working with Kuhan!
- David Thorstad 5 Oct 2025 7:32 UTC12 points4 ∶ 8in reply to: Ajeya’s comment on: Ajeya’s Quick takes
There’s not very much evidence for existential risk from biological causes. I’ve had a very hard time getting anyone to even tell me what they are concerned about, and when they do it is not very plausible.
- David Thorstad 11 Sep 2025 20:52 UTC4 points0 ∶ 0in reply to: OscarD🔸’s comment on: Essays on longtermism
Good question! I’m assuming just the .pdf, which is what Oxford University Press does for most academic books. But I will check and let you know if they’re doing an epub.
Since the book is open access, I’m not sure if they secured the pdfs. If they didn’t you could try a converter?
Essays on longtermism
Ilan and Shakked Noy have a nice piece on one aspect of this, “The Short-Termism of ‘Hard’ Economics.” It’s forthcoming in Essays on longtermism, hopefully within a month or two. They argue that preferences for methodological hardness in economics make it difficult to publish longtermist research within economics.
To be fair, economics as a field is widely regarded as one of the strongest and most successful in academia right now. There are definitely challenges, of which Bob and the Noy’s identify some, but I don’t want to give the impression that academics are too down on economics as a field. They’ve been killing it since the mid-20th century.
I also don’t think economists are crazy to think that their top journals are sometimes more rigorous than leading interdisciplinary journals, which can be a bit more headline-chasing and often don’t give authors enough words to be fully rigorous.
Honestly I agree. Titotal’s work is fabulous (and I’m happy to use that adjective) but it’s usually not a good idea to put someone’s hackles up when you want to tell them they are wrong.
- David Thorstad 30 Mar 2025 16:14 UTC4 points1 ∶ 0
Error
The value NIL is not of type SIMPLE-STRING when binding #:USER-ID162
- David Thorstad 17 Mar 2025 6:46 UTC20 points8 ∶ 0in reply to: Dustin Moskovitz’s comment on: Ozzie Gooen’s Shortform
Please come back. I can’t say that we agree on very much, but you are often a voice of reason and your voice will be missed.
- David Thorstad 18 Dec 2024 16:40 UTC9 points0 ∶ 0in reply to: Jonas Hallgren 🔸’s comment on: Blog update: Reflective altruism
Thanks Jonas! I really appreciate your constructive engagement.
I’m not very sympathetic to pure time preference – in fact, most philosophers hate pure time preference and are leading the academic charge against it. I’m also not very sympathetic to person-affecting views, though many philosophers whom I respect are more sympathetic. I don’t rely on either of these views in my arguments, because I think they are false.
In general, my impression on the topic of advancing progress is that longtermists have not yet identified plausible interventions which could do this well enough to be worth funding. There has certainly been a lot written about the value of advancing progress, but not many projects have been carried out to actually do this. That tends to suggest to me that I should hold off on commenting about the value of advancing progress until we have more concrete proposals on the table. It also tends to suggest that advancing progress may be hard.I think that covers most of the content of Toby’s posts (advancing progress and temporal discounting). Perhaps one more thing that would be important to stress is that I would really like to see Oxford philosophers following the GPI model of putting out smaller numbers of rigorous academic papers rather than larger numbers of forum posts and online reports. It’s not very common for philosophers to engage with one another on internet forums – such writing is usually regarded as a form of public outreach. My first reaction to EA Forum posts by Toby and others would be that I’d like to see them written up as research papers. When ideas are written up online before they are published in academic venues, we tend to be skeptical that the research has actually been done or that it would hold up to scrutiny if it were.
On power-seeking, one of the most important features of academic papers is that they aim to tackle a single topic in depth, rather than a broad range of topics in less depth. So for example, my power-seeking paper does not engage with evolutionary arguments by Hendrycks and others because they are different arguments and would need to be addressed in a different paper.
To be honest, my selection of arguments to engage with is largely driven by the ability of those making the arguments to place them in top journals and conferences. Academics tend to be skeptical of arguments that have not cleared this bar, and there is little career value in addressing them. The reason why I addressed the singularity hypothesis and power-seeking is that both have had high-profile backing by quality authors (Bostrom, Chalmers) or in high-profile conferences (NeurIPS). I’d be happy to engage with evolutionary arguments when they clear this bar, but they’re a good ways off yet and I’m a bit skeptical that they will clear it.
I don’t want to legislate a particular definition of power, because I don’t think there’s a good way to determine what that should be. My concern is with the argument from power-seeking. This argument needs to use a notion of power such that AI power-seeking would lead to existentially catastrophic human disempowerment. We can define power in ways that make power-seeking easier to establish, but this mostly passes the buck, since we will then need to argue why power-seeking in this sense would lead to existentially catastrophic human disempowerment.
One example of this kind of buck-passing is in the Turner et al. papers. For Turner and colleagues, power is (roughly) the ability to achieve valuable outcomes or to promote an agent’s goals. It’s not so hard to show that agents will tend to be power-seeking in this sense, but this also doesn’t imply that the results will be existentially catastrophic without further argument, and that argument will probably take us substantially beyond anything like the traditional argument from power-seeking.
Likewise, we might, as you suggest, think about power-seeking as free-energy minimization and adopt a view on which most or all systems seek to minimize free energy. But precisely because this view takes free energy minimization to be common, it won’t put us very far on the way towards arguing that the results will be existentially catastrophic, nor even that they will involve human disempowerment (in the sense of failure to minimize free energy).
I agree with you that it is unfair to conclude much from the failure of MIRI arguments. The discussion of the MIRI paper is cut from the version of the power-seeking paper that I submitted to a journal. Academic readers are likely to have had high credence that the MIRI argument was bad before reading it, so they will not update much on the failure of this argument. I included this argument in the extended version of the paper because I know that some effective altruists were moved by it, and also because I didn’t know of any other formal work that broke substantially from the Turner et al. framework. I think that the MIRI argument is not anywhere near as good as the Turner et al. argument. While I have many disagreements with Turner et al., I want to stress that their results are mathematically sophisticated and academically rigorous. The MIRI paper is netiher.
I hope that helps!
- David Thorstad 17 Dec 2024 14:41 UTC34 points13 ∶ 5in reply to: justsaying’s comment on: Blog update: Reflective altruism
Thanks!
My discussion of Alexander is based on a number of sources. One of the primary sources is the email that you mention, though I do not think it is accurate to say that this source forms almost the entire basis of my case.
I do not publish documents whose authenticity I have significant reason to doubt, and in this case I excluded several documents because I could not confirm their authenticity to my satisfaction. In the case of the email that you discuss, my reasons for posting the email are as follows.
First, to my knowledge Alexander has never denied authorship of the email. This would, presumably, be a natural first step for someone who did not write it. Second, I asked publicly for credible denials by anyone, not just Alexander, of the authenticity of the email and found none. Third, I wrote personally to Alexander and offered him the opportunity to comment on the authenticity of the email. Alexander declined.
I am surprised by your comments about disclaimers. My intention was to include enough disclaimers to make a tax lawyer blush. For example:
First, The quote which leads the article does include a disclaimer: it is cited as stemming from an “(Alleged) email to Topher Brennan”.
Second, The preliminary notes to the article directly express my attitudes towards authenticity, and offer to publicly apologize if I am wrong in attributing views to Alexander or anyone else. (So far, no one has offered any evidence that I was wrong, or even attempted to offer any sort of denial, evidenced or not.). I wrote: “A note about authenticity. I have done my best to verify that the cited views in this post are the work of the authors they are attributed to. This is difficult because Alexander has alleged that some of the most troubling comments attributed to him are not his. My policy has been to stick with the sources whose authenticity I am most confident in, and to the best of my knowledge the authenticity of all passages quoted in this post has not been publicly disputed by their alleged authors. (Alexander declined a request for comment on the authenticity of passages quoted in this post). However, I am conscious of the possibility of error. While I have closed comments on all posts in this series, I am highly open to correction on any matters of fact, and if I am wrong in attributing any views to an individual I will correct them and publicly apologize. You can reach me at reflectivealtruismeditor@gmail.com.”
Third, immediately before presenting Alexander’s email I write: “I am not able to confirm the veracity of the email, so readers are asked to use their own judgment, but I am also not aware of credible attempts by anyone, including Alexander, to deny the veracity of this email (Alexander declined to comment on the veracity of the email. I did check for other credible attempts to deny its veracity.).”
These are not the only disclaimers included in the article, but I think that they are more than adequate and included in the proper places.
You seem skeptical of some of the implications that I draw from this email. I quote the email in full, then draw six implications, giving evidence for each. If you are concerned about some of these implications, you are of course welcome to say which implications you are concerned about and give reasons to think that they do not follow.
I do acknowledge that Alexander has pushed back against some of the worst excesses of race science, including neoreaction. For example, I wrote: “I would be remiss if I did not remind readers that not all of Alexander’s contributions to this space are bad. Alexander has penned one of the most successful and best-known criticisms of the excesses of the reactionary movement, in the form of his 2013 “Anti-reactionary FAQ“, which has inspired useful follow-ups from others. Alexander does write some posts supportive of the cause of social justice, such as his “Social justice for the highly demanding of rigor,” though the implication that rigor is not already present in discussions of social justice may not be appreciated by all readers. Alexander has drawn attention to the real and important phenomenon of sexual harassment perpetrated by women, or against men. And we will see below that Alexander calls out Richard Hanania for his strange obsession with office romance.”
All of these behaviors are consistent with the conclusions that I draw in the article. I do not, and will not say that Alexander has never done anything to push back against racism or sexism. That would be a bald-faced lie. It is possible to push back against some bad behaviors and beliefs on these fronts while engaging in other troubling behaviors and holding other troubling beliefs.
- David Thorstad 16 Dec 2024 13:31 UTC4 points0 ∶ 0in reply to: Toby Tremlett🔹’s comment on: Blog update: Reflective altruism
Thanks Toby!
Maybe biorisk or the time of perils could be good topics for a debate week?
Blog update: Reflective altruism
I’m quite disappointed by this article. I talked to her and tried to steer her towards more substantive and novel concerns. I know that other knowledgeable people talked to her as well. That didn’t seem to make much of an impact.
- David Thorstad 24 May 2024 15:40 UTC8 points2 ∶ 11in reply to: Jaime Sevilla’s comment on: Summary: Against the singularity hypothesis
This is a paper about the technological singularity, not the economic singularity.
The economic singularity is an active area of discussion among leading economists. It is generally regarded as a fringe view, but one that many are willing to take seriously enough to discuss. Professional economists are capable of carrying this discussion out on their own, and I am happy to leave them to it. If your institute would like to contribute to this discussion, I would advise you to publish your work in a leading economics journal and to present your work at reputable economics departments and conferences. This would probably involve employing researchers with PhDs in economics and appointments in economics departments to conduct the relevant research. If you do not want to move the scholarly literature, you should defer to the literature, which is at present quite skeptical of most theses under the heading of the economic singularity.
It is important not to equivocate between accelerating growth (a claim about the rate of change of growth rates over time) and accelerated growth (a claim about growth rates having jumped to a higher level). The claim about “pretty intense accelerated growth (up to x10 greater than today’s frontier economies) is a claim of the second sort, whereas the singularity hypothesis is a claim of the first sort.
It is also important not to treat exponential growth (which grows at a constant relative rate) as accelerating growth. The claim about 10% annual growth rates is a claim about exponential growth, which is not accelerating growth.
It is deeply misleading to suggest that accelerating economic growth “has been the norm for most of human history”. The fastest growth we have ever seen over a sustained period is exponential growth, and that is constant, not accelerating. Most historical growth rates were far slower than economic growth today. I think you might mean that we have transitioned over time from slower to faster growth modes. That is not to say that any of these growth modes have been types of accelerating growth.
- David Thorstad 22 May 2024 15:37 UTC16 points1 ∶ 2in reply to: Jaime Sevilla’s comment on: Summary: Against the singularity hypothesis
The paper does not claim that diminishing returns are a decisive obstacle to the singularity hypothesis (of course they aren’t: just strengthen your growth assumptions proportionally). It lists diminishing returns as one of five reasons to be skeptical of the singularity hypothesis, then asks for a strong argument in favor of the singularity hypothesis to overcome them. It reviews leading arguments for the singularity hypothesis and argues they aren’t strong enough to do the trick.
Thanks Richard!
Writing is done for an audience. Effective altruists have a very particular practice of stating their personal credences in the hypotheses that they discuss. While this is not my practice, in writing for effective altruists I try to be as precise as I can about the relative plausibility that I assign to various hypotheses and the effect that this might have on their expected value.
When writing for academic audiences, I do not discuss uncertainty unless I have something to add which my audience will find to be novel and adequately supported.
I don’t remind academic readers that uncertainty matters, because all of them know that on many moral theories uncertainty matters and many (but not all) accept such theories. I don’t remind academic readers of how uncertainty matters on some popular approaches, such as expected value theory, because all of my readers know this and many (but fewer) accept such theories. The most likely result of invoking expected value theory would be to provoke protests that I am situating my argument within a framework which some of my readers do not accept, and that would be a distraction.
I don’t state my personal probability assignments to claims such as the time of perils hypothesis because I don’t take myself to have given adequate grounds for a probability assignment. Readers would rightly object that my subjective probability assignments had not been adequately supported by the arguments in the paper, and I would be forced to remove them by referees, if the paper were not rejected out of hand.
For the same reason, I don’t use language forcing my personal probability assignments on readers. There are always more arguments to consider, and readers differ quite dramatically in their priors. For that reason, concluding a paper with the conclusion that a claim like the time of perils hypothesis has a probability on the order of 10^(-100) or 10^(-200) would again, rightly provoke the objection that this claim has not been adequately supported.
When I write, for example, that arguments for the time of perils hypothesis are inconclusive, my intention is to allow readers to make up their own minds as to precisely how poorly those arguments fare and what the resulting probability assignments should be. Academic readers very much dislike being told what to think, and they don’t care a whit for what I think.
As a data point, almost all of my readers are substantially less confident in many of the claims that I criticize than I am. The most common reason why my papers criticizing effective altruism are rejected from leading journals is that referees or editors take the positions criticized to be so poor that they do not warrant comment. (For example, my paper on power-seeking theorems was rejected from BJPS by an editor who wrote, “The arguments critically evaluated in the paper are just all over the place, verging from silly napkin-math, to speculative metaphysics, to formal explorations of reinforcement learning agents. A small minority of that could be considered philosophy of computer science, but the rest of it, in my view, is computer scientists verging into bad philosophy of mind and futurism … The targets of this criticism definitely want to pretend they’re doing science; I worry that publishing a critical takedown of these arguments could lend legitimacy to that appearance.”)
Against this background, there is not much pressure to remind readers that the positions in question could be highly improbable. Most think this already, and the only thing I am likely to do is to provoke quick rejections like the above, or to annoy the inevitable referee (an outlier among my readers) selected for their sympathies with the position being criticized.
To tell the truth, I often try to be even more noncommittal in the language of my papers than the published version would suggest. For example, the submitted draft of “Mistakes in the moral mathematics of existential risk” said in the introduction that “under many assumptions, once these mistakes are corrected, the value of existential risk mitigation will be far from astronomical.” A referee complained that this was not strong enough, because (on their view) the only assumptions worth considering were those on which the value of existential risk mitigation is rendered extremely minimal. So I changed the wording to “Under many assumptions, once these mistakes are corrected, short-termist interventions will be more valuable than long-termist interventions, even within models proposed by leading effective altruists.” Why did I discuss these assumptions, instead of a broader class of assumptions under which the value of existential risk mitigation is merely non-astronomical? Because that’s what my audience wanted to talk about.
In general, I would encourage you to focus in your writing on the substantive descriptive and normative issues that divide you from your opponents. Anyone worth talking to understands how uncertainty works. The most interesting divisions are not elementary mistakes about multiplication, but substantive questions about probabilities, utilities, decision theories, and the like. You will make more significant contributions to the state of the discussion if you focus on identifying the most important claims that in fact divide you from your opponents and on giving extended arguments for those claims.
To invent and claim to resolve disagreements based on elementary fallacies is likely to have the effect of pushing away the few philosophers still genuinely willing to have substantive normative and descriptive conversations with effective altruists. We are not enthusiastic about trivialities.