I primarily write academic papers and do outreach through my blog. I do try to post here when possible (and I always appreciate cross-posts!), but please do check dthorstad.com for my academic papers and reflectivealtruism.com for outreach.
David Thorstad
Karma: 1,438
The paper does not claim that diminishing returns are a decisive obstacle to the singularity hypothesis (of course they aren’t: just strengthen your growth assumptions proportionally). It lists diminishing returns as one of five reasons to be skeptical of the singularity hypothesis, then asks for a strong argument in favor of the singularity hypothesis to overcome them. It reviews leading arguments for the singularity hypothesis and argues they aren’t strong enough to do the trick.
Summary: Against the singularity hypothesis
<3
As far as not interacting with someone, there are lots of people I ignore on the basis of bad behavior. As far as not taking the types of concerns they raise seriously, I’d like to think that it’s possible to separate the person from the concerns.
For example, in my discipline (philosophy) there is a truly excellent ranking of philosophy departments. It’s also run by a terrible human being. Most of us still use the ranking, just with the warning not to engage with this individual, which we repeat to our students.
Pleasantly surprised by this (ditto for David Mathers’ comment). Maybe I will try this? I care a great deal about this issue, but not (yet) enough to burn my ability to speak with EAs.
I’m starting to think that I need to write these up. I thought you folks knew. I don’t think the EA community will react very well if I do write these up, so I have tended to hold off.
It is possible that many complaints about Torres are true and also that Torres raises important concerns. I would not like to see personal criticism of Torres become a substitute for engagement with criticism by Torres.
I sometimes worry that concern about Torres has made the community less receptive to any criticism of racism, whether or not raised by Torres. That strikes me as an outcome that should definitely be avoided.
I’d like to hope that academics are aiming for a level of understanding above that of a typical user on an Internet forum.
All academic works have a right to reply. Many journals print response papers and it is a live option to submit responses to critical papers, including mine. It is also common to respond to others in the context of a larger paper. The only limit to the right of academic reply is that the response must be of suitable quality and interest to satisfy expert reviewers.
That’s fair! But I also think most op-eds on any topic are pretty bad. As for academic papers, I have to say it took me at least a year to write anything good about EA, and that was on a research-only postdoc with 50% of my research time devoted to longtermism.
There’s an awful lot that has been written on these topics, and catching up on the state of the art can’t be rushed without bad results.
Strongly agree. I think there’s also a motivation gap in knowledge acquisition. If you don’t think there’s much promise in an idea or a movement, it usually doesn’t make sense to spend years learning about it. This leads to large numbers of very good academics writing poorly-informed criticisms. But this shouldn’t be taken to indicate that there’s nothing behind the criticisms. It’s just that it doesn’t pay off career-wise for these people to spend years learning enough to press the criticisms better.
Glad it helped! All credit to Nicholas who wrote 99% of it. If you have a minute, I uploaded a talk version of the paper last week. Would love to hear what you think, especially re accessibility:
Summary: Mistakes in the Moral Mathematics of Existential Risk (David Thorstad)
You folks impress me! But seriously, that’s a big ask.
To be fair, this could trigger lawsuits. I hope someone is reflecting on FTX, but I wouldn’t expect anyone to be keen on discussing their own involvement with FTX publicly and in great detail.
Here’s a gentle introduction to the kinds of worries people have (https://spectrum.ieee.org/power-problems-might-drive-chip-specialization). Of the cited references “the chips are down for moore’s law” is probably best on this issue, but a little longer/harder. There’s plenty of literature on problems with heat dissipation if you search the academic literature. I can dig up references on energy if you want, but with Sam Altman saying we need a fundamental energy revolution even to get to AGI, is there really much controversy over the idea that we’ll need a lot of energy to get to superintelligence?
Ah—that comes from the discontinuity claim. If you have accelerating growth that isn’t sustained for very long, you get something like population growth from 1800-2000, where the end result is impressive but hardly a discontinuity comparable to crossing the event horizon of a black hole.
(The only way to go around the assumption of sustained growth would be to post one or a few discontinuous leaps towards superintelligence. But that’s harder to defend, and it abandons what was classically taken to ground the singularity hypothesis, namely the appeal to recursive self-improvement).
Here’s the talk version for anyone who finds it easier to listen to videos:
Summary: Against the Singularity Hypothesis (David Thorstad)
Congrats Elliott! Looks like a nice paper.
This is a paper about the technological singularity, not the economic singularity.
The economic singularity is an active area of discussion among leading economists. It is generally regarded as a fringe view, but one that many are willing to take seriously enough to discuss. Professional economists are capable of carrying this discussion out on their own, and I am happy to leave them to it. If your institute would like to contribute to this discussion, I would advise you to publish your work in a leading economics journal and to present your work at reputable economics departments and conferences. This would probably involve employing researchers with PhDs in economics and appointments in economics departments to conduct the relevant research. If you do not want to move the scholarly literature, you should defer to the literature, which is at present quite skeptical of most theses under the heading of the economic singularity.
It is important not to equivocate between accelerating growth (a claim about the rate of change of growth rates over time) and accelerated growth (a claim about growth rates having jumped to a higher level). The claim about “pretty intense accelerated growth (up to x10 greater than today’s frontier economies) is a claim of the second sort, whereas the singularity hypothesis is a claim of the first sort.
It is also important not to treat exponential growth (which grows at a constant relative rate) as accelerating growth. The claim about 10% annual growth rates is a claim about exponential growth, which is not accelerating growth.
It is deeply misleading to suggest that accelerating economic growth “has been the norm for most of human history”. The fastest growth we have ever seen over a sustained period is exponential growth, and that is constant, not accelerating. Most historical growth rates were far slower than economic growth today. I think you might mean that we have transitioned over time from slower to faster growth modes. That is not to say that any of these growth modes have been types of accelerating growth.