This is partly planning fallacy, partly real life being a lot busier than expected and Forum writing being one of the first things to drop, and partly increasingly feeling gloom and disillusionment with EA and so not having the same motivation to write or contribute to the Forum as I did previously.
For the things that I am still thinking of writing I’ll add comments to this post separately to votes and comments can be attributed to each idea individually.
I do want to write something along the lines of “Alignment is a Political Philosophy Problem”
My takes on AI, and the problem of x-risk, have been in flux over the last 1.5 years, but they do seem to be more and focused on the idea of power and politics, as opposed to finding a mythical ‘correct’ utility function for a hypothesised superintelligence. Making TAI/AGI/ASI go well therefore falls in the reference class of ‘principal agent problem’/‘public choice theory’/‘social contract theory’ rather than ‘timeless decision theory/coherent extrapolated volition’. The latter 2 are poor answers to an incorrect framing of the question.
Nora Belrose’s various posts on scepticism of the case for AI Safety, and even on Safety policy proposals conditional on the first case being right.[1]
I also think this view helps explain the huge range of backlash that AI Safety received over SB1047 and after the awfully botched OpenAI board coup. They were both attempted exercises in political power, and the pushback often came criticising this instead of looking on the ‘object level’ of risk arguments. I increasingly think that this is not an ‘irrational’ response but perfectly thing, and “AI Safety” needs to pursue more co-operative strategies that credibly signal legitimacy.
I don’t think anyone wants or needs another “Why I’m leaving EA” post but I suppose if people really wanted to hear it I could write it up. I’m not sure I have anything new or super insightful to share on the topic.
I have some initial data on the popularity and public/elite perception of EA that I wanted to write into a full post, something along the lines of What is EA’s reputation, 2.5 years after FTX? I might combine my old idea of a Forum data analytics update into this one to save time.
My initial data/investigation into this question ended being a lot more negative than other surveys of EA. The main takeaways are:
Declining use of the Forum, both in total and amongst influential EAs
EA has a very poor reputation in the public intellectual sphere, especially on Twitter
Many previously highly engaged/talented users quietly leaving the movement
An increasing philosophical pushback to the tenets of EA, especially from the New/Alt/Tech Right, instead of the more common ‘the ideas are right, but the movement is wrong in practice’[1]
An increasing rift between Rationalism/LW and EA
Lack of a compelling ‘fightback’ from EA leaders or institutions
Doing this research did contribute to me being a lot more gloomy about the state of EA, but I think I do want to write this one up to make the knowledge more public, and allow people to poke flaws in it if possible.
My previous attempt at predicting what I was going to write got 1⁄4, which ain’t great.
This is partly planning fallacy, partly real life being a lot busier than expected and Forum writing being one of the first things to drop, and partly increasingly feeling gloom and disillusionment with EA and so not having the same motivation to write or contribute to the Forum as I did previously.
For the things that I am still thinking of writing I’ll add comments to this post separately to votes and comments can be attributed to each idea individually.
I do want to write something along the lines of “Alignment is a Political Philosophy Problem”
My takes on AI, and the problem of x-risk, have been in flux over the last 1.5 years, but they do seem to be more and focused on the idea of power and politics, as opposed to finding a mythical ‘correct’ utility function for a hypothesised superintelligence. Making TAI/AGI/ASI go well therefore falls in the reference class of ‘principal agent problem’/‘public choice theory’/‘social contract theory’ rather than ‘timeless decision theory/coherent extrapolated volition’. The latter 2 are poor answers to an incorrect framing of the question.
Writing that influenced my on this journey:
Tan Zhi Xuan’s whole work, especially Beyond Preferences in AI Alignment
Joe Carlsmith’s Otherness and control in the age of AGI sequence
Matthew Barnett’s various posts on AI recently, especially viewing it as an ‘institutional design’ problem
Nora Belrose’s various posts on scepticism of the case for AI Safety, and even on Safety policy proposals conditional on the first case being right.[1]
The recent Gradual Disempowerment post is something along the lines I’m thinking of too
I also think this view helps explain the huge range of backlash that AI Safety received over SB1047 and after the awfully botched OpenAI board coup. They were both attempted exercises in political power, and the pushback often came criticising this instead of looking on the ‘object level’ of risk arguments. I increasingly think that this is not an ‘irrational’ response but perfectly thing, and “AI Safety” needs to pursue more co-operative strategies that credibly signal legitimacy.
I think the downvotes these got are, in retrospect, a poor sign for epistemic health
I don’t think anyone wants or needs another “Why I’m leaving EA” post but I suppose if people really wanted to hear it I could write it up. I’m not sure I have anything new or super insightful to share on the topic.
I have some initial data on the popularity and public/elite perception of EA that I wanted to write into a full post, something along the lines of What is EA’s reputation, 2.5 years after FTX? I might combine my old idea of a Forum data analytics update into this one to save time.
My initial data/investigation into this question ended being a lot more negative than other surveys of EA. The main takeaways are:
Declining use of the Forum, both in total and amongst influential EAs
EA has a very poor reputation in the public intellectual sphere, especially on Twitter
Many previously highly engaged/talented users quietly leaving the movement
An increasing philosophical pushback to the tenets of EA, especially from the New/Alt/Tech Right, instead of the more common ‘the ideas are right, but the movement is wrong in practice’[1]
An increasing rift between Rationalism/LW and EA
Lack of a compelling ‘fightback’ from EA leaders or institutions
Doing this research did contribute to me being a lot more gloomy about the state of EA, but I think I do want to write this one up to make the knowledge more public, and allow people to poke flaws in it if possible.
To me this signals more values-based conflict, which makes it harder to find pareto-improving ways to co-operate with other groups