I agree. EA has a cost-effectiveness problem that conflicts with its truth-seeking attempts. EAâs main driving force is cost-effectiveness, above all elseâeven above truth itself.
EA is highly incentivised to create and spread apocalyptic doom narratives. This is because apocalyptic doom narratives are good at recruiting people to EAâs âletâs work to decrease the probability of apocalyptic doom (because that has lots of expected value given future population projections)â cause area. And funding-wise, EA community funding (at least in the UK) is pretty much entirely about trying to make more people work in these areas.
EA is also populated by the kinds of people who respond to apocalyptic doom narratives, for the basic reason that if they didnât they wouldnât have ended up in EA. So stuff that promotes these narratives does well in EAâs attention economy.
EA just doesnât have anywhere near as much ÂŁ$⏠to spend as academia does. Itâs also very interested in doing stuff and willing to tolerate errors as long as the stuff gets done. Therefore, its academic standards are far lower.
I really donât know how youâd fix this. I donât think research into catastrophic risks should be conducted on a shoestring budget and by a pseudoreligion/âcitizen science community. I think it should be government funded and probably sit within the wider defense and security portfolio.
However Iâll give EA some grace for essentially being a citizen science community, for the same reason I donât waste effort grumping about the statistical errors made by participants in the Big Garden Birdwatch.
This is a beautifully written comment, and succinct, and funny, and true.
I would give EA much more grace if its self-image was the same as what I presume the Big Garden Birdwatchâs self-image is. Part of what gets me tilted out of my mind about the EA community is when people express this almost messianic Chosen Ones self-image â which ties into the pseudo-religious aspect you mentioned.
The high-impact, low-probability logic of existential risk is hypnotically alluring. If a 1 in 1 quintillion chance of reducing existential risk is equivalent to 100 human lives, what does that imply in terms of your moral responsibility when discussing existential risk? If you have things to say that could cast doubt on existential risk arguments, should you self-censor and hold your tongue? If you speak out and youâre wrong, it could be the moral equivalent of killing 100 people. Would it be okay to lie? To exaggerate? Why not? Wouldnât you lie or exaggerate to save 100 lives? If the Nazis knocked at your door, wouldnât you lie to save Anne Frank in the attic?
I donât think many people are actually outright lying when it comes to existential risk. But I do think people are self-censoring when it comes to criticism, and I do think people are willing to make excuses for really low-quality products like AI 2027 or 80,000 Hoursâ video on it because anything that builds momentum for existential risk fear is plausibly extremely high in expected value.
I agree. EA has a cost-effectiveness problem that conflicts with its truth-seeking attempts. EAâs main driving force is cost-effectiveness, above all elseâeven above truth itself.
EA is highly incentivised to create and spread apocalyptic doom narratives. This is because apocalyptic doom narratives are good at recruiting people to EAâs âletâs work to decrease the probability of apocalyptic doom (because that has lots of expected value given future population projections)â cause area. And funding-wise, EA community funding (at least in the UK) is pretty much entirely about trying to make more people work in these areas.
EA is also populated by the kinds of people who respond to apocalyptic doom narratives, for the basic reason that if they didnât they wouldnât have ended up in EA. So stuff that promotes these narratives does well in EAâs attention economy.
EA just doesnât have anywhere near as much ÂŁ$⏠to spend as academia does. Itâs also very interested in doing stuff and willing to tolerate errors as long as the stuff gets done. Therefore, its academic standards are far lower.
I really donât know how youâd fix this. I donât think research into catastrophic risks should be conducted on a shoestring budget and by a pseudoreligion/âcitizen science community. I think it should be government funded and probably sit within the wider defense and security portfolio.
However Iâll give EA some grace for essentially being a citizen science community, for the same reason I donât waste effort grumping about the statistical errors made by participants in the Big Garden Birdwatch.
This is a beautifully written comment, and succinct, and funny, and true.
I would give EA much more grace if its self-image was the same as what I presume the Big Garden Birdwatchâs self-image is. Part of what gets me tilted out of my mind about the EA community is when people express this almost messianic Chosen Ones self-image â which ties into the pseudo-religious aspect you mentioned.
The high-impact, low-probability logic of existential risk is hypnotically alluring. If a 1 in 1 quintillion chance of reducing existential risk is equivalent to 100 human lives, what does that imply in terms of your moral responsibility when discussing existential risk? If you have things to say that could cast doubt on existential risk arguments, should you self-censor and hold your tongue? If you speak out and youâre wrong, it could be the moral equivalent of killing 100 people. Would it be okay to lie? To exaggerate? Why not? Wouldnât you lie or exaggerate to save 100 lives? If the Nazis knocked at your door, wouldnât you lie to save Anne Frank in the attic?
I donât think many people are actually outright lying when it comes to existential risk. But I do think people are self-censoring when it comes to criticism, and I do think people are willing to make excuses for really low-quality products like AI 2027 or 80,000 Hoursâ video on it because anything that builds momentum for existential risk fear is plausibly extremely high in expected value.