Thank you for giving this some thought. The impression I have from this is that the concerns you raise relate to either spurious levels of accuracy or edge cases.
But maybe my impression is wrong. If you could provide some specific examples where either 80k or another org has made a wrong decision because of these considerations, I would find this piece much more interesting.
I’ve also edited the Bonus section. As a rule, we aren’t accounting for correlations between the factors, and there’s no general upper bound on how much this could bias our estimates.
I wrote above that 80,000 Hours should perhaps have given “Health in poor countries” a negative rating for Neglectedness. No other examples come to mind.
I’m more worried about cause area analyses written by individuals (mostly) on their own than the work of EA orgs, which typically goes through more review and is done more formally.
Thank you for giving this some thought. The impression I have from this is that the concerns you raise relate to either spurious levels of accuracy or edge cases.
But maybe my impression is wrong. If you could provide some specific examples where either 80k or another org has made a wrong decision because of these considerations, I would find this piece much more interesting.
I’ve also edited the Bonus section. As a rule, we aren’t accounting for correlations between the factors, and there’s no general upper bound on how much this could bias our estimates.
I’ve also thought of a few possible examples for the Bonus section here, and added them to the post.
I wrote above that 80,000 Hours should perhaps have given “Health in poor countries” a negative rating for Neglectedness. No other examples come to mind.
I’m more worried about cause area analyses written by individuals (mostly) on their own than the work of EA orgs, which typically goes through more review and is done more formally.