“First, I think a fairer point of comparison isn’t between best and worst but rather between the best measurable intervention and picking randomly. And if you pick randomly, you expect to get the mean effectiveness (rather than the worst or the median).”
I’m not sure if this is fair if you’re trying to communicate the amount of value that could be created by getting more people to switch strategies.
Let’s say everyone picks their strategy randomly. Then they read some information that suggests that some strategies are far more effective than others. Those who are already executing top-10% interventions conclude that they should stick with their current strategies, while some fraction of the other 90% are persuaded to switch. If everyone who switches strategies comes from that bottom-90% group, then the average change in value will look closer to 100x rather than 10x—because if you exclude the positive outliers then the mean will look much lower, and in fact closer to the median.
If you’re trying to suggest that choosing the correct cause area is more important than choosing the correct strategy, because there’s “only” a 10x value difference in choosing the correct strategy, I think you’d need to show why this mean-over-median approach is correct to apply to strategy selection but incorrect to apply to cause area selection. Couldn’t you equally argue that regression to the mean indicates we’ll make errors in thinking some cause areas are 1000x more important or neglected than others?
I agree different comparisons are relevant in different situations.
A comparison with the median is also helpful, since it e.g. tells us the gain that the people currently doing the bottom 50% of interventions could get if they switched.
Though I think the comparison to the mean is very relevant (and hasn’t had enough attention) since it’s the effectiveness of what the average person donates to, supposing we don’t know anything about them. Or alternatively it’s the effectiveness you end up with if you pick without using data.
I think you’d need to show why this mean-over-median approach is correct to apply to strategy selection but incorrect to apply to cause area selection. Couldn’t you equally argue that regression to the mean indicates we’ll make errors in thinking some cause areas are 1000x more important or neglected than others?
Yes absolutely.
I think regression to the mean is a bigger issue for cause selection than solution selection. I’ve tried to take this into account when thinking about between-cause differences, but could have underestimated it.
Basically, I think it’s easier to pick the top 1% of causes than the top 1% of solutions, and there’s probably also greater variance between causes.
(One way to get an intuition for this is that only <0.001% of world GDP goes into targeted xrisk reduction or ending factory farming, while ~10% of world GDP is spent on addressing social issues in rich countries.)
I’m not sure if this is fair if you’re trying to communicate the amount of value that could be created by getting more people to switch strategies.
Let’s say everyone picks their strategy randomly. Then they read some information that suggests that some strategies are far more effective than others. Those who are already executing top-10% interventions conclude that they should stick with their current strategies, while some fraction of the other 90% are persuaded to switch. If everyone who switches strategies comes from that bottom-90% group, then the average change in value will look closer to 100x rather than 10x—because if you exclude the positive outliers then the mean will look much lower, and in fact closer to the median.
If you’re trying to suggest that choosing the correct cause area is more important than choosing the correct strategy, because there’s “only” a 10x value difference in choosing the correct strategy, I think you’d need to show why this mean-over-median approach is correct to apply to strategy selection but incorrect to apply to cause area selection. Couldn’t you equally argue that regression to the mean indicates we’ll make errors in thinking some cause areas are 1000x more important or neglected than others?
I agree different comparisons are relevant in different situations.
A comparison with the median is also helpful, since it e.g. tells us the gain that the people currently doing the bottom 50% of interventions could get if they switched.
Though I think the comparison to the mean is very relevant (and hasn’t had enough attention) since it’s the effectiveness of what the average person donates to, supposing we don’t know anything about them. Or alternatively it’s the effectiveness you end up with if you pick without using data.
Yes absolutely.
I think regression to the mean is a bigger issue for cause selection than solution selection. I’ve tried to take this into account when thinking about between-cause differences, but could have underestimated it.
Basically, I think it’s easier to pick the top 1% of causes than the top 1% of solutions, and there’s probably also greater variance between causes.
(One way to get an intuition for this is that only <0.001% of world GDP goes into targeted xrisk reduction or ending factory farming, while ~10% of world GDP is spent on addressing social issues in rich countries.)