When we “impact rate” the plan changes, we also try to make an initial assessment of how much is counterfactually due to us (as well as how much extra impact results non-counterfactually adjusted).
We then to more in-depth analysis of the counterfactuals in crucial cases. Because we think the impact of plan changes it fat tailed, if we can understand the top 5% of them, we get a reasonable overall picture.
We do this analysis in documents like this: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/
Each individual case is debateable, but I think there’s a large enough volume of cases now to justify that we’re having a substantial impact.
Do you have any info on how reliable self-reports are wrt counterfactuals about career changes and EWWC pledging?
I can imagine that people would not be very good at predicting that accurately.
One would expect some social acceptability bias that might require substantial creativity and work to measure.
Hi there,
It’s definitely hard for people to estimate.
When we “impact rate” the plan changes, we also try to make an initial assessment of how much is counterfactually due to us (as well as how much extra impact results non-counterfactually adjusted).
We then to more in-depth analysis of the counterfactuals in crucial cases. Because we think the impact of plan changes it fat tailed, if we can understand the top 5% of them, we get a reasonable overall picture. We do this analysis in documents like this: https://80000hours.org/2016/12/has-80000-hours-justified-its-costs/
Each individual case is debateable, but I think there’s a large enough volume of cases now to justify that we’re having a substantial impact.