I’m glad you feel you’re getting a proper sense! It’s surprisingly hard to describe all the information in an informative and intelligible way.
I’m afraid I have no idea why the comment was downvoted. Given the number of upvotes, it could have just been a mistake?
The calculation we used was for people who joined before 2013, and it was 137 out of 268 people, or 51%. Updating it using the 2013 and 2014 cohorts didn’t make much difference (a very slight increase in the percentage), so we continued using the same figure. The cohort data linked to above gives an indication of how many members for 2013 and 2014 gave us enough information to calculate a pledge percentage.
With regard to the counterfactuals, we did at one point in the past ask that. People felt their answers weren’t more accurate when given later on though. The thing that seemed to be going on was that while when you’re considering joining but haven’t yet you have a fairly good sense of how you might act if you join, and if you don’t, but the longer you’re a member the less you identify with the counterfactual of not having joined and so the weirder it feels to think about what would have happened if it had been true. Asking people about these counterfactuals makes it harder to get other information we need (like income and giving data, and updated professions), either because we put it on the same email/survey making people more likely to put off responding to that, or because we send out an additional email/survey, increasing people’s fatigue of being asked stuff by us. So overall asking this seemed to make our calculations worse rather than better.
We’re actually not the only EA org who tries to calculate our overall impact rather than just value which has already come to fruition – for example, 80,000 Hours also does. This calculation is actually very different from one which would try to capture our ‘potential impact for 40 years into the future’. If we ceased all our activities today, I would expect that quite a few people would join and donate due to our past activities. I also expect that we’ve gained quite a bit of experience which will make us more effective in future than we have been in the past. So this calculation doesn’t even capture the potential impact of all the activities we’ve undertaken so far, let alone our potential impact into the future.
Yes, I think it’s very important for EA organisations to try to get a comprehensive picture of their overall impact, both for internal use and for informing others.
I’m glad you feel you’re getting a proper sense! It’s surprisingly hard to describe all the information in an informative and intelligible way.
I’m afraid I have no idea why the comment was downvoted. Given the number of upvotes, it could have just been a mistake?
The calculation we used was for people who joined before 2013, and it was 137 out of 268 people, or 51%. Updating it using the 2013 and 2014 cohorts didn’t make much difference (a very slight increase in the percentage), so we continued using the same figure. The cohort data linked to above gives an indication of how many members for 2013 and 2014 gave us enough information to calculate a pledge percentage.
With regard to the counterfactuals, we did at one point in the past ask that. People felt their answers weren’t more accurate when given later on though. The thing that seemed to be going on was that while when you’re considering joining but haven’t yet you have a fairly good sense of how you might act if you join, and if you don’t, but the longer you’re a member the less you identify with the counterfactual of not having joined and so the weirder it feels to think about what would have happened if it had been true. Asking people about these counterfactuals makes it harder to get other information we need (like income and giving data, and updated professions), either because we put it on the same email/survey making people more likely to put off responding to that, or because we send out an additional email/survey, increasing people’s fatigue of being asked stuff by us. So overall asking this seemed to make our calculations worse rather than better.
We’re actually not the only EA org who tries to calculate our overall impact rather than just value which has already come to fruition – for example, 80,000 Hours also does. This calculation is actually very different from one which would try to capture our ‘potential impact for 40 years into the future’. If we ceased all our activities today, I would expect that quite a few people would join and donate due to our past activities. I also expect that we’ve gained quite a bit of experience which will make us more effective in future than we have been in the past. So this calculation doesn’t even capture the potential impact of all the activities we’ve undertaken so far, let alone our potential impact into the future. Yes, I think it’s very important for EA organisations to try to get a comprehensive picture of their overall impact, both for internal use and for informing others.
Thanks again for answering these questions Michelle and Sam :)