Super interesting—thanks for posting! I was pretty surprised that GiveWell weights their donor survey so highly for their moral weights (60%). I was wondering what’s the rationale for it being 60%, given that these people are both (a) probably not the most knowledgeable about context-specific information (as GiveWell points out here) and (b) also not the recipients themselves. It seems more reasonable to have the GiveWell staff, who have a much better understanding of specific programs and other context, and the beneficiary surveys, to have the majority say in moral weight calculations. I don’t find the arguments below particularly convincing re favouring the donor survey over staff survey:
We have fairly few staff, compared to the number of people who can be surveyed via other methods.
Staff don’t have a unique ability to make these moral judgments. Staff also have limited insight into what the lives of people impacted by our recommendations are actually like, and what would be the most helpful to them.
Staff-assigned moral weights are hard for charities to predict, in that there can be wide swings based on changes in staff composition.
In past years, staff engaged to varying degrees, and then all those responses were aggregated without weighting for level of engagement.
On 1) GiveWell’s staff size (I think?) is about 60 people relative to the donor survey which seems to have been 70 people. Whilst this might be a reason to favour the donor survey in the future if it was about 400 people, I don’t think it’s a great reason to make it 60% now as the donor sample size is similarly small
On 3) Given the sample sizes are pretty similar for both categories, aren’t the donor moral-weights also open to swings? I guess it somewhat depends if you have more stability in your staff or your major donors!
On 4) Again, isn’t this also true for the donor survey? Surely it’s best to weight both groups via engagement so this issue isn’t present for either population. I can also imagine GiveWell staff who can take paid time to work on these moral weights can engage more fully than donors with other full-time jobs.
Finally, I really have no a priori reason to believe that most GiveWell major donors are particularly well-informed about moral weights, so unsure why we should defer to them so much.
For all this, I’m roughly assuming most GiveWell staff could take part in the moral weights but appreciate it might be a smaller group of 10-20 who are just doing the research or program-specific work. I think the arguments are weaker but still stand in that case.
And for future work, GiveWell mentions they want to do more donor surveys but don’t mention surveys of development professionals or other program-specific experts (e.g. malaria experts). This feels a bit odd, as this group seems like a more reliable population relative to donors for the same reasons as above (more specific knowledge).
I’m less sure about this but: I’m also a bit worried about GiveWell saying they want to avoid large swings from year to year (which totally makes sense so organisations can have predictable future incomes). This might unnecessarily keep the moral weights value close to less-than-ideal values, rather than updating with new evidence, which is especially problematic if you think the starting point (based a lot on the donor survey) is less than ideal.
Finally, is the donor survey public and if so, where could I find it?
I agree that this methodology should be better explained and justified. In a previous blog post on Open Philanthropy’s cause prioritisation framework, I noted that Open Philanthropy defers to GiveWell for what the ‘moral weights’ about the badness of death should be, but that GiveWell doesn’t provide a rationale for its own choice (see image below).
I, with colleagues at HLI, are working on a document that specifically looking at philosophical issues about how to compare improving to saving lives. We hope to publish that within 4 weeks.
Super interesting—thanks for posting! I was pretty surprised that GiveWell weights their donor survey so highly for their moral weights (60%). I was wondering what’s the rationale for it being 60%, given that these people are both (a) probably not the most knowledgeable about context-specific information (as GiveWell points out here) and (b) also not the recipients themselves. It seems more reasonable to have the GiveWell staff, who have a much better understanding of specific programs and other context, and the beneficiary surveys, to have the majority say in moral weight calculations. I don’t find the arguments below particularly convincing re favouring the donor survey over staff survey:
On 1) GiveWell’s staff size (I think?) is about 60 people relative to the donor survey which seems to have been 70 people. Whilst this might be a reason to favour the donor survey in the future if it was about 400 people, I don’t think it’s a great reason to make it 60% now as the donor sample size is similarly small
On 3) Given the sample sizes are pretty similar for both categories, aren’t the donor moral-weights also open to swings? I guess it somewhat depends if you have more stability in your staff or your major donors!
On 4) Again, isn’t this also true for the donor survey? Surely it’s best to weight both groups via engagement so this issue isn’t present for either population. I can also imagine GiveWell staff who can take paid time to work on these moral weights can engage more fully than donors with other full-time jobs.
Finally, I really have no a priori reason to believe that most GiveWell major donors are particularly well-informed about moral weights, so unsure why we should defer to them so much.
For all this, I’m roughly assuming most GiveWell staff could take part in the moral weights but appreciate it might be a smaller group of 10-20 who are just doing the research or program-specific work. I think the arguments are weaker but still stand in that case.
And for future work, GiveWell mentions they want to do more donor surveys but don’t mention surveys of development professionals or other program-specific experts (e.g. malaria experts). This feels a bit odd, as this group seems like a more reliable population relative to donors for the same reasons as above (more specific knowledge).
I’m less sure about this but: I’m also a bit worried about GiveWell saying they want to avoid large swings from year to year (which totally makes sense so organisations can have predictable future incomes). This might unnecessarily keep the moral weights value close to less-than-ideal values, rather than updating with new evidence, which is especially problematic if you think the starting point (based a lot on the donor survey) is less than ideal.
Finally, is the donor survey public and if so, where could I find it?
I agree that this methodology should be better explained and justified. In a previous blog post on Open Philanthropy’s cause prioritisation framework, I noted that Open Philanthropy defers to GiveWell for what the ‘moral weights’ about the badness of death should be, but that GiveWell doesn’t provide a rationale for its own choice (see image below).
I, with colleagues at HLI, are working on a document that specifically looking at philosophical issues about how to compare improving to saving lives. We hope to publish that within 4 weeks.