This is not especially egregious in a fundraising post, and I understand that in these you have to adopt the persona of a marketer and can’t add too many qualifications and doubts. So I don’t think it’s necessarily bad that you said this. But, as an intellectual matter, I don’t think it’s quite fair to count “[DFID] reallocating £2.5bn to fund research into treating and responding to the diseases that cause the most suffering rather than direct work” as one of your “tangible results so far”. This was discussed plenty on the Facebook group, and as several people pointed out there was no clear evidence that you rather than the very many other groups that commented on DFID’s proposals were responsible for this particular spending decision.
While it seems really important to remember that there were numerous organisations involved, it also seems important to track results like these. Otherwise you run the risk of discounting the impact of any project in which multiple actors were involved unless you have a precise break down of the causal power of each. That would essentially mean that it never looked worthwhile to engage with governments (where there are always a large number of players, and it’s rarely acceptable for information to be made available on the causal power of each). That might be right, but I’d hesitant to make such a strong statement (compared to the more moderate one which GPP is making).
you run the risk of discounting the impact of any project in which multiple actors were involved unless you have a precise break down of the causal power of each. That would essentially mean that it never looked worthwhile to engage with governments
If you look at what I said, you’ll see it doesn’t imply that—it’s simply saying we should do our best job at estimating the impact of GPP, with a spelt out justification of this, and that without this the reallocation of the £2.5bn shouldn’t be relied upon as a tangible result.
Yes this is absolutely not a thing that just GPP did—which is why I tried to call out in this post that several other groups were important to recommending it! (And also something I emphasised in the facebook post you link to.)
I don’t know how many groups fed into the overall process and I’m sure there were big parts of the process I have no knowledge about. I know of two other quite significant entities that have publicly made very similar recommendations (Angus Deaton and the Centre for Global Development) as well as about half a dozen other entities that made similar but slightly narrower suggestions (many of which we cited). The general development aid sector is clearly enormous, but the field of people proposing this sort of thing is smaller.
Assigning causal credit for policy outcomes is very complicated. It obviously matters to us to assess it, so that we can tell if it’s worth doing more work in an area. What we do is just talk to the people we made recommendations to and ask them how significant a role our recommendation played. Usually people prefer we don’t share their reflections further, which is unfortunate but inevitable.
Yes this is absolutely not a thing that just GPP did—which is why I tried to call out in this post that several other groups were important to recommending it!
I have to admit I skimmed over that as I was reading it. It does make it especially unegregious, and I tried to be clear that I didn’t think you were doing anything wrong! I believe it was easy to skim over because it wasn’t flagged as a qualification to the claim that this was a tangible result, but as I said I understand you can’t really make qualifications explicit when needing to raise money.
(And also something I emphasised in the facebook post you link to.)
I’m not very confident on this estimate, but I’d hazard that between 5-50 causally connected groups will have made a recommendation related to the balance of research vs direct work in global health as part of the DfID budget (in either direction).
“The development of ODA five year strategies at Dfid and elsewhere are the result of an enormous number of factors and the interventions of hundreds of experts and even more interventions by NGOs in developing countries. I could name 50 or more groups in regular relationship with Dfid who, along with folks from the GPP could and will be taking a share of the credit.”
“Yes, it’s great that GPP got to have some input, but I agree with Frances to remember all the vast number of others having input.”
“5% credit for any single small organization involved in ODA advocacy would be incredibly high, the main inputs most listened to in ODA come from major players like the World Bank, WHO, Gates Foundation, IMF, individual other ODAs, certain independent and university based researchers ( say Chris Murray,in his own way Hans Rosling, certainly Angus Deaton; Dfid staff talk to tons of people at various levels) to a lesser degree groups like the African Union; when you then move to NGOs, the input of World Vision, Save the Children, CARE, Caritas, etc are strong. I think it is great that GWWC, CEA have worked hard on this effort and join in congratulating them. In fact, the release from GWWC, et al and Seb’s comments here are right on in appropriate modesty about their impact. Groups that promote the importance about having evidence of impact are especially subject to criticism if they overstate “hype” achievements. So, checking the desire to trake more credit than due is wise. The important news is what Difd is setting in place and howmuch goos it can do. Kudoes to Dfid which has played a leading role in the ODA community in pushing and researching measuring effectiveness. Hope this is not seen as poring cold water on an important collaborative effort by EA.”
Although note that “one of several” referred to groups making this particular direction of recommendation. That will cut it back a bit from the full set of people advising DFID.
(Disclaimer: I work at the Global Priorities Project)
This is not especially egregious in a fundraising post, and I understand that in these you have to adopt the persona of a marketer and can’t add too many qualifications and doubts. So I don’t think it’s necessarily bad that you said this. But, as an intellectual matter, I don’t think it’s quite fair to count “[DFID] reallocating £2.5bn to fund research into treating and responding to the diseases that cause the most suffering rather than direct work” as one of your “tangible results so far”. This was discussed plenty on the Facebook group, and as several people pointed out there was no clear evidence that you rather than the very many other groups that commented on DFID’s proposals were responsible for this particular spending decision.
While it seems really important to remember that there were numerous organisations involved, it also seems important to track results like these. Otherwise you run the risk of discounting the impact of any project in which multiple actors were involved unless you have a precise break down of the causal power of each. That would essentially mean that it never looked worthwhile to engage with governments (where there are always a large number of players, and it’s rarely acceptable for information to be made available on the causal power of each). That might be right, but I’d hesitant to make such a strong statement (compared to the more moderate one which GPP is making).
If you look at what I said, you’ll see it doesn’t imply that—it’s simply saying we should do our best job at estimating the impact of GPP, with a spelt out justification of this, and that without this the reallocation of the £2.5bn shouldn’t be relied upon as a tangible result.
Yes this is absolutely not a thing that just GPP did—which is why I tried to call out in this post that several other groups were important to recommending it! (And also something I emphasised in the facebook post you link to.)
I don’t know how many groups fed into the overall process and I’m sure there were big parts of the process I have no knowledge about. I know of two other quite significant entities that have publicly made very similar recommendations (Angus Deaton and the Centre for Global Development) as well as about half a dozen other entities that made similar but slightly narrower suggestions (many of which we cited). The general development aid sector is clearly enormous, but the field of people proposing this sort of thing is smaller.
Assigning causal credit for policy outcomes is very complicated. It obviously matters to us to assess it, so that we can tell if it’s worth doing more work in an area. What we do is just talk to the people we made recommendations to and ask them how significant a role our recommendation played. Usually people prefer we don’t share their reflections further, which is unfortunate but inevitable.
I have to admit I skimmed over that as I was reading it. It does make it especially unegregious, and I tried to be clear that I didn’t think you were doing anything wrong! I believe it was easy to skim over because it wasn’t flagged as a qualification to the claim that this was a tangible result, but as I said I understand you can’t really make qualifications explicit when needing to raise money.
That’s entirely true.
“you rather than the very many other groups” The post above said “several.” I think the number of players here is incredibly important.
Can we get ballpark estimates on how many is the several/very many other players of equivalent or higher weight in this field? 5? 50? 500?
EDIT: Why was my question downvoted? I feel like it’s an important question, and asked in good faith.
I’m not very confident on this estimate, but I’d hazard that between 5-50 causally connected groups will have made a recommendation related to the balance of research vs direct work in global health as part of the DfID budget (in either direction).
That’s maybe a 75% confidence interval.
Here are the relevant comments from one of the posts:
“The development of ODA five year strategies at Dfid and elsewhere are the result of an enormous number of factors and the interventions of hundreds of experts and even more interventions by NGOs in developing countries. I could name 50 or more groups in regular relationship with Dfid who, along with folks from the GPP could and will be taking a share of the credit.”
“Yes, it’s great that GPP got to have some input, but I agree with Frances to remember all the vast number of others having input.”
“5% credit for any single small organization involved in ODA advocacy would be incredibly high, the main inputs most listened to in ODA come from major players like the World Bank, WHO, Gates Foundation, IMF, individual other ODAs, certain independent and university based researchers ( say Chris Murray,in his own way Hans Rosling, certainly Angus Deaton; Dfid staff talk to tons of people at various levels) to a lesser degree groups like the African Union; when you then move to NGOs, the input of World Vision, Save the Children, CARE, Caritas, etc are strong. I think it is great that GWWC, CEA have worked hard on this effort and join in congratulating them. In fact, the release from GWWC, et al and Seb’s comments here are right on in appropriate modesty about their impact. Groups that promote the importance about having evidence of impact are especially subject to criticism if they overstate “hype” achievements. So, checking the desire to trake more credit than due is wise. The important news is what Difd is setting in place and howmuch goos it can do. Kudoes to Dfid which has played a leading role in the ODA community in pushing and researching measuring effectiveness. Hope this is not seen as poring cold water on an important collaborative effort by EA.”
Hmm, “one of several” does seem to be a significant overstatement in that regard.
2.5 billion is also a lot though.
Although note that “one of several” referred to groups making this particular direction of recommendation. That will cut it back a bit from the full set of people advising DFID.
(Disclaimer: I work at the Global Priorities Project)