In cases where there is an established science or academic field or mainstream expert community, the default stance of people in EA should be nearly complete deference to expert opinion, with deference moderately decreasing only when people become properly educated (i.e., via formal education or a process approximating formal education) or credentialed in a subject.
On the other hand, many early critiques of GiveWell were basically âWho are you, with no background in global development or in traditional philanthropy, to think you can provide good charity evaluations?â
That seems like a perfectly reasonable, fair challenge to put to GiveWell. Thatâs the right question for people to ask! My impression is that GiveWell, starting early on and continuing today, has acted with humility and put in the grinding hard work to build credibility over time.
I donât think GiveWell would have been a meaningfully useful project if a few people just spent a bit of their spare time over a few weeks to produce the research and recommendations. (Yet that seems sufficient in some cases, such as covid-19, for people in EA to decide to overrule expert opinion.)
It could have been different if there were already a whole field or expert community doing GiveWell-style cost-effectiveness evaluations for global health charities. Part of what GiveWell did was identify a gap in the âmarketâ (so to speak) and fill it. They werenât just replicating the effort of experts.
[Edited on Dec. 16 at 8:50 PM Eastern to add: to clarify, I mean the job GiveWell did was more akin to science journalism or science communication than original scientific research. They were building off of expert consensus rather than challenging it.]
As I recall, GiveWell initially was a project the founders tried to do in their spare time, and then quickly realized was such a big task it would have to be a full-time job. I also hazily recall them doing a lot of work to get up to speed, and also that they had, early on, (and still have) a good process for taking corrections from people outside the organization.
I think as a non-expert waltzing into an established field, you deserve the skepticism and the challenges you will initially get. That is something for you to overcome, that you should welcome as a test. If that is too hard, then the project is too hard.
After all, this is not about status, esteem, ego, or pride, right? Itâs about doing good work, and about doing right by the aid recipients or the ultimate beneficiaries of the work. Itâs not about being right, itâs about getting it right.
Based on my observations and interactions, EA had much more epistemic modesty and much less of a messiah complex in the early-to-mid-2010s. Although I saw a genuinely scary and disturbing sign of a messiah complex from a LEAN employee/âvolunteer as early as 2016, who mused on a Skype call about the possibility of EA solving the worldâs problems in priority sequence. (That person, incidentally, also explicitly encouraged me and my fellow university EA group organizer in the Skype meeting to emotionally manipulate students into becoming dedicated effective altruists in a way that I took to be plainly unethical and repugnant. So, overall, a demented conversation.)
The message and tone in the early-to-mid-2010s was more along the lines of: âwe have these ideas about our obligation to give to charity and about charity cost-effectiveness that might sound radical and counterintuitive, but we want to have a careful, reasonable conversation about it and see if we can convince people weâre on the right trackâ. Whereas by now, in the mid-2020s, the message and tone feels more like: âthe EA community has superior rationality to 99.9% of people on Earth, including experts and academics in fields we just started thinking about 2 weeks ago, and itâs up to us to save the world â no, the lightcone!â. I can see how the latter would be narratively and emotionally compelling to some people, but itâs also extremely off-putting to many other people (like me, perhaps also to Gregory Lewis and other people he knows with backgrounds in public health/âmedicine/âepidemiology/âvirology/âetc., although I donât want to speak for him or the people he knows).
They werenât just replicating the effort of experts.
In fact, they were largely building off the efforts of recognized domain experts. See, e.g.,this bibliography from 2010 of sources used in the âinitial formation of [its] list of priority programs in international aid,â and this 2009 analysis of bednet programs.
> early critiques of GiveWell were basically âWho are you, with no background in global development or in traditional philanthropy, to think you can provide good charity evaluations?â
That seems like a perfectly reasonable, fair challenge to put to GiveWell. Thatâs the right question for people to ask!
I agree with this if you read the challenge literally, but the actual challenges were usually closer to a reflexive dismissal without actually engaging with GiveWellâs work.
Also, I disagree that the only way we were able to build trust in GiveWell was through this:
only when people become properly educated (i.e., via formal education or a process approximating formal education) or credentialed in a subject.
We can often just look at object-level work, study research & responses to the research, and make up our mind. Credentials are often useful to navigate this, but not always necessary.
but the actual challenges were usually closer to a reflexive dismissal
I donât know the specific, actual criticisms of GiveWell youâre referring to, so I canât comment on them â how fair or reasonable they were.
My point is more abstract: just that, in general, it is fair to be to challenge non-experts who are trying to do serious work in area outside of their expertise. It is a challenge that anyone in the position of the GiveWell founders should gladly and willingly accept, or else theyâre not up to the job.
Reputation, trust, and credibility in an area where you are a neophyte is not a right owed to you automatically. Itâs something you earn by providing evidence that you trustworthy, credible, and deserve a good reputation.
We can often just look at object-level work, study research & responses to the research, and make up our mind. Credentials are often useful to navigate this, but not always necessary.
This is hazy and general, so I donât know what you specifically mean by it. But there are all kinds of reasons that non-experts are, in general, not competent to assess the research on a topic. For example, they might be unacquainted with the nuances of statistics, experimental designs, and theories of underlying mechanisms involved in studies on a certain topic. Errors or caveats that an expert would catch might be missed by an amateur. And so on.
I am extremely skeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of study, since this would seem to imply that individual or group possesses preternatural abilities that just arenât realistic given what we know about human limitations. I think the sort of Tony Stark or Sherlock Holmes general-purposes geniuses of fiction are only fictional. But even if they existed, we would know who they are, and they would have a litany of objectively impressive accomplishments.
In cases where there is an established science or academic field or mainstream expert community, the default stance of people in EA should be nearly complete deference to expert opinion, with deference moderately decreasing only when people become properly educated (i.e., via formal education or a process approximating formal education) or credentialed in a subject.
If you took this seriously, in 2011 youâd have had no basis to trust GiveWell (quite new to charity evaluation, not strongly connected to the field, no credentials) over Charity Navigator (10 years of existence, considered mainstream experts, CEO with 30 years of experience in charity sector).
But, you could have just looked at their website (GiveWell, Charity Navigator) and tried to figure out yourself whether one of these organisations is better at evaluating charities.
I am extremely skeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of study, since this would seem to imply that individual or group possesses preternatural abilities that just arenât realistic given what we know about human limitations.
This feels like a Motte (âskeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of studyâ) and Bailey (almost complete deference with deference only decreasing with formal education or credentials). GiveWell obviously never claimed to be experts in much beyond GHW charity evaluation.
If you took this seriously, in 2011 youâd have had no basis to trust GiveWell (quite new to charity evaluation, not strongly connected to the field, no credentials) over Charity Navigator (10 years of existence, considered mainstream experts, CEO with 30 years of experience in charity sector).
Well, no. Because I did hold that view very seriously (as I still do) in the late 2000s and early 2010s, and I came to trust GiveWell.
Charity Navigator doesnât even claim to evaluate cost-effectiveness; they donât do cost-effectiveness estimates.
Even prior to GiveWell, there were similar ideas kicking around. A clunky early term that was used was âphilanthrocapitalismâ (which is a mouthful and also ambiguous). It meant that charities should seek an ROI in terms of impact like businesses do in terms of profit.
Back in the day, I read the development economist William Easterlyâs blog Aid Watch (a project of NYUâs Development Research Institute) and he called it something like the smart aid movement, or the smart giving movement.
The old blog is still there in the Wayback Machine, but the Wayback Machine doesnât allow for keyword search, so itâs hard to track down specific posts.
I had forgotten until I just went spelunking in the archive that William Easterly and Peter Singer had a debate in 2009 about global poverty, foreign aid, and charity effectiveness. The blog post summary says that even though it was a debate and they disagreed on things, they agreed on recommendations to donate to some specific charities.
My point here is that charity effectiveness had been a public conversation involving aid experts like Easterly going back a long time. You never would have taken away from this public conversation that you should pay attention to something like Charity Navigator rather than something like GiveWell.
In the late 2000s and early 2010s, what international development experts would have told you to look at Charity Navigator?
This feels like a Motte (âskeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of studyâ) and Bailey (almost complete deference with deference only decreasing with formal education or credentials). GiveWell obviously never claimed to be experts in much beyond GHW charity evaluation.
I might have done a poor job getting across what Iâm trying to say. Let me try again.
What I mean is that, in order for a person or a group of people to avoid deferring to experts in a field, they would have to be competent at assessing research in that field. And maybe they are for one or a few fields, but not all fields. So, at some point, they have to defer to experts on some things â on many things, actually.
What I said about this wasnât intended as a commentary on GiveWell â sorry for the confusion. I think GiveWellâs approach was sensible. They realized that competently assessing the relevant research on global poverty/âglobal health would be a full-time job, and they would need to learn a lot, and get a lot of input from experts â and still probably make some big mistakes. I think thatâs an admirable approach, and the right way to do it.
I think this is quite different from spending a few weeks researching covid and trying to second-guess expert communities, rather than just trying to find out what the consensus views among expert communities are. If some people in EA had decided in, say, 2018 to start focusing full-time on epidemiology and public health, and then started weighing in on covid-19 in 2020 â while actively seeking input from experts â that would have been closer to the GiveWell approach.
On the other hand, many early critiques of GiveWell were basically âWho are you, with no background in global development or in traditional philanthropy, to think you can provide good charity evaluations?â
That seems like a perfectly reasonable, fair challenge to put to GiveWell. Thatâs the right question for people to ask! My impression is that GiveWell, starting early on and continuing today, has acted with humility and put in the grinding hard work to build credibility over time.
I donât think GiveWell would have been a meaningfully useful project if a few people just spent a bit of their spare time over a few weeks to produce the research and recommendations. (Yet that seems sufficient in some cases, such as covid-19, for people in EA to decide to overrule expert opinion.)
It could have been different if there were already a whole field or expert community doing GiveWell-style cost-effectiveness evaluations for global health charities. Part of what GiveWell did was identify a gap in the âmarketâ (so to speak) and fill it. They werenât just replicating the effort of experts.
[Edited on Dec. 16 at 8:50 PM Eastern to add: to clarify, I mean the job GiveWell did was more akin to science journalism or science communication than original scientific research. They were building off of expert consensus rather than challenging it.]
As I recall, GiveWell initially was a project the founders tried to do in their spare time, and then quickly realized was such a big task it would have to be a full-time job. I also hazily recall them doing a lot of work to get up to speed, and also that they had, early on, (and still have) a good process for taking corrections from people outside the organization.
I think as a non-expert waltzing into an established field, you deserve the skepticism and the challenges you will initially get. That is something for you to overcome, that you should welcome as a test. If that is too hard, then the project is too hard.
After all, this is not about status, esteem, ego, or pride, right? Itâs about doing good work, and about doing right by the aid recipients or the ultimate beneficiaries of the work. Itâs not about being right, itâs about getting it right.
Based on my observations and interactions, EA had much more epistemic modesty and much less of a messiah complex in the early-to-mid-2010s. Although I saw a genuinely scary and disturbing sign of a messiah complex from a LEAN employee/âvolunteer as early as 2016, who mused on a Skype call about the possibility of EA solving the worldâs problems in priority sequence. (That person, incidentally, also explicitly encouraged me and my fellow university EA group organizer in the Skype meeting to emotionally manipulate students into becoming dedicated effective altruists in a way that I took to be plainly unethical and repugnant. So, overall, a demented conversation.)
The message and tone in the early-to-mid-2010s was more along the lines of: âwe have these ideas about our obligation to give to charity and about charity cost-effectiveness that might sound radical and counterintuitive, but we want to have a careful, reasonable conversation about it and see if we can convince people weâre on the right trackâ. Whereas by now, in the mid-2020s, the message and tone feels more like: âthe EA community has superior rationality to 99.9% of people on Earth, including experts and academics in fields we just started thinking about 2 weeks ago, and itâs up to us to save the world â no, the lightcone!â. I can see how the latter would be narratively and emotionally compelling to some people, but itâs also extremely off-putting to many other people (like me, perhaps also to Gregory Lewis and other people he knows with backgrounds in public health/âmedicine/âepidemiology/âvirology/âetc., although I donât want to speak for him or the people he knows).
In fact, they were largely building off the efforts of recognized domain experts. See, e.g., this bibliography from 2010 of sources used in the âinitial formation of [its] list of priority programs in international aid,â and this 2009 analysis of bednet programs.
I agree with this if you read the challenge literally, but the actual challenges were usually closer to a reflexive dismissal without actually engaging with GiveWellâs work.
Also, I disagree that the only way we were able to build trust in GiveWell was through this:
We can often just look at object-level work, study research & responses to the research, and make up our mind. Credentials are often useful to navigate this, but not always necessary.
I donât know the specific, actual criticisms of GiveWell youâre referring to, so I canât comment on them â how fair or reasonable they were.
My point is more abstract: just that, in general, it is fair to be to challenge non-experts who are trying to do serious work in area outside of their expertise. It is a challenge that anyone in the position of the GiveWell founders should gladly and willingly accept, or else theyâre not up to the job.
Reputation, trust, and credibility in an area where you are a neophyte is not a right owed to you automatically. Itâs something you earn by providing evidence that you trustworthy, credible, and deserve a good reputation.
This is hazy and general, so I donât know what you specifically mean by it. But there are all kinds of reasons that non-experts are, in general, not competent to assess the research on a topic. For example, they might be unacquainted with the nuances of statistics, experimental designs, and theories of underlying mechanisms involved in studies on a certain topic. Errors or caveats that an expert would catch might be missed by an amateur. And so on.
I am extremely skeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of study, since this would seem to imply that individual or group possesses preternatural abilities that just arenât realistic given what we know about human limitations. I think the sort of Tony Stark or Sherlock Holmes general-purposes geniuses of fiction are only fictional. But even if they existed, we would know who they are, and they would have a litany of objectively impressive accomplishments.
If you took this seriously, in 2011 youâd have had no basis to trust GiveWell (quite new to charity evaluation, not strongly connected to the field, no credentials) over Charity Navigator (10 years of existence, considered mainstream experts, CEO with 30 years of experience in charity sector).
But, you could have just looked at their website (GiveWell, Charity Navigator) and tried to figure out yourself whether one of these organisations is better at evaluating charities.
This feels like a Motte (âskeptical of any claim that an individual or a group is competent at assessing research in any and all extant fields of studyâ) and Bailey (almost complete deference with deference only decreasing with formal education or credentials). GiveWell obviously never claimed to be experts in much beyond GHW charity evaluation.
Well, no. Because I did hold that view very seriously (as I still do) in the late 2000s and early 2010s, and I came to trust GiveWell.
Charity Navigator doesnât even claim to evaluate cost-effectiveness; they donât do cost-effectiveness estimates.
Even prior to GiveWell, there were similar ideas kicking around. A clunky early term that was used was âphilanthrocapitalismâ (which is a mouthful and also ambiguous). It meant that charities should seek an ROI in terms of impact like businesses do in terms of profit.
Back in the day, I read the development economist William Easterlyâs blog Aid Watch (a project of NYUâs Development Research Institute) and he called it something like the smart aid movement, or the smart giving movement.
The old blog is still there in the Wayback Machine, but the Wayback Machine doesnât allow for keyword search, so itâs hard to track down specific posts.
I had forgotten until I just went spelunking in the archive that William Easterly and Peter Singer had a debate in 2009 about global poverty, foreign aid, and charity effectiveness. The blog post summary says that even though it was a debate and they disagreed on things, they agreed on recommendations to donate to some specific charities.
My point here is that charity effectiveness had been a public conversation involving aid experts like Easterly going back a long time. You never would have taken away from this public conversation that you should pay attention to something like Charity Navigator rather than something like GiveWell.
In the late 2000s and early 2010s, what international development experts would have told you to look at Charity Navigator?
I might have done a poor job getting across what Iâm trying to say. Let me try again.
What I mean is that, in order for a person or a group of people to avoid deferring to experts in a field, they would have to be competent at assessing research in that field. And maybe they are for one or a few fields, but not all fields. So, at some point, they have to defer to experts on some things â on many things, actually.
What I said about this wasnât intended as a commentary on GiveWell â sorry for the confusion. I think GiveWellâs approach was sensible. They realized that competently assessing the relevant research on global poverty/âglobal health would be a full-time job, and they would need to learn a lot, and get a lot of input from experts â and still probably make some big mistakes. I think thatâs an admirable approach, and the right way to do it.
I think this is quite different from spending a few weeks researching covid and trying to second-guess expert communities, rather than just trying to find out what the consensus views among expert communities are. If some people in EA had decided in, say, 2018 to start focusing full-time on epidemiology and public health, and then started weighing in on covid-19 in 2020 â while actively seeking input from experts â that would have been closer to the GiveWell approach.