First, on honesty. As I said above, I completely agree with you on honesty: “bad arguments for a good conclusion are not justified.” This is one of my (and I’d say the EA community as a whole) strongest values. Arguments are not soldiers, their only value is in their own truth. SSC’s In Favor of Niceness, Community, and Civilization sums up my views very well. I’m glad we’re after the same goal.
That said, in popular writing, it’s impossible to reflect the true complexity of what’s being described. So the goal is to simplify as much as possible, while losing as little truth as possible. If someone simplifies in a way that’s importantly misleading, that’s an important failure and should be condemned. But the more I dig into each of these arguments, the more I’m convinced MacAskill is doing a very good job maintaining truth while simplifying.
Charity Navigator. MacAskill says “One popular way of evaluating a charity is to look at financial information regarding how the charity spends its money.” He says that CN takes this approach, and then quotes CN saying that many of the best charities spend 25% or less on overhead. You say this is a misquote, because CN later says that high overhead can be OK if balanced by other indicators of financial health. CN says they like to see charities “that are able to grow their revenue at least at the rate of inflation, that continue to invest in their programs and that have some money saved for a rainy day.”
I see absolutely no misrepresentation here. MacAskill says CN evaluates based on financials such as overhead pay, and quotes CN saying that. He never says that CN only looks at overhead pay, neglecting other financials. In fact, his quote of CN says that overhead indicator is a “strong indicator” in “most” charities, which nobody would interpret as claiming that CN literally only evaluates overhead. The fact that CN does in fact care about financials other than overhead is abundantly clear when reading MacAskill’s summary. MacAskill perfectly represents their view. I doubt someone from CN would ever take issue with that first paragraph.
Playpumps. Charge by charge: 1. After checking out both the UN and SKAT reports, I agree with MacAskill: they’re “damning”. 2. MacAskill says “But in order to pump water, PlayPumps need constant force, and children playing on them would quickly get exhausted.” You quote UNICEF saying “Some primary school children complained of becoming tired very quickly after pushing the pump, particularly as additional torque is required with each rotation to commence the upstroke of the piston.” Look at a video of one in motion, it’s clear that it spins easy for a little while but also constantly requires new force. No misrepresentation. 3. “Children sometimes fell off and broke limbs” is an exaggeration. One child fractured their arm, not multiple. MacAskill misrepresented the number of injuries. 4. The reporter said that PlayPump requires 27 hours of pumping a day in order to meet its ambition of supplying 15 liters a day to 10 million people using 4000 PlayPumps. Assuming one PlayPump per village, that means a village of 2500 would require 27 hours a day of PlayPump to meet their water needs. The only editorializing MacAskill does is call a village of 2500 “typical”. No misrepresentation. 5. MacAskill that PlayPumps often replaced old pumps. You correctly point out that in most countries, that did not happen. Bottom line: You’re right that (i) MacAskill exaggerates the number of children who broke bones; it was one reported case, not multiple; and (ii) MacAskill incorrectly implies that PlayPumps often replaced old pumps, when in fact they rarely did.
Again, thank you for continuing to engage in this in a fair and receptive way. But after spending a lot of time looking into this, I’m less convinced than I ever was of your argument. You have four good points: (i) MacAskill should’ve used other deworming evidence; (ii) MacAskill exaggerated the number of children who broke bones on PlayPumps; (iii) MacAskill incorrectly implies that PlayPumps often replaced old pumps, when in fact they rarely did; (iv) MacAskill incorrectly reported the question asked by a survey on ethical companies. You might have a good point with the John Bunker DALY estimates, but I haven’t looked into it enough.
Framed in the right way, these four points would be helpful, useful feedback for MacAskill. Four slips in 200 pages seems impressively good, but MacAskill surely would have promptly updated his Errata page, and that would be that. Nothing significant whatsoever about the book would’ve changed. But because they were framed as “William MacAskill is a liar”, nobody else has been willing to engage your points, lest they legitimize clearly unfair criticism. Yes, he didn’t make the best response to your points, but to be frank, they were quite unorganized and hard to follow—it’s taken me upwards of 5 hours in sum to get to the bottom of your claims.
At this point, I really don’t think you can justifiably continue to hold your either of your positions: that DGB is significantly inaccurate, or that MacAskill is dishonest. I really do believe that you’re in this in good faith, and that your main error (save the ad hominem attack, likely a judgement error) was in not getting to the bottom of these questions. But now the questions feel very well resolved. Unless the four issues listed above constitute systemic inaccuracy, I really don’t see an argument for it.
Sincerely, thank you for engaging, and if you find these arguments correct, I hope you’ll uphold our value of honesty and apologize to MacAskill for the ad hominem attacks, as well as give him a kinder, more accurate explanation of his inaccuracies. I hope I’ve helped.
I’ll headline this by saying that I completely believe you’re doing this in good faith, I agree with several of your criticisms, and I think this deserves to be openly discussed. But I also strongly disagree with your conclusion about MacAskill’s honesty, and, even if I thought it was plausible, it still would be an unnecessary breach of etiquette that makes open conversation near impossible. I really think you should stop making this an argument about MacAskill’s personal honesty. Have the facts debate, leave ad hominem aside so everyone can fully engage, and if you’re proven right on the facts, then raise your honesty concerns.
First I’d like to address your individual points, then your claims about MacAskill.
Misreporting the deworming study. I think this is your best point. It seems entirely correct that if textbooks fail because they don’t improve test scores, that deworming should fail by the same metric. But I agree with /u/ScottAlexander that, in popular writing, you often don’t have the space to specifically go through all the literature on why deworming is better. MacAskill’s deworming claims were misleading on one level, in that the specific argument he provided is not a good one, but also fair on another level: MacAskill/GiveWell has looked tons into deworming, concluded that it’s better than textbooks, and this is the easiest way to illustrate why in a single sentence. Nobody reading this is looking for a survey of the evidence base on deworming; they’re reading it as an introduction to thinking critically about interventions. Bottom line: MacAskill probably should’ve found a better example/line of defense that was literally true, but even this literally false claim serves its purpose in making a broader, true point.
Interpreting GiveWell literally. Jan’s comment was perfect: GiveWell is not the supreme authority on how to interpret their numbers. Holden prefers to give extra weight to expected values with low uncertainty, MacAskill doesn’t, and that’s a legitimate disagreement. In any case, if you think people shouldn’t ever interpret GiveWell’s estimates literally when pitching EA, that’s not a problem with MacAskill, it’s a problem with >90% of the EA community. Bottom line: I think you should drop this argument, I just don’t think it’s correct.
Misrepresenting Charity Navigator. As MacAskill admits, it’s inaccurate to conflate overhead costs and CEO pay. Good find, the specific criticism was correct. But after thinking it through, I think MacAskill’s argument, while botching that single detail, is still a fair criticism of an accurate overall characterization of Charity Navigator. Let’s focus on the donut example. MacAskill says that if a donut charity had a low-paid CEO, CN would rate them highly. You correctly identify that CN cares about things other than CEO pay, and is willing to give good ratings to charities with highly paid CEOs if they do well on other metrics, namely financial stability, accountability, and transparency. BUT, MacAskill’s point I believe would be that none of those other CN metrics have to do with the effectiveness of the intervention or the cause area. CN will let financial stability and low employee costs outweigh a highly-paid CEO, but they won’t let a terrible cause bring down your rating. So if you had a highly efficient, financial well-managed donut charity, CN really would give them a good rating. Bottom line: MacAskill mistakenly conflates CEO pay with overhead costs. But that’s incredibly minor, and no reader is going to be annoyed by it. His fundamental point is correct: CN doesn’t care about cause area or intervention effectiveness, and that’s silly to the point of absurdity.
Further, even if you still think MacAskill unfairly represented CN’s position, I’m willing to cut him a bit slack on it. Do check out their hit piece on effective altruism. It’s aggressive, demeaning, and rude. Yes, it would’ve been better if MacAskill took the perfect high road, but if the inaccuracy really is minor, I think we can excuse it.
Exaggerating PlayPump’s failures. At first, I bought what you said in your comment. Everyone can read what you have to say themselves, but basically, it seems like MacAskill may have exaggerated the reports he cites discussing the failures of the PlayPump. But after a quick Google, it seems like this is another example of a specific line of argumentation that really isn’t rigorous, but that tries to make a fair point in a single sentence. PlayPump was a disaster, everyone agrees, and MacAskill was absolutely not the first to say so. So although MacAskill could’ve better explained specifically why it was a failure, without exaggerating reports, his conclusion is completely fair. I absolutely agree with the importance of honesty, and that bad arguments for a good conclusion are not justified. But this is popular writing, and he really doesn’t have space to fully review all the ins and outs of PlayPumps. Bottom line: I wish MacAskill more accurately justified his view, but nobody who looks into this should feel misled about the overall point of the failure of PlayPumps.
Conclusion: I think you correctly identify several inaccuracies in DGB. But after looking into them myself, I think you really overestimated the importance of these inaccuracies. Except perhaps the deworming example, none of these inaccuracies, if corrected, would change anything important about the conclusions of the book.
Even if you think I’m underestimating the level of inaccuracy, it seems near impossible that this is a sign of malice. If you go into a Barnes and Noble and pick out the popular nonfiction sitting to the left and right of DGB, I think you’d find dozens of inaccuracies far more important than these. Popular writing needs to oversimplify complex debates. DGB does an admirable job of preserving truth while simplifying.
I’ll reiterate that I really do believe in your good faith. You found inaccuracies, and you began worrying about MacAskill’s honesty, which drove you to find more inaccuracies. I think if you step back and consider the charitable interpretation of these flaws, though, you’ll realize that there are good reasons why they’re minor, and that it’s highly unlikely that this is the result of malice.
But finally, regardless of your conclusions on MacAskill’s honesty, I’ll say again that it’s absolutely destructive to open discourse and everyone’s goals to headline your post calling MacAskill a liar. If you want the community to engage this conversation, you have to stick to the substantive disagreements. If consensus concludes that MacAskill importantly and repeatedly fails, people will question his honesty on their own. But I think if the open debate is had, you’ll eventually come around to thinking that these inaccuracies are minor, inconsequential, and accidental.