I don’t have any commentary on this beyond the text, but I love the spirit of this contest! I hope to see many similar studies conducted in the future, inside and outside of the EA community, with a variety of arguments and other media.
Written by Eric Schwitzgebel
The Contest
Last fall, Fiery Cushman and I announced a contest: We would award $1000 ($500 to the author and $500 to the author’s choice of charity) to the author of an argument that effectively convinces research participants to donate a surprise bonus payment to charity at rates statistically higher than a control group.
The context was this: Chris McVey, Josh May, and I had several times tried and failed to write arguments that would be effective in increasing participants’ donation rates. When we presented participants emotionally moving narratives about children who had been rescued by charitable donations, charitable donations were higher than in a control condition—but never when we presented ordinary philosophical arguments that donation is good or is your duty. See here for a brief write-up of one version of this paradigm. We wondered whether the failure might just be the result of our inability to write convincing arguments. Therefore, Fiery and I decided to put out the call.
The rules governing entries were somewhat complicated—see the original contest announcement for details—but mainly we wanted to see if an argument in favor of donation could be effective without using narrative elements, or mentioning specific individuals, or having vivid emotional content. We couldn’t completely forbid emotional content, since even straightforward factual presentation of the facts of human suffering isn’t emotionally neutral. But the main idea was just to have ordinary, dry philosophy of the sort ordinarily done by ordinary, dry analytic philosophers.
Our plan was to issue the call, select at most 20 arguments among those submitted, and see if any of those arguments could beat a control condition in which participants read part of a middle school physics text. If more than one argument beat control, the award would go to the author of the argument with the highest mean donation.
After some delay due to the pandemic… we now have a winner!
The winner was a submission cowritten by Peter Singer and Matthew Lindauer, which we will share in its entirety below.
Gathering Submissions and Phase 1 Testing
We were delighted by the community’s response to our contest call. We received about 100 submissions, about half from professional philosophers, psychologists, and experimental economists and about half from others who had heard about the contest through social media or otherwise.
We only had the resources to test twenty arguments, so in accordance with our plan, we had to cull the 100 down to 20. In selecting arguments, we considered several factors, including the extent to which the argument was in the spirit of the contest (i.e., a relatively dry philosophical argument) and the extent to which the argument seemed to us well-written and likely to be convincing. We also wanted the arguments to manifest a diverse range of approaches.
So many of the arguments seemed promising that agreeing among ourselves on a balanced set of 20 proved to be a challenge. By the time we had selected our 20 and written and tested the software for administrating the study, the U.S. was shutting down due to the pandemic. We then faced the question of whether we should suspend the study because of the pandemic, out of concerns that responses during the pandemic might not be representative of responses during more ordinary times. We were concerned, for example, that online workers in the U.S. might be facing unusual financial hardship which would lead to lower rates of donation.
We went ahead with the first phase of testing in late April. In this phase, about 2500 participants randomly read one of the 20 selected arguments. After reading the argument, participants clicked to a new page on which they read the following:
Upon completion of this study, 10% of participants will receive an additional $10. You have the option to donate some portion of this $10 to your choice among six well-known, effective charities. If you are one of the recipients of the additional $10, the portion you decide to keep will appear as a bonus credited to your Mechanical Turk worker account, and the portion you decide to donate will be given to the charity you pick from the list below.
Note: You must pass the comprehension question and show no signs of suspicious responding to receive the $10. Receipt of the $10 is NOT conditional, however, on how much you choose to donate if you receive the $10.
If you are one of the recipients of the additional $10, how much of your additional $10 would you like to donate?
[response scale $0 to $10 in $1 increments]
Which charity would you like your chosen donation amount to go to? For more information, or to donate directly, please follow the highlighted links to each charity.
Against Malaria Foundation: “To provide funding for long-lasting insecticide-treated net (LLIN) distribution (for protection against malaria) in developing countries.”
Doctors Without Borders / Médecins Sans Frontières: “Medical care where it is needed most.”
Give Directly: “Distributing cash to very poor individuals in Kenya and Uganda.”
Global Alliance for Improved Nutrition: “To tackle the human suffering caused by malnutrition around the world.”
Helen Keller International: “Save the sight and lives of the world’s most vulnerable and disadvantaged.”
Multiple Myeloma Research Foundation: “We collect, interpret and activate the largest collection of quality information and put it to work for every person with multiple myeloma.”
[These charities were listed in randomized order.]
After this question we asked some other questions aimed at exploring the psychological basis of any differences in response. For example, in follow up questions, participants were asked questions about their attitudes and reactions to the text, e.g., how convincing they found the text, whether their attitude changed, and whether they donated more than they otherwise would have.
We also asked some demographic questions, and we asked participants whether they were experiencing unusual financial hardship due to the pandemic and whether concerns about the pandemic had influenced their answers.
Participants who failed a comprehension check (about 4% of participants) were excluded.
In the first round of testing, we had about 120 included participants per argument, across the 20 arguments. The mean donation rate was $2.88 out of $10, which was substantially lower than the mean donation rate of about $3.50 that we have seen in other versions of the experiment. This may have been due to the pandemic: The majority of participants reported at least “slight hardship” due to the pandemic, and 26% reported moderate or significant hardship.
The mean donation by argument varied from $2.22 for the apparently least effective argument to $3.54 for the apparently most effective argument. However, it was not clear whether the arguments actually differed in their effectiveness: A statistical test for difference in means was only marginally significant (ANOVA [19, 2406], F = 1.58, p = .054).
However, our aim in phase 1 was not to reach any definitive conclusions but rather to select the five best performing arguments for further testing. (Preliminary Monte Carlo modeling had suggested that the ultimately best performing argument would likely already be among the top five after 2000 trials.) The best performing five arguments had mean donation rates from $3.10 to $3.54.
Phase 2 Testing: The Winner
In Phase 2, each of the five selected arguments was viewed by about 335 participants, while 471 participants viewed the middle school science text. The results were clear: All five of the arguments substantially outperformed the control condition. Thus, the null results of our earlier research failed to replicate with these new and presumably more effective arguments.
Mean donation ranged from $3.32 to $3.98 for the five arguments, compared to only $2.58 in the control condition. An overall analysis of variance was highly statistically significant (ANOVA [5, 2148], F = 11.8, p < .001). In t-tests at an alpha level of .01 (to correct for multiple comparisons), each argument individually significantly outperformed the control condition (all t > 3.5, all p < .001). However, no difference was statistically detectable among the arguments (in Tukey post-hoc comparisons on the ANOVA).
Here are the results in a bar chart, with error bars representing 95% confidence intervals.
As you can see, the winner in Phase 2 was Argument 9 by a nose. Argument 9 was also the winner by a nose in Phase 1, and thus the winner overall.
Here is the text of Argument 9, which was submitted by Peter Singer and Matthew Lindauer:
Many people in poor countries suffer from a condition called trachoma. Trachoma is the major cause of preventable blindness in the world. Trachoma starts with bacteria that get in the eyes of children, especially children living in hot and dusty conditions where hygiene is poor. If not treated, a child with trachoma bacteria will begin to suffer from blurred vision and will gradually go blind, though this process may take many years. A very cheap treatment is available that cures the condition before blindness develops. As little as $25, donated to an effective agency, can prevent someone going blind later in life.
How much would you pay to prevent your own child becoming blind? Most of us would pay $25,000, $250,000, or even more, if we could afford it. The suffering of children in poor countries must matter more than one-thousandth as much as the suffering of our own child. That’s why it is good to support one of the effective agencies that are preventing blindness from trachoma, and need more donations to reach more people.
When we asked Singer and Lindauer to verify their claim about the cost of treating trachoma, they referred us to Cook et al. 2006, which estimates a cost of $7.14 in 2004 U.S. dollars for a treatment with a 77% cure rate. Singer and Lindauer raised the estimate to $25 to err on the conservative side and account for inflation.
At the end of this post is an appendix containing the other four finalist arguments. We caution against inferences based on specific features of the trachoma argument that are not also shared by these other arguments which performed similarly.
Now although the trachoma argument only won by a nose, in our follow-up questions about participants’ attitudes toward the text, it won handily, with a mean attitude of 8.4 on a scale from −21 to +21, compared to means of 3.2 to 6.3 for the other texts and 4.7 for the control text. In other words, participants not only actually donated at rates substantially above the rates in the control condition, but also they said they donated more than they would otherwise have donated and that the text was persuasive. This was not as true for the other texts, none of which were significantly different from control on this measure (ANOVA [5, 2148], F = 16.2, p < .001; in Tukey pairwise comparisons argument 9 beats all others and no other argument beats control).
Conclusion
Hopefully, we can replicate these results after the pandemic is over. In the meantime, I draw the tentative conclusion that the presentation of texts like Singer and Lindauer’s can indeed lead people to donate more to charity than they otherwise would have, contrary to what was suggested by some of my earlier null results. Singer and Lindauer’s text not only won the contest but stood out in tending to produce positive reactions from its readers, compared to the other arguments we tested.
We will share more data and thoughts later, as well as the texts and results of all tested arguments, but this is enough for today.
Congratulations to Peter and Matt!
Appendix
Argument #3, by Julius Hege (mean donation $3.32):
There are few things that pretty much everyone agrees on. The value of charity is one of those few things. Philosophers are famous for being quarrelsome and agreeing on very little. But in a poll of professional ethicists, 91% responded that a typical person in their position should give to charity. A full 96% report donating themselves last year.[1]
Almost all religious traditions agree as well. For Christians, charity is one the seven virtues. John 3:17 states: “But if anyone has the world’s goods and sees his brother in need, yet closes his heart against him, how does God’s love abide in him?”
For Muslims, almsgiving (Zakat) is one of the five pillars of Islam. There is also voluntary charity (Sadaqah) going beyond that, which is also widely praised. In Judaism as well there is the concept of Tzedakah, which literally translates to “righteousness”, but often refers to charity. It sees giving not just as an act of benevolence, but as a duty one has to fulfil.
The public also agrees: According to the Charities Aid Foundation, about 88% of people in the UK engaged in at least one charitable action last year.[2] In the US, 86% of respondents believe it’s important for them to continue to give time and money to charity.[3]
Not only is there a wide agreement that charity is good, many people even agree that we should give large amounts. For example, Matthew 19:24 states: “Again I tell you, it is easier for a camel to go through the eye of a needle than for someone who is rich to enter the kingdom of God.”
This unanimity is not surprising given the tremendous achievements of international development. Extreme poverty is often defined as living on less than what $1.90 a day can purchase in the developed world. It is often marked by lack of adequate food, drinking water, and basic medicine. In 1980, over 40% of the world population lived in this extreme poverty. Today, only 10% do.[4] In the same time, global life expectancy has increased by more than 10 years.[5] And because this poverty is so extreme, it is also very cheap to fix: An extra $10 for a person in the developed world is nice, but often wouldn’t even pay for a single meal at a restaurant. But it could also buy 5 long-lasting bednets preventing malaria, or deworm 20 children, or stave off malnutrition by distributing iodized salt to 50 people in need.[6]
In conclusion, given how charity is seen as nearly unanimously good and that it can make a larger difference to the world’s poorest, it seems like the case for charity is as strong as it could be.
[1] https://doi.org/10.1080/09515089.2012.727135
[2] https://www.cafonline.org/docs/default-source/about-us-publications/caf-uk-giving-2018-report.pdf,
[3] https://www.philanthropyroundtable.org/almanac/statistics/national-poll
[4] https://ourworldindata.org/extreme-poverty#the-evolution-of-poverty-by-world-regions
[5] https://ourworldindata.org/life-expectancy#life-expectancy-has-improved-globally
[6] https://www.thelifeyoucansave.org/impact-calculator
[Note: Participants saw the footnotes, but the links were not clickable, in accord with the rules of the contest.]
Argument #5, by Adriano Mannino (mean donation $3.84):
Imagine a red button. If you push it, two things will happen. First, you will receive $10. Second, a serious risk of contracting malaria will be inflicted on four children. They might contract the disease, might suffer terribly and might die from it. Would you push the red button?
It seems that pushing this button would be excessively selfish and cruel. By pushing it, you would put your own interest in receiving $10 above the interests of four children in avoiding the malaria risk.
Now, imagine someone randomly puts $10 in front of you. You could take and keep the money, but you’re also offered the opportunity to push a green button for $10 instead. If you push it, four children will be saved from the risk of contracting malaria. Mosquito bed nets will be distributed to them, for a total cost of $10. Sleeping under mosquito nets is a highly effective way to prevent infections in regions where malaria is rife. Two children can sleep under one bed net, and two nets will be distributed for $10. So, instead of keeping the $10, you can use them to save four children from the malaria risk, by pushing the green button. Would you push it?
Failing to push the green button is very similar to pushing the red button. If you push the red button, you receive $10, while four children are exposed to the deadly malaria risk. Similarly, if you fail to push the green button, you receive $10, while four children are exposed to the deadly malaria risk. This trade-off – putting $10 above the interests of four children in avoiding malaria – is precisely why pushing the red button seemed so problematic. Therefore, if you would not push the red button, pushing the green button should be a logical choice. By pushing the green button, you forego $10 but save four children from the deadly malaria risk. This should be a great deal, particularly if you wouldn’t push the red button.
Argument #12, by Erik Nook (mean donation $3.86):
One’s life is the product of one’s choices. Soon you will have a choice to make: Do I take $10 or do I give it to charity? Philosophers have thought of several reasons why donating is the right choice to make today, so I’ll tell you about them. But ultimately, the choice is yours. You should feel good about whatever choice you make, but first, take some time to think about why donating might be the better option.
Donating to charity does more “good” than taking money for oneself. Some philosophers think that we should aim to maximize good outcomes in the world, even if sometimes this means that individual people don’t get what they would like. This is called utilitarianism. An example of this approach is that it is a good idea to make a medicine that can save 1 million people rather than one that could only save 1 person. Soon you will have the opportunity to give money to a charity that helps a large number of people. These philosophers would say that this should be prioritized over what the $10 could do for yourself. Even though it might be painful to not have $10 in your own life, giving up this money is just the right thing to do “for the greater good”.
Selflessness is in itself a “good”. Philosophers also think that we should make choices that in themselves are moral. This is the basis of many religious and non-religious codes of ethics. One thing that all religions and codes of ethics agree upon is that giving to other people is a good thing to do. Choosing to give today means that you are making a choice that aligns with what human beings have thought for centuries is a good thing for people to do.
Selflessness can create a culture that encourages other people to do good things. Both psychologists and philosophers have shown that giving is contagious. People who think that other people donate lots of money are also more likely to donate. This creates a culture, a ripple-effect, in which donating leads to more donating. So if you donate today and tell other people about it, you are creating a culture that not only achieves a good thing in your donation but also increases good things happening in the world. You can do a lot of good by donating today.
Selflessness feels good. Lastly, philosophers and psychologists have shown that donating feels good, meaning that you can feel pride, relief, and joy from donating today. Psychologists know that these feelings can improve your well-being and some philosophers would say that these feelings bring meaning to your life and are important to pursue. As such, donating today not only does good things for other people, it also does good things for you.
I hope these ideas get you thinking about the powerful and positive consequences of choosing to donate today. Thank you for your time.
Argument #14, by Jesse Blackburn (mean donation $3.52):
Think about this for a moment: someone you know is suddenly to swap places with one of the 2 billion human beings alive right now who do not have access to clean, uncontaminated drinking water. Or perhaps with one of the 821 million people who suffered from hunger in 2018. What lengths would you go to help the person you know? You might be motivated to stop all you were doing and not rest until you had helped them. Now consider for a moment that you are unable to help. Would you expect others to help? What if someone was able to help, merely by contributing a few dollars, but did not do so. If you think that such a failure is a moral abhorrence, then I ask you to reflect on your own behavior. Would you allow someone else to endure these conditions simply because you do not care to bother yourself with the cost of helping them? I suspect that you have answered no this question, and yet, I put it to you that you do allow this happen. Every single day you have the opportunity to spare a small amount of money to provide a fellow human with the same basic access to food or drinking water – how often have you done this? For most people, our privileged access to clean water and food was not our choice, we were merely fortunate to be born in the right country at the right time, but we can choose to extend that privilege. I am trying to convince you that it is in our power to help and that, if the positions were reversed, if you (or someone you know) needed help and other chose not to help, you would consider them immoral. You are, right now, able to very easily help another human being, consider what you would expect of other people when you make this decision.
Thanks for doing this research, nice work.
Could you make your figure a little larger, it’s hard to read on a desktop. It might also be easier for the reader if each of the five arguments had a one-word name to keep track of the gist of their actual content.
I don’t think this is quite right. Arguments 5 and 12 are very much within the confidence interval for Argument 9. Eyeballing it I would guess we can only be about 60% confident that argument 9 would do better again if you repeated the experiment.
I would summarise the results as follow:
All five arguments substantially outperformed the control, on average increasing giving by around 45%.
We also had some evidence that Arguments 5, 9 and 12 all outperformed Arguments 3 and 14, perhaps having about 30% more impact.
Eric Schwitzgebel responded as follows to a similar comment on his wall:
But many won’t interpret it that way and further clarification would have been good, yes.
Edit: Schwitzgebel’s post actually had another title: “Contest Winner! A Philosophical Argument That Effectively Convinces Research Participants to Donate to Charity”
I wonder if something similar could also be done but with donations to long-term issues instead? I.e. the same set-up but searching for the most convincing long-term arguments. Would this be of interest? (I’ve been thinking about setting something up along the lines of this).
Yes, I was just going to ask if anyone had looked at longtermist arguments in a similar way, or even just compiled a similar list of any short, punchy longtermist pitches that are out there. I’ve been thinking of printing out some pamphlets or something to distribute around town when I go for walks, and it might be nice to be able to represent multiple EA pillars on one pamplet.
I also think it would be interesting to see results on longtermism because it’s a much stranger, less familiar idea (more different than other charity messaging people have heard before), so it might be harder to explain in a short format, but there might be correspondingly big wins from introducing people to such a totally new concept.
An anonymous comment someone asked me to post for them (similar to Mati Roy’s recent comment):
I took “It felt kind of symmetric” to be referring to Guided By The Beauty Of Our Weapons, and “starting with a conclusion and then trying to find all the best arguments for that conclusion” to be referring to The Bottom Line.
I replied (edited):
I’d help to fund that study (at least, the anti-donation study) if someone put it together. Have you or the anonymous commenter proposed it to Schwitzgebel? For all I know, he might react very enthusiastically.
(If no one’s asked him yet, would you mind if I passed this comment along?)
I haven’t talked to Schwitzgebel, and you’re of course welcome to pass this all on. :)
I’m impressed by the top 5 entries, approximately in the order of the mean donation amount they caused.
I submitted an entry to this contest which I thought was decent when I wrote it, but now seems really mediocre upon re-reading it (see my reply to this comment for my entry).
One thing I noticed about all five of the Top 5 arguments (though not my entry) is that they all can be interpreted as guilting the reader into donating. That is, there is an unstated implication the reader could draw that the reader would be a bad person if they chose not to donate:
Argument #9: After reading this winning argument, the reader might think: “Now if I don’t donate the $10 I’d be admitting that I don’t value the suffering of children in poor countries even one-thousandth as much as my own child (or someone I know’s child). What a terrible person I’d be. I don’t want to feel like a bad person so I’ll donate.”
Argument #3: Someone might think: “Practically everyone agrees that giving to charity is good, so if I don’t donate the $10 that would make me bad. I don’t want to feel like a bad person so I’ll donate.”
Argument #5: “If I take the $10 rather than donate it, I’d be putting my own interest in receiving $10 above the interests of four children who don’t want malaria, which would make me a bad person. I don’t want to feel like a bad person so I’ll donate.”
Argument $12: “I just read that I should feel good about whether I decide to ‘take’ or ‘give’ the $10. And also that I should prioritize helping a large number of people over the value of $10 for myself. So now I’m not sure that I could feel good about ‘taking’ the money for myself. I don’t want to feel guilty over $10 so I’ll donate.”
Argument #14: “‘Every single day you have the opportunity to spare a small amount of money to provide a fellow human with the same basic access to food or drinking water – how often have you done this?’ Clearly I’d be a bad person if I decided to take $10 that is offered to me rather than give the $10 to provide a fellow human with basic access to food or drinking water. I don’t want to feel like a bad person, so I’ll donate.
I think an important thing to consider with this study (as with most psychology-style experiments) is the generalisability/external validity of the results, and in particular the extent to which the effects may only be short-lived and may primarily reflect things like demand characteristics and social desirability bias.
These results might not matter much if they just reflect the best ways to get people to give an extra dollar right after being shown some relevant text. What matters more is the bets ways to get people to give substantially more, or to give moderate amounts in an ongoing way (even when they’re not being observed and haven’t seen some relevant text right beforehand).
And I think this is worth bearing in mind when we think about the value of arguments that may induce some degree of guilt. These results suggests that those sorts of arguments may work best for influencing people to give slightly more in this low-stakes and unusual setting (though personally Argument 12 didn’t seem very guilt-inducing to me). But it still seems plausible that those sorts of arguments don’t work especially well—or even work fairly badly—for leading to larger or more ongoing donations (e.g., because people get annoyed or stressed by these arguments and thus stop engaging over time or when no one’s looking).
That said, I have no specific evidence for that, and I’m not saying I think that’s especially likely. Anecdotally, it does seem that arguments that could induce some degree of guilt have been important in many EAs’ journeys into EA, and in many EAs’ current thinking. (That’s the case for me, for example.)
My entry (475 words):
My entry is different than all five of the Top 5 entries in that my entry is the only one to not engage with the objection “but what about the value of $10 for myself?”
The primary reason why people don’t give presumably is because they’d rather have the money for their own uses.
All five of the Top 5 arguments engage with this idea by implying in one way or another that taking the money for your own use would make you a selfish or bad person.
My entry seems mediocre (in part) because it only highlights the benefits of effective giving to others. It does not attempt to make the reader feel guilty about turning down these bargain opportunities and taking the $10 for oneself.
I would suggest editing the title to “contest results” because this isn’t testing a theory with controls in the way a “study” would, but I am known for being a bit of a pedant.
Presumably the theory is that philosophical argument can(not) increase donations, and it sounds like they had a randomised control in the form of an unrelated text.
Ah sorry, I had heard about the competition from others, heard the quality of writing varied significantly, and so assumed that it wasn’t really a study. My bad!