Study results: The most convincing argument for effective donations

Link post

I don’t have any com­men­tary on this be­yond the text, but I love the spirit of this con­test! I hope to see many similar stud­ies con­ducted in the fu­ture, in­side and out­side of the EA com­mu­nity, with a va­ri­ety of ar­gu­ments and other me­dia.


Writ­ten by Eric Schwitzgebel

The Contest

Last fall, Fiery Cush­man and I an­nounced a con­test: We would award $1000 ($500 to the au­thor and $500 to the au­thor’s choice of char­ity) to the au­thor of an ar­gu­ment that effec­tively con­vinces re­search par­ti­ci­pants to donate a sur­prise bonus pay­ment to char­ity at rates statis­ti­cally higher than a con­trol group.

The con­text was this: Chris McVey, Josh May, and I had sev­eral times tried and failed to write ar­gu­ments that would be effec­tive in in­creas­ing par­ti­ci­pants’ dona­tion rates. When we pre­sented par­ti­ci­pants emo­tion­ally mov­ing nar­ra­tives about chil­dren who had been res­cued by char­i­ta­ble dona­tions, char­i­ta­ble dona­tions were higher than in a con­trol con­di­tion—but never when we pre­sented or­di­nary philo­soph­i­cal ar­gu­ments that dona­tion is good or is your duty. See here for a brief write-up of one ver­sion of this paradigm. We won­dered whether the failure might just be the re­sult of our in­abil­ity to write con­vinc­ing ar­gu­ments. There­fore, Fiery and I de­cided to put out the call.

The rules gov­ern­ing en­tries were some­what com­pli­cated—see the origi­nal con­test an­nounce­ment for de­tails—but mainly we wanted to see if an ar­gu­ment in fa­vor of dona­tion could be effec­tive with­out us­ing nar­ra­tive el­e­ments, or men­tion­ing spe­cific in­di­vi­d­u­als, or hav­ing vivid emo­tional con­tent. We couldn’t com­pletely for­bid emo­tional con­tent, since even straight­for­ward fac­tual pre­sen­ta­tion of the facts of hu­man suffer­ing isn’t emo­tion­ally neu­tral. But the main idea was just to have or­di­nary, dry philos­o­phy of the sort or­di­nar­ily done by or­di­nary, dry an­a­lytic philoso­phers.

Our plan was to is­sue the call, se­lect at most 20 ar­gu­ments among those sub­mit­ted, and see if any of those ar­gu­ments could beat a con­trol con­di­tion in which par­ti­ci­pants read part of a mid­dle school physics text. If more than one ar­gu­ment beat con­trol, the award would go to the au­thor of the ar­gu­ment with the high­est mean dona­tion.

After some de­lay due to the pan­demic… we now have a win­ner!

The win­ner was a sub­mis­sion cowrit­ten by Peter Singer and Matthew Lin­dauer, which we will share in its en­tirety be­low.


Gather­ing Sub­mis­sions and Phase 1 Testing

We were delighted by the com­mu­nity’s re­sponse to our con­test call. We re­ceived about 100 sub­mis­sions, about half from pro­fes­sional philoso­phers, psy­chol­o­gists, and ex­per­i­men­tal economists and about half from oth­ers who had heard about the con­test through so­cial me­dia or oth­er­wise.

We only had the re­sources to test twenty ar­gu­ments, so in ac­cor­dance with our plan, we had to cull the 100 down to 20. In se­lect­ing ar­gu­ments, we con­sid­ered sev­eral fac­tors, in­clud­ing the ex­tent to which the ar­gu­ment was in the spirit of the con­test (i.e., a rel­a­tively dry philo­soph­i­cal ar­gu­ment) and the ex­tent to which the ar­gu­ment seemed to us well-writ­ten and likely to be con­vinc­ing. We also wanted the ar­gu­ments to man­i­fest a di­verse range of ap­proaches.

So many of the ar­gu­ments seemed promis­ing that agree­ing among our­selves on a bal­anced set of 20 proved to be a challenge. By the time we had se­lected our 20 and writ­ten and tested the soft­ware for ad­minis­trat­ing the study, the U.S. was shut­ting down due to the pan­demic. We then faced the ques­tion of whether we should sus­pend the study be­cause of the pan­demic, out of con­cerns that re­sponses dur­ing the pan­demic might not be rep­re­sen­ta­tive of re­sponses dur­ing more or­di­nary times. We were con­cerned, for ex­am­ple, that on­line work­ers in the U.S. might be fac­ing un­usual fi­nan­cial hard­ship which would lead to lower rates of dona­tion.

We went ahead with the first phase of test­ing in late April. In this phase, about 2500 par­ti­ci­pants ran­domly read one of the 20 se­lected ar­gu­ments. After read­ing the ar­gu­ment, par­ti­ci­pants clicked to a new page on which they read the fol­low­ing:

Upon com­ple­tion of this study, 10% of par­ti­ci­pants will re­ceive an ad­di­tional $10. You have the op­tion to donate some por­tion of this $10 to your choice among six well-known, effec­tive char­i­ties. If you are one of the re­cip­i­ents of the ad­di­tional $10, the por­tion you de­cide to keep will ap­pear as a bonus cred­ited to your Me­chan­i­cal Turk worker ac­count, and the por­tion you de­cide to donate will be given to the char­ity you pick from the list be­low.
Note: You must pass the com­pre­hen­sion ques­tion and show no signs of sus­pi­cious re­spond­ing to re­ceive the $10. Re­ceipt of the $10 is NOT con­di­tional, how­ever, on how much you choose to donate if you re­ceive the $10.
If you are one of the re­cip­i­ents of the ad­di­tional $10, how much of your ad­di­tional $10 would you like to donate?
[re­sponse scale $0 to $10 in $1 in­cre­ments]
Which char­ity would you like your cho­sen dona­tion amount to go to? For more in­for­ma­tion, or to donate di­rectly, please fol­low the high­lighted links to each char­ity.

[Th­ese char­i­ties were listed in ran­dom­ized or­der.]

After this ques­tion we asked some other ques­tions aimed at ex­plor­ing the psy­cholog­i­cal ba­sis of any differ­ences in re­sponse. For ex­am­ple, in fol­low up ques­tions, par­ti­ci­pants were asked ques­tions about their at­ti­tudes and re­ac­tions to the text, e.g., how con­vinc­ing they found the text, whether their at­ti­tude changed, and whether they donated more than they oth­er­wise would have.

We also asked some de­mo­graphic ques­tions, and we asked par­ti­ci­pants whether they were ex­pe­rienc­ing un­usual fi­nan­cial hard­ship due to the pan­demic and whether con­cerns about the pan­demic had in­fluenced their an­swers.

Par­ti­ci­pants who failed a com­pre­hen­sion check (about 4% of par­ti­ci­pants) were ex­cluded.

In the first round of test­ing, we had about 120 in­cluded par­ti­ci­pants per ar­gu­ment, across the 20 ar­gu­ments. The mean dona­tion rate was $2.88 out of $10, which was sub­stan­tially lower than the mean dona­tion rate of about $3.50 that we have seen in other ver­sions of the ex­per­i­ment. This may have been due to the pan­demic: The ma­jor­ity of par­ti­ci­pants re­ported at least “slight hard­ship” due to the pan­demic, and 26% re­ported mod­er­ate or sig­nifi­cant hard­ship.

The mean dona­tion by ar­gu­ment varied from $2.22 for the ap­par­ently least effec­tive ar­gu­ment to $3.54 for the ap­par­ently most effec­tive ar­gu­ment. How­ever, it was not clear whether the ar­gu­ments ac­tu­ally differed in their effec­tive­ness: A statis­ti­cal test for differ­ence in means was only marginally sig­nifi­cant (ANOVA [19, 2406], F = 1.58, p = .054).

How­ever, our aim in phase 1 was not to reach any defini­tive con­clu­sions but rather to se­lect the five best perform­ing ar­gu­ments for fur­ther test­ing. (Pre­limi­nary Monte Carlo mod­el­ing had sug­gested that the ul­ti­mately best perform­ing ar­gu­ment would likely already be among the top five af­ter 2000 tri­als.) The best perform­ing five ar­gu­ments had mean dona­tion rates from $3.10 to $3.54.

Phase 2 Test­ing: The Winner

In Phase 2, each of the five se­lected ar­gu­ments was viewed by about 335 par­ti­ci­pants, while 471 par­ti­ci­pants viewed the mid­dle school sci­ence text. The re­sults were clear: All five of the ar­gu­ments sub­stan­tially out­performed the con­trol con­di­tion. Thus, the null re­sults of our ear­lier re­search failed to repli­cate with these new and pre­sum­ably more effec­tive ar­gu­ments.

Mean dona­tion ranged from $3.32 to $3.98 for the five ar­gu­ments, com­pared to only $2.58 in the con­trol con­di­tion. An over­all anal­y­sis of var­i­ance was highly statis­ti­cally sig­nifi­cant (ANOVA [5, 2148], F = 11.8, p < .001). In t-tests at an alpha level of .01 (to cor­rect for mul­ti­ple com­par­i­sons), each ar­gu­ment in­di­vi­d­u­ally sig­nifi­cantly out­performed the con­trol con­di­tion (all t > 3.5, all p < .001). How­ever, no differ­ence was statis­ti­cally de­tectable among the ar­gu­ments (in Tukey post-hoc com­par­i­sons on the ANOVA).

Here are the re­sults in a bar chart, with er­ror bars rep­re­sent­ing 95% con­fi­dence in­ter­vals.


As you can see, the win­ner in Phase 2 was Ar­gu­ment 9 by a nose. Ar­gu­ment 9 was also the win­ner by a nose in Phase 1, and thus the win­ner over­all.

Here is the text of Ar­gu­ment 9, which was sub­mit­ted by Peter Singer and Matthew Lin­dauer:

Many peo­ple in poor coun­tries suffer from a con­di­tion called tra­choma. Tra­choma is the ma­jor cause of pre­ventable blind­ness in the world. Tra­choma starts with bac­te­ria that get in the eyes of chil­dren, es­pe­cially chil­dren liv­ing in hot and dusty con­di­tions where hy­giene is poor. If not treated, a child with tra­choma bac­te­ria will be­gin to suffer from blurred vi­sion and will grad­u­ally go blind, though this pro­cess may take many years. A very cheap treat­ment is available that cures the con­di­tion be­fore blind­ness de­vel­ops. As lit­tle as $25, donated to an effec­tive agency, can pre­vent some­one go­ing blind later in life.
How much would you pay to pre­vent your own child be­com­ing blind? Most of us would pay $25,000, $250,000, or even more, if we could af­ford it. The suffer­ing of chil­dren in poor coun­tries must mat­ter more than one-thou­sandth as much as the suffer­ing of our own child. That’s why it is good to sup­port one of the effec­tive agen­cies that are pre­vent­ing blind­ness from tra­choma, and need more dona­tions to reach more peo­ple.

When we asked Singer and Lin­dauer to ver­ify their claim about the cost of treat­ing tra­choma, they referred us to Cook et al. 2006, which es­ti­mates a cost of $7.14 in 2004 U.S. dol­lars for a treat­ment with a 77% cure rate. Singer and Lin­dauer raised the es­ti­mate to $25 to err on the con­ser­va­tive side and ac­count for in­fla­tion.

At the end of this post is an ap­pendix con­tain­ing the other four fi­nal­ist ar­gu­ments. We cau­tion against in­fer­ences based on spe­cific fea­tures of the tra­choma ar­gu­ment that are not also shared by these other ar­gu­ments which performed similarly.

Now al­though the tra­choma ar­gu­ment only won by a nose, in our fol­low-up ques­tions about par­ti­ci­pants’ at­ti­tudes to­ward the text, it won hand­ily, with a mean at­ti­tude of 8.4 on a scale from −21 to +21, com­pared to means of 3.2 to 6.3 for the other texts and 4.7 for the con­trol text. In other words, par­ti­ci­pants not only ac­tu­ally donated at rates sub­stan­tially above the rates in the con­trol con­di­tion, but also they said they donated more than they would oth­er­wise have donated and that the text was per­sua­sive. This was not as true for the other texts, none of which were sig­nifi­cantly differ­ent from con­trol on this mea­sure (ANOVA [5, 2148], F = 16.2, p < .001; in Tukey pair­wise com­par­i­sons ar­gu­ment 9 beats all oth­ers and no other ar­gu­ment beats con­trol).

Conclusion

Hope­fully, we can repli­cate these re­sults af­ter the pan­demic is over. In the mean­time, I draw the ten­ta­tive con­clu­sion that the pre­sen­ta­tion of texts like Singer and Lin­dauer’s can in­deed lead peo­ple to donate more to char­ity than they oth­er­wise would have, con­trary to what was sug­gested by some of my ear­lier null re­sults. Singer and Lin­dauer’s text not only won the con­test but stood out in tend­ing to pro­duce pos­i­tive re­ac­tions from its read­ers, com­pared to the other ar­gu­ments we tested.

We will share more data and thoughts later, as well as the texts and re­sults of all tested ar­gu­ments, but this is enough for to­day.

Con­grat­u­la­tions to Peter and Matt!

Appendix

Ar­gu­ment #3, by Julius Hege (mean dona­tion $3.32):

There are few things that pretty much ev­ery­one agrees on. The value of char­ity is one of those few things. Philoso­phers are fa­mous for be­ing quar­rel­some and agree­ing on very lit­tle. But in a poll of pro­fes­sional ethi­cists, 91% re­sponded that a typ­i­cal per­son in their po­si­tion should give to char­ity. A full 96% re­port donat­ing them­selves last year.[1]

Al­most all re­li­gious tra­di­tions agree as well. For Chris­ti­ans, char­ity is one the seven virtues. John 3:17 states: “But if any­one has the world’s goods and sees his brother in need, yet closes his heart against him, how does God’s love abide in him?”

For Mus­lims, alms­giv­ing (Zakat) is one of the five pillars of Is­lam. There is also vol­un­tary char­ity (Sadaqah) go­ing be­yond that, which is also widely praised. In Ju­daism as well there is the con­cept of Tzedakah, which liter­ally trans­lates to “righ­teous­ness”, but of­ten refers to char­ity. It sees giv­ing not just as an act of benev­olence, but as a duty one has to fulfil.

The pub­lic also agrees: Ac­cord­ing to the Char­i­ties Aid Foun­da­tion, about 88% of peo­ple in the UK en­gaged in at least one char­i­ta­ble ac­tion last year.[2] In the US, 86% of re­spon­dents be­lieve it’s im­por­tant for them to con­tinue to give time and money to char­ity.[3]

Not only is there a wide agree­ment that char­ity is good, many peo­ple even agree that we should give large amounts. For ex­am­ple, Matthew 19:24 states: “Again I tell you, it is eas­ier for a camel to go through the eye of a nee­dle than for some­one who is rich to en­ter the king­dom of God.”

This una­n­im­ity is not sur­pris­ing given the tremen­dous achieve­ments of in­ter­na­tional de­vel­op­ment. Ex­treme poverty is of­ten defined as liv­ing on less than what $1.90 a day can pur­chase in the de­vel­oped world. It is of­ten marked by lack of ad­e­quate food, drink­ing wa­ter, and ba­sic medicine. In 1980, over 40% of the world pop­u­la­tion lived in this ex­treme poverty. To­day, only 10% do.[4] In the same time, global life ex­pec­tancy has in­creased by more than 10 years.[5] And be­cause this poverty is so ex­treme, it is also very cheap to fix: An ex­tra $10 for a per­son in the de­vel­oped world is nice, but of­ten wouldn’t even pay for a sin­gle meal at a restau­rant. But it could also buy 5 long-last­ing bed­nets pre­vent­ing malaria, or de­worm 20 chil­dren, or stave off malnu­tri­tion by dis­tribut­ing iodized salt to 50 peo­ple in need.[6]

In con­clu­sion, given how char­ity is seen as nearly unan­i­mously good and that it can make a larger differ­ence to the world’s poor­est, it seems like the case for char­ity is as strong as it could be.

[1] https://​​doi.org/​​10.1080/​​09515089.2012.727135
[2] https://​​www.cafon­line.org/​​docs/​​de­fault-source/​​about-us-pub­li­ca­tions/​​caf-uk-giv­ing-2018-re­port.pdf,
[3] https://​​www.philan­thropy­roundtable.org/​​al­manac/​​statis­tics/​​na­tional-poll
[4] https://​​our­wor­ld­in­data.org/​​ex­treme-poverty#the-evolu­tion-of-poverty-by-world-re­gions
[5] https://​​our­wor­ld­in­data.org/​​life-ex­pec­tancy#life-ex­pec­tancy-has-im­proved-globally
[6] https://​​www.the­lifey­ou­cansave.org/​​im­pact-calculator

[Note: Par­ti­ci­pants saw the foot­notes, but the links were not click­able, in ac­cord with the rules of the con­test.]


Ar­gu­ment #5, by Adri­ano Man­nino (mean dona­tion $3.84):

Imag­ine a red but­ton. If you push it, two things will hap­pen. First, you will re­ceive $10. Se­cond, a se­ri­ous risk of con­tract­ing malaria will be in­flicted on four chil­dren. They might con­tract the dis­ease, might suffer ter­ribly and might die from it. Would you push the red but­ton?

It seems that push­ing this but­ton would be ex­ces­sively self­ish and cruel. By push­ing it, you would put your own in­ter­est in re­ceiv­ing $10 above the in­ter­ests of four chil­dren in avoid­ing the malaria risk.

Now, imag­ine some­one ran­domly puts $10 in front of you. You could take and keep the money, but you’re also offered the op­por­tu­nity to push a green but­ton for $10 in­stead. If you push it, four chil­dren will be saved from the risk of con­tract­ing malaria. Mosquito bed nets will be dis­tributed to them, for a to­tal cost of $10. Sleep­ing un­der mosquito nets is a highly effec­tive way to pre­vent in­fec­tions in re­gions where malaria is rife. Two chil­dren can sleep un­der one bed net, and two nets will be dis­tributed for $10. So, in­stead of keep­ing the $10, you can use them to save four chil­dren from the malaria risk, by push­ing the green but­ton. Would you push it?

Failing to push the green but­ton is very similar to push­ing the red but­ton. If you push the red but­ton, you re­ceive $10, while four chil­dren are ex­posed to the deadly malaria risk. Similarly, if you fail to push the green but­ton, you re­ceive $10, while four chil­dren are ex­posed to the deadly malaria risk. This trade-off – putting $10 above the in­ter­ests of four chil­dren in avoid­ing malaria – is pre­cisely why push­ing the red but­ton seemed so prob­le­matic. There­fore, if you would not push the red but­ton, push­ing the green but­ton should be a log­i­cal choice. By push­ing the green but­ton, you forego $10 but save four chil­dren from the deadly malaria risk. This should be a great deal, par­tic­u­larly if you wouldn’t push the red but­ton.


Ar­gu­ment #12, by Erik Nook (mean dona­tion $3.86):

One’s life is the product of one’s choices. Soon you will have a choice to make: Do I take $10 or do I give it to char­ity? Philoso­phers have thought of sev­eral rea­sons why donat­ing is the right choice to make to­day, so I’ll tell you about them. But ul­ti­mately, the choice is yours. You should feel good about what­ever choice you make, but first, take some time to think about why donat­ing might be the bet­ter op­tion.

Donat­ing to char­ity does more “good” than tak­ing money for one­self. Some philoso­phers think that we should aim to max­i­mize good out­comes in the world, even if some­times this means that in­di­vi­d­ual peo­ple don’t get what they would like. This is called util­i­tar­i­anism. An ex­am­ple of this ap­proach is that it is a good idea to make a medicine that can save 1 mil­lion peo­ple rather than one that could only save 1 per­son. Soon you will have the op­por­tu­nity to give money to a char­ity that helps a large num­ber of peo­ple. Th­ese philoso­phers would say that this should be pri­ori­tized over what the $10 could do for your­self. Even though it might be painful to not have $10 in your own life, giv­ing up this money is just the right thing to do “for the greater good”.

Selfless­ness is in it­self a “good”. Philoso­phers also think that we should make choices that in them­selves are moral. This is the ba­sis of many re­li­gious and non-re­li­gious codes of ethics. One thing that all re­li­gions and codes of ethics agree upon is that giv­ing to other peo­ple is a good thing to do. Choos­ing to give to­day means that you are mak­ing a choice that al­igns with what hu­man be­ings have thought for cen­turies is a good thing for peo­ple to do.

Selfless­ness can cre­ate a cul­ture that en­courages other peo­ple to do good things. Both psy­chol­o­gists and philoso­phers have shown that giv­ing is con­ta­gious. Peo­ple who think that other peo­ple donate lots of money are also more likely to donate. This cre­ates a cul­ture, a rip­ple-effect, in which donat­ing leads to more donat­ing. So if you donate to­day and tell other peo­ple about it, you are cre­at­ing a cul­ture that not only achieves a good thing in your dona­tion but also in­creases good things hap­pen­ing in the world. You can do a lot of good by donat­ing to­day.

Selfless­ness feels good. Lastly, philoso­phers and psy­chol­o­gists have shown that donat­ing feels good, mean­ing that you can feel pride, re­lief, and joy from donat­ing to­day. Psy­chol­o­gists know that these feel­ings can im­prove your well-be­ing and some philoso­phers would say that these feel­ings bring mean­ing to your life and are im­por­tant to pur­sue. As such, donat­ing to­day not only does good things for other peo­ple, it also does good things for you.

I hope these ideas get you think­ing about the pow­er­ful and pos­i­tive con­se­quences of choos­ing to donate to­day. Thank you for your time.


Ar­gu­ment #14, by Jesse Black­burn (mean dona­tion $3.52):

Think about this for a mo­ment: some­one you know is sud­denly to swap places with one of the 2 billion hu­man be­ings al­ive right now who do not have ac­cess to clean, un­con­tam­i­nated drink­ing wa­ter. Or per­haps with one of the 821 mil­lion peo­ple who suffered from hunger in 2018. What lengths would you go to help the per­son you know? You might be mo­ti­vated to stop all you were do­ing and not rest un­til you had helped them. Now con­sider for a mo­ment that you are un­able to help. Would you ex­pect oth­ers to help? What if some­one was able to help, merely by con­tribut­ing a few dol­lars, but did not do so. If you think that such a failure is a moral ab­hor­rence, then I ask you to re­flect on your own be­hav­ior. Would you al­low some­one else to en­dure these con­di­tions sim­ply be­cause you do not care to bother your­self with the cost of helping them? I sus­pect that you have an­swered no this ques­tion, and yet, I put it to you that you do al­low this hap­pen. Every sin­gle day you have the op­por­tu­nity to spare a small amount of money to provide a fel­low hu­man with the same ba­sic ac­cess to food or drink­ing wa­ter – how of­ten have you done this? For most peo­ple, our priv­ileged ac­cess to clean wa­ter and food was not our choice, we were merely for­tu­nate to be born in the right coun­try at the right time, but we can choose to ex­tend that priv­ilege. I am try­ing to con­vince you that it is in our power to help and that, if the po­si­tions were re­versed, if you (or some­one you know) needed help and other chose not to help, you would con­sider them im­moral. You are, right now, able to very eas­ily help an­other hu­man be­ing, con­sider what you would ex­pect of other peo­ple when you make this de­ci­sion.