I’ll Fund You to Give Away ‘Doing Good Better’ - Surprisingly Effective?
This is an expanded version of a post I made to the EA Facebook group in December 2018. Thanks to Aaron Gertler for reading a draft of this post, finding many typos (I’m sure I’ve added more since), and making many good suggestions. Besides commenting here, you can contact me about this project at eabookgiveaway [at] gmail [dot] com.
Summary
I got a surprising number of responses when I offered to send out free copies of Doing Good Better on social media. It seemed like it could be a scalable, cost-effective form of outreach, so I have created a survey to assess impact. In the following post, I try to convince others (you!) to make the offer—in order to gain a larger sample size. I detail what I did. I give some reasons why you should consider it, and suggestions how to do it. I estimate how long it will take to publish meaningful results, and I request feedback.
Surprising Response
Somewhat impulsively, back in December, I posted to Facebook and Instagram, offering to send a copy of Doing Good Better to anyone who wanted to read it. It seemed like a nice thing to do around the holidays, but I wasn’t expecting much response—especially since I rarely post on Facebook, and often hear crickets when I do.
But the algorithm gods must have smiled. 21 people responded to my offer, 3 more to a repost, and another 3 to a friend who was inspired to make the same offer. In total 28 people, that I know, of received the book. It broke down like this:
14 of my FB friends and 2 IG followers got physical books
4 FB friends got ebooks
1 FB friend commented that he’d bought his own book
1 friend told me in-person that he’d given one as a gift
3 of my brother’s FB friends got physical books when he shared my post
3 of a friend’s FB friends got physical books when she made her own offer
Additionally, I sent articles to 2 people who expressed interest in reading about EA, but who didn’t want to commit to a book.
Other EA Book Giveaways
When I posted about it to the EA Facebook group, people were interested to read survey results, and generally positive about the idea. I also learned that EA New Zealand has an ongoing offer of a free copy of the book, and that EA London has also handed out books (DGB, as well as Superintelligence, Poor Economics, The Life You Can Save and 80,000 Hours) at their events. They both shared their survey questions.
Catherine from EA NZ told me that they have now given away over 900 copies. She said that they are in the midst of analyzing recent data, but she has concerns it may not be as effective as they initially thought (or perhaps not as effective now as at the beginning).
According to EA London’s summary, ”...it seemed that handing out books at EA London events increased the amount of engagement”, though, to my knowledge, they didn’t test against any kind of control group (I.E. attendees who weren’t offered a book). It’s also worth noting that their book recipients were attending an EA event and, therefore, had at least some familiarity with EA.
I was interested in how they decided when to send the survey, as I was concerned that it often takes me a while to get around to reading a new book. Catherine confirmed that a lot of people responded that they hadn’t read it, but that the survey was a reminder to do so. She said she was planning to change it to instruct people NOT to fill out the survey if they haven’t read it, then to send a reminder to those people later. This seemed like a really great idea.
Surveying Recipients
My goal was to measure, as much as possible, the value from recipients reading the book vs counterfactual. I found it helpful to break it down in a very obvious way:
(A) the value of recipients’ actions after reading the book
(minus)
(B) the value of what they would have done anyway
(minus)
(C) the cost
For A, I didn’t want to ask what people intend to do, as I am very uncertain how that correlates with what they actually do. Rather, I asked what changes recipients have already made. The obvious drawback to this approach is that much of the value of their actions may not yet be realized, because it doesn’t capture anything done after taking the survey.
To remedy that, the last question asks if people would be willing to take a follow-up survey after a year (the follow-up will then ask for another one, etc. etc.). Some may not be willing to do this, but the survey is pretty short and unimposing, and it seems like willingness to do it would correlate highly with interest in EA—which, in turn, would correlate highly with actions taken/value.
This certainly won’t catch everything everyone does as a result of reading the book, but it seems better than tracking people’s intentions, then trying to figure out how to value them. I recognize that this is a bit of an unusual way to do it, so I would appreciate any feedback.
For B, I was initially thinking of asking people if the book was their introduction to EA, or to somehow gauge their involvement in EA before and after, but it’s unclear how that information would fit cleanly into the equation—especially if they already had some knowledge of EA. I decided to just ask people what they have done, that they wouldn’t have otherwise. I do have some concerns that people might misunderstand, or not be able to estimate this accurately. If this instruction does not seem clear, or you feel you have a better idea, please comment.
C is obviously not a part of the survey, but just carefully tracking the time and money spent, which I have done.
The Survey can be found here. (I will wait a couple of weeks to send it out, in order to incorporate any changes from community feedback.)
Why You Should Consider Doing It
To help me gather more data. It’s unclear how many of my ~28 recipients will take the survey, but it’s clear that even if they all did, it’s not much of a sample size.
Even if it doesn’t turn out to be cost-effective, it’s valuable to find out.
It’s one of the least-intrusive ways to “preach” EA ideas I can think of. You post about it. People respond if they are interested, and they just ignore it if they aren’t.
If you are uncomfortable, or especially bad at (as I am) explaining EA in-person, you can still do this.
Even if you’ve already discussed EA with many of your friends, there are probably some with whom you have not, or who might be more receptive to a book than a personal conversation.
It’s inexpensive, at about $12 a book (more on costs below). If you can’t afford that, I can help you cover the cost.
It’s easy, and should only take a couple of hours of your time.
How to Do it
If you’d like to do it, please comment on this post, or email me at eabookgiveaway [at] gmail [dot] com. I plan to can keep track of who’s doing it, so I can follow up to see how it went. The following guide is based entirely on my intuitions and my own experience doing it. It will be valuable to integrate feedback from other people who have done it as well.
Writing/Posting
Here’s what I wrote (posted along with an image of the book cover):
“Content warning: Sincerity. For the holidays this year, I am giving out copies of this book—just message me your address and I’ll send you one! Effective Altruism is a movement I’ve become increasingly excited about over the past couple of years. Besides the inherent #warmfuzzies of helping others, it’s also an excellent antidote to the #culturewars and reading about #trump #brexit #opioidepidemic or whatever depressing news item. This book is a great primer. Well written, more entertaining than you’d think, not too technical and pretty much inspiring. Seriously, message me and I will mail you a copy for Christmakwaanzakah.”
Here is some of my reasoning and a few other ideas and suggestions:
I really wanted it to sound like a personal offer to my friends, not a copy/pasted promotion or something. I think using your own voice is perhaps the most important thing.
I tried to keep it fairly short. Essential ingredients: 1) the offer, 2) mention EA and my feelings about it, 3) my brief thoughts about the book. The tongue-in-cheek “content warning” part made sense for me, to differentiate it from the light-hearted stuff I mostly post.
I’m only on Facebook and Instagram, but I don’t see why it wouldn’t work on Twitter, or anywhere else. The majority of my responses were from Facebook, but I got a couple from Instagram too—despite the fact that that account is private, with far fewer followers.
It may have helped that I posted it around the holidays, but I don’t think that warrants waiting very long to do it. Perhaps it could be tied it to another holiday or a birthday? IE “For my birthday I am giving away copies of this book that really means a lot to me.” I know the FB algorithm loves showing birthday posts.
I think it’s possible that posting it as an image (the cover of the book) helped it gain traction on FB, but I don’t have any concrete evidence of this.
After posting it, I also commented on my own post, to offer to send a couple articles about EA to anyone who didn’t want to commit to a book—two people took me up on that.
To get traction on FB, you might consider private messaging a couple close/EA friends soon after posting, and asking them to like it and comment on it
I’d recommend making sure you have the time to respond, and or books over the following couple of days. It doesn’t take that long, but probably best not to do it right before starting a big project, or going on vacation or something.
Responding
This probably goes without saying, but make sure you check for replies frequently for a few days after making them.
Private message anyone who responds, and get their mailing address.
Even if you’ve private messaged them already, consider replying to friends’ comments on the post as well. That should help the post get more traction.
Keeping Track
Here’s a blank version of the Google sheet I used to track things. To use it, choose File > Make a Copy, to save it to your own account.
It contains fields for:
Name and address for sending the book.
Method of contact (FB/IG/TW, etc.), so you remember how to send them the survey.
Date the book was sent, so you know when to send the survey.
Cost of the book/shipping, for calculating cost-effectiveness.
Cost of your time spent—This probably doesn’t apply to everyone, but, as a freelancer, it’s pretty easy for me to put a dollar value on my time (IE what I could have earned-to-give in that amount of time). If you can do that, I think it’s good, but I wouldn’t bother if you are doing it in your spare time, when you wouldn’t otherwise be working.
Ordering Books
For domestic US addresses, I used Amazon.com. It cost a little less than $12 per book with Amazon Prime free shipping.
For International addresses, I found that Amazon logins seem to work universally for all countries’ Amazon sites—your contact and payment information should already be in there. Not every country has a site, so you may have to shop around to find the best deals for international shipping. I was able to send books to the UK and Egypt from Amazon.co.uk, and to Croatia from Amazon.de—all for pretty reasonable rates.
I had Amazon Prime already for my US account, but I signed up for a free 30 day trial on Amazon UK to get free shipping for a local order there—it stays in effect for the full 30 days, even if you cancel immediately, so you don’t have to worry about forgetting to do it later. If you don’t already have Prime, this might be a good way to get free shipping on all the books you order for domestic recipients.
For eBooks, you can buy them to give as a gift to people who live domestically (at least on Amazon US).
Unfortunately, I did not find any way to buy them for someone in another country. If you buy it from Amazon US, they can’t redeem it, and you can’t buy it from their country’s Amazon site if you don’t have a local billing address. It may be possible for you to buy them a gift card, but you’ll have to it on their country’s Amazon site, as they aren’t transferrable. I tried this with Amazon Mexico and, inexplicably, it seemed like they didn’t sell gift cards online (this may not be a problem for other countries). If anyone has a better solution for giving eBooks internationally, please comment (googling didn’t turn up much). I feel compelled to note, despite it being for the benefit of the EA cause, it is not legal to share non-DRM copies of the eBook without paying for them, so please don’t do/suggest that.
Sending Surveys
Four months will likely have passed by the time I send it out. Intuitively, that seems about right—it always takes me a while to get around to reading a new book. I think it’s fine if you want to send it sooner or later than that though (especially since they will have the option to get a reminder if they haven’t read it yet).
If you’d like a reminder to send it out after four months, please say so in a comment on this post, or email me at eabookgiveaway [at] gmail [dot] com.
You don’t need someone’s email address to send them the survey (it asks for it if they agree to a follow up survey, or if they want to be reminded to take it later), so you can just share the link with them via private message or whatever.
You may want to explain in a nice way (again, your own words are probably best) how it is very quick (like 5 minutes) and helps us figure out whether we should recommend giving away books to others.
You may also want to link to it in a comment on your original post, as there may be people who liked the post and bought the book on their own, without ever commenting (one person told me they had done this when I saw them a few months later).
And that should do it! Of course, feedback on all this is very much welcome—especially if you’ve found something else that worked or didn’t. I consider this post a work in progress and will updated it as needed.
Financial Aid
While I think it’s definitely best if it is seen as a personal gift/offer, I understand that this may be difficult for some people to afford. For the moment, I am happy to reimburse for the cost of the books, or order the copies for those who are unable to (this will come out of my personal donation budget, likely otherwise headed for GiveWell unrestricted). Email me at eabookgiveaway [at] gmail [dot] com to coordinate.
Evaluating and Publishing Results
Assuming the survey approach described above, there may not be much to report initially. I will probably wait until the first follow up surveys are completed (Fall 2020?) to write a full report—unless results are particularly remarkable before then.
I will publish the results to the EA forum, and, if it turns out to be cost-effective, I will build a website (updated regularly) for them as well, where I will also post most of the information contained herein. (As my day job is web developer, this would be cheap/easy for me to create and maintain.)
Feedback Requested
I would love feedback/comments on any of the above—but specifically:
The survey questions/approach
The instructions—specifically clarity and, after the fact, things that worked/didn’t work
Information about how to buy an eBook for someone in another country
This post itself (longtime reader...)
Fantastic post, Jeremy! I’m a bit biased, since I had the chance to see earlier drafts, but I really like the generous spirit of this initiative, and it seems like a low-risk, high-potential way to grow the community. It’s very kind of you to offer funding to others who want to try their own giveaways.
In fact, I might just try this myself come Giving Season; I’ve set a reminder in my calendar to think about it on November 15th. Thanks for the idea.
Regarding the survey: Consider changing the wording on question #9:
I’d remove the section in bold. If people are really interested in EA, they’ll hopefully give you contact information either way; if they’re on the fence, they might feel a bit objectified being referred to as sources of impact, or guilty about donating once and planning not to do so in the future (I can imagine giving $100 to GiveWell, then seeing the survey and losing my warm glow because I haven’t had “the bulk of my impact”).
This is a highly speculative suggestion, though, and I don’t think it makes a big difference either way.
Thanks Aaron. Glad to hear you might try it.
Good point about that survey question. I was really trying to get across the importance of tracking future actions, but I agree that it could come across off-putting. What do you think of this?
That version does sound better. One more suggested version:
Thank you for taking the time to share what you’ve done. Since we also asked about your future plans, could we follow up with one more short survey a year from now, to see what happened?
If that’s alright with you, please enter your email address below—it will not be shared with anyone, or used for any other purpose.
I’m hoping this feels a bit less high-pressure than “what you may still do”, but you could also remove “to see what happened” to help with that.
I see what you’re saying, but I (intentionally) didn’t ask about future plans so I’m not sure if that part works. While I definitely don’t want it to be high pressure, I do want to somehow emphasize the importance of tracking future actions. I wonder if this strikes a balance.
I really appreciate you taking the time to back-and-forth with me about this!
I like that wording, and don’t have any changes to suggest.
Having done a little bit of graduate work on survey design, I’ll put on my survey-design hat and offer a bunch of suggestions. If you’d like to discuss further, please feel free to reach out directly on—twitter / @gmail.com / calendly—my username is my full name.
Before getting to the question, a few points on sample design and research ethics:
1) You might want to provide distinct links to the survey for each subset of answerers, i.e. those who you gave the book and are asking directly, or those who each other person has done so for. You may also want to link to a different survey to ask the secondary people who borrowed the book. This allows you to later consider if there are differences in effectiveness.
2) You should clarify that the data will be analyzed and shared, but will first have any personally identifying information removed. (i.e. don’t share the raw results with email addresses with ANYONE.)
3) Best-best practice would be to pre-register your analysis plan.
On to the questions! I have a bunch of language and structure nitpicks I’d suggest changing:
re: 1 - “I have’t read/finished it yet, but I plan to,” I think that’s ambiguous—people might have stopped reading, but still be impacted. I’d change to the following options:
I read it
I haven’t begun reading it yet—Please skip to the final 2 questions
I just began reading it and plan to finish—Please skip to the final 2 questions
I have read a substantial part of the book, but didn’t finish it.
I do not plan on reading it.
(Also, don’t ask for an email address here, instead say “If you have made any changes to your life based on this, please continue. ” People are more likely to feel willing to provide extra information as they move later in the survey
Relatedly, re: “2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? If so, please list each charity, with US dollar amount, on its own line.”
I’m concerned that people will find the wording or the request pushy / invasive and refuse to answer or stop the survey completely. I’d suggest splitting this into a gentler 3-part version: (Again, people who start complying are more likely to continue filling out more-invasive questions.)
2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? Yes/No
2a. If so, we would be interested in evaluating how effective giving away the book was, and would like to know the total amount you have donated. (# Box)
2b. If you are willing to provide further details please list the charities and US dollar amounts, on separate lines. (Textbox)
For the remaining questions, I’d also recommend refactoring and making the flow better;
3. Have you considered made any other life changes since reading the book? Yes / No
(If not, please skip to question 7.)
3a. If so, did you change or consider changing your: Dietary choices (See Q4) / Career Plans (See Q5) / Other (See Q6)? (Checkboxes)
4. If you changed your Dietary choices, have you become Vegan (4a) / Vegetarian (4a) / Reduced meat consumption (4b) / Other (4c) / Considered this but have not (yet) changed anything (4c)?
4a. How many months ago?
4b. How much less eggs or meat?
4c. What else did you change / consider changing?
5. Have you made or considered making career changes?(Current Q6 options)
5a. Details (text)
6. Other changes (text)
7. Put question 10 here, before asking about the follow up.
Also: Give them the option to provide email addresses instead of asking their friend themselves, if they think it would be less intrusive. Note that the email will be short, and the email addresses will not be kept beyond sending a single email asking them to take the survey. (Note: I ALSO think people are a bit more likely to send the request themselves if given this option as an alternative.)
8. Then, Q9 / If you would be willing to take another follow-up survey, please enter… (Make language include either the “haven’t read / finished yet” AND asking for a follow-up—you will know which is which based on the above.)
Thanks David! Great suggestions. I am just back from a trip, but I will dig in more deeply this week and make some revisions and reply to you in more depth—either here or via email. Thanks again!
You may want to consider another cheaper and easier way to test a book giveaway, that will have some built in measurement/follow-up processes. As I mentioned on another thread, there’s going to be
If you’re interested in helping distribute free copies, please let me know!
It’s good to see people trying small low cost experiments like this although I suspect there are diminishing returns to how many people will be interested if you try a similar post again but that’s also why it’s good that you’ve given a framework for others to copy.
At EA London I haven’t given as many books out in the last 12 months as the previous 12 as I’ve usually only given them to people after a 1-1 meeting or significant interest from someone at an event where it seems more likely that they’ll actually read it and want to take away some advice rather than offering to give it to people who are more interested in the idea of a “free book”.
Yes, I had considered doing another post, but I guess I was thinking probably not. It would certainly be diminishing returns, yeah. I, and I’m sure many others, wouldn’t want to start seeming spammy about it. Maybe after a couple of years.
It’s an interesting point. Do you know who you’ve given it to, and would you consider sending them a separate copy of the survey once it’s been released, to gauge effectiveness / relative effectiveness?
@Davidmanheim I am not sure if they are still doing it, but I know they were doing a survey. It is linked to in the Google Doc of their summary that I linked to above.
Awesome post.
Suggestion: I have found in person feedback to useful alongside surveys. Suggest making a bit of effort to talk to people in person, especially if it is friends you see anyway, and including this data into a final impact estimate.
Good idea. I will definitely do that.