Having done a little bit of graduate work on survey design, I’ll put on my survey-design hat and offer a bunch of suggestions. If you’d like to discuss further, please feel free to reach out directly on—twitter / @gmail.com / calendly—my username is my full name.
Before getting to the question, a few points on sample design and research ethics:
1) You might want to provide distinct links to the survey for each subset of answerers, i.e. those who you gave the book and are asking directly, or those who each other person has done so for. You may also want to link to a different survey to ask the secondary people who borrowed the book. This allows you to later consider if there are differences in effectiveness.
2) You should clarify that the data will be analyzed and shared, but will first have any personally identifying information removed. (i.e. don’t share the raw results with email addresses with ANYONE.)
3) Best-best practice would be to pre-register your analysis plan.
On to the questions! I have a bunch of language and structure nitpicks I’d suggest changing:
re: 1 - “I have’t read/finished it yet, but I plan to,” I think that’s ambiguous—people might have stopped reading, but still be impacted. I’d change to the following options:
I read it
I haven’t begun reading it yet—Please skip to the final 2 questions
I just began reading it and plan to finish—Please skip to the final 2 questions
I have read a substantial part of the book, but didn’t finish it.
I do not plan on reading it.
(Also, don’t ask for an email address here, instead say “If you have made any changes to your life based on this, please continue. ” People are more likely to feel willing to provide extra information as they move later in the survey
Relatedly, re: “2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? If so, please list each charity, with US dollar amount, on its own line.”
I’m concerned that people will find the wording or the request pushy / invasive and refuse to answer or stop the survey completely. I’d suggest splitting this into a gentler 3-part version: (Again, people who start complying are more likely to continue filling out more-invasive questions.)
2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? Yes/No
2a. If so, we would be interested in evaluating how effective giving away the book was, and would like to know the total amount you have donated. (# Box)
2b. If you are willing to provide further details please list the charities and US dollar amounts, on separate lines. (Textbox)
For the remaining questions, I’d also recommend refactoring and making the flow better;
3. Have you considered made any other life changes since reading the book? Yes / No
(If not, please skip to question 7.)
3a. If so, did you change or consider changing your: Dietary choices (See Q4) / Career Plans (See Q5) / Other (See Q6)? (Checkboxes)
4. If you changed your Dietary choices, have you become Vegan (4a) / Vegetarian (4a) / Reduced meat consumption (4b) / Other (4c) / Considered this but have not (yet) changed anything (4c)?
4a. How many months ago?
4b. How much less eggs or meat?
4c. What else did you change / consider changing?
5. Have you made or considered making career changes?(Current Q6 options)
5a. Details (text)
6. Other changes (text)
7. Put question 10 here, before asking about the follow up.
Also: Give them the option to provide email addresses instead of asking their friend themselves, if they think it would be less intrusive. Note that the email will be short, and the email addresses will not be kept beyond sending a single email asking them to take the survey. (Note: I ALSO think people are a bit more likely to send the request themselves if given this option as an alternative.)
8. Then, Q9 / If you would be willing to take another follow-up survey, please enter… (Make language include either the “haven’t read / finished yet” AND asking for a follow-up—you will know which is which based on the above.)
Thanks David! Great suggestions. I am just back from a trip, but I will dig in more deeply this week and make some revisions and reply to you in more depth—either here or via email. Thanks again!
Having done a little bit of graduate work on survey design, I’ll put on my survey-design hat and offer a bunch of suggestions. If you’d like to discuss further, please feel free to reach out directly on—twitter / @gmail.com / calendly—my username is my full name.
Before getting to the question, a few points on sample design and research ethics:
1) You might want to provide distinct links to the survey for each subset of answerers, i.e. those who you gave the book and are asking directly, or those who each other person has done so for. You may also want to link to a different survey to ask the secondary people who borrowed the book. This allows you to later consider if there are differences in effectiveness.
2) You should clarify that the data will be analyzed and shared, but will first have any personally identifying information removed. (i.e. don’t share the raw results with email addresses with ANYONE.)
3) Best-best practice would be to pre-register your analysis plan.
On to the questions! I have a bunch of language and structure nitpicks I’d suggest changing:
re: 1 - “I have’t read/finished it yet, but I plan to,” I think that’s ambiguous—people might have stopped reading, but still be impacted. I’d change to the following options:
I read it
I haven’t begun reading it yet—Please skip to the final 2 questions
I just began reading it and plan to finish—Please skip to the final 2 questions
I have read a substantial part of the book, but didn’t finish it.
I do not plan on reading it.
(Also, don’t ask for an email address here, instead say “If you have made any changes to your life based on this, please continue. ” People are more likely to feel willing to provide extra information as they move later in the survey
Relatedly, re: “2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? If so, please list each charity, with US dollar amount, on its own line.”
I’m concerned that people will find the wording or the request pushy / invasive and refuse to answer or stop the survey completely. I’d suggest splitting this into a gentler 3-part version: (Again, people who start complying are more likely to continue filling out more-invasive questions.)
2. Since reading the book, have you donated to any effective charities, that you wouldn’t have otherwise? Yes/No
2a. If so, we would be interested in evaluating how effective giving away the book was, and would like to know the total amount you have donated. (# Box)
2b. If you are willing to provide further details please list the charities and US dollar amounts, on separate lines. (Textbox)
For the remaining questions, I’d also recommend refactoring and making the flow better;
3. Have you considered made any other life changes since reading the book? Yes / No
(If not, please skip to question 7.)
3a. If so, did you change or consider changing your: Dietary choices (See Q4) / Career Plans (See Q5) / Other (See Q6)? (Checkboxes)
4. If you changed your Dietary choices, have you become Vegan (4a) / Vegetarian (4a) / Reduced meat consumption (4b) / Other (4c) / Considered this but have not (yet) changed anything (4c)?
4a. How many months ago?
4b. How much less eggs or meat?
4c. What else did you change / consider changing?
5. Have you made or considered making career changes?(Current Q6 options)
5a. Details (text)
6. Other changes (text)
7. Put question 10 here, before asking about the follow up.
Also: Give them the option to provide email addresses instead of asking their friend themselves, if they think it would be less intrusive. Note that the email will be short, and the email addresses will not be kept beyond sending a single email asking them to take the survey. (Note: I ALSO think people are a bit more likely to send the request themselves if given this option as an alternative.)
8. Then, Q9 / If you would be willing to take another follow-up survey, please enter… (Make language include either the “haven’t read / finished yet” AND asking for a follow-up—you will know which is which based on the above.)
Thanks David! Great suggestions. I am just back from a trip, but I will dig in more deeply this week and make some revisions and reply to you in more depth—either here or via email. Thanks again!