Thanks for this, it was useful advice even for me as someone who has done quite a few surveys.
I agree with the suggestions and with most of what DM has said to follow up.
Probably, the most generalisable lesson we learned is more meta: if you’re thinking of running a large scale survey, you should seriously consider having at least one person on your team who has experience with surveys.
I definitely agree with this. An approach that we use at READI is to try to involve several ‘topic’ and ‘methodological’ experts in every research project to minimise the risk of errors and oversights. We share a protocol/plan with them before we preregister it so that they can assess materials. Then we engage them again at analysis and write up so they can check that we executed the plan appropriately.
It generally doesn’t take much of their time (maybe 2-20 hours each over a project—we should have a good estimate of the exact time taken in the future once we have had more projects complete (i.e., end in publication)).
In my experience, such experts are also are relatively happy with the pay-off of a publication or acknowledgement for a few hours of work so it is a relatively fair and sustainable exchange (again, we will soon have better data on that).
Finally, this probably won’t be useful for most readers but just in case, these resources (a free chapter/ and two related blog posts (part 1/ part 2)) provide a lot of introductory advice for doing ‘audience research’.
I’d also suggest the Sage series on research methods as a good resource for non-experts who want at least a basic level of understanding of what to do. In this case, Fowler’s “Survey Research Methods” would have provided most of these insights without trial and error—it’s around 150 pages, but it’s not heavy reading.
Thanks for this, it was useful advice even for me as someone who has done quite a few surveys.
I agree with the suggestions and with most of what DM has said to follow up.
I definitely agree with this. An approach that we use at READI is to try to involve several ‘topic’ and ‘methodological’ experts in every research project to minimise the risk of errors and oversights. We share a protocol/plan with them before we preregister it so that they can assess materials. Then we engage them again at analysis and write up so they can check that we executed the plan appropriately.
It generally doesn’t take much of their time (maybe 2-20 hours each over a project—we should have a good estimate of the exact time taken in the future once we have had more projects complete (i.e., end in publication)).
In my experience, such experts are also are relatively happy with the pay-off of a publication or acknowledgement for a few hours of work so it is a relatively fair and sustainable exchange (again, we will soon have better data on that).
Finally, this probably won’t be useful for most readers but just in case, these resources (a free chapter/ and two related blog posts (part 1/ part 2)) provide a lot of introductory advice for doing ‘audience research’.
I’d also suggest the Sage series on research methods as a good resource for non-experts who want at least a basic level of understanding of what to do. In this case, Fowler’s “Survey Research Methods” would have provided most of these insights without trial and error—it’s around 150 pages, but it’s not heavy reading.