we know it included people who said they not heard of the term.
People will say anything on surveys. Many respondents go through clicking randomly. You can write a question that says, “Please answer C,” and >10% of respondents will still click something other than C.
This year it might be worth including a mandatory questions saying something like “Check C to promise not to go through clicking randomly”, both as a test and a reminder.
I regularly do this when designing consumer surveys as part of m professional work—the concern in those instances is that respondents are mainly completing the survey for a small monetary reward and so are incentivised to click through as fast as possible. To help my own survey development skills, I participate in several online panels and can confirm that whilst not exactly standard practice, a non-negligible proportion of online consumer surveys will include questions like this used to screen out respondents who are not paying attention.
This is less of a concern for the EA survey, but is almost costless to include such a screening question so seems like an easy way to help validate any resulting analysis or conclusions.
People will say anything on surveys. Many respondents go through clicking randomly. You can write a question that says, “Please answer C,” and >10% of respondents will still click something other than C.
This year it might be worth including a mandatory questions saying something like “Check C to promise not to go through clicking randomly”, both as a test and a reminder.
I regularly do this when designing consumer surveys as part of m professional work—the concern in those instances is that respondents are mainly completing the survey for a small monetary reward and so are incentivised to click through as fast as possible. To help my own survey development skills, I participate in several online panels and can confirm that whilst not exactly standard practice, a non-negligible proportion of online consumer surveys will include questions like this used to screen out respondents who are not paying attention.
This is less of a concern for the EA survey, but is almost costless to include such a screening question so seems like an easy way to help validate any resulting analysis or conclusions.