Thanks for your feedback! It’s very useful for us to receive public feedback about what questions are most valued by the community.
Your concerns seem entirely reasonable. Unfortunately, we face a lot of tough choices where not dropping any particular question means having to drop others instead. (And many people think that the survey is too long anyway implying that perhaps we should cut more questions as well.)
I think running these particular questions every other year (rather than cutting them outright) may have the potential to provide much of the value of including them every year, given that historically the numbers have not changed significantly across years. I would be less inclined to think this if we could perform additional analyses with these variables (e.g. to see whether people with different politics have lower NPS scores), but unfortunately with only ~3% of respondents being right-of-centre, there’s a limit to how much we can do with the variable. (This doesn’t apply to the diet measure which actually was informative in some of our models.)
Thanks for the suggestion. We have considered it and might implement it in future years for some questions. For a lot of variables, I think we’d rather have most data from almost all respondents every other year, than data from half of respondents every year. This is particularly so for those variables which we want to use in analyses combined with other variables, but applies less in the case of variables like politics where we can’t really do that.
Thanks for your feedback! It’s very useful for us to receive public feedback about what questions are most valued by the community.
Your concerns seem entirely reasonable. Unfortunately, we face a lot of tough choices where not dropping any particular question means having to drop others instead. (And many people think that the survey is too long anyway implying that perhaps we should cut more questions as well.)
I think running these particular questions every other year (rather than cutting them outright) may have the potential to provide much of the value of including them every year, given that historically the numbers have not changed significantly across years. I would be less inclined to think this if we could perform additional analyses with these variables (e.g. to see whether people with different politics have lower NPS scores), but unfortunately with only ~3% of respondents being right-of-centre, there’s a limit to how much we can do with the variable. (This doesn’t apply to the diet measure which actually was informative in some of our models.)
Have you considered running different question sets to different people (randomly assigned)?
It could expand the range of questions you can ask.
Thanks for the suggestion. We have considered it and might implement it in future years for some questions. For a lot of variables, I think we’d rather have most data from almost all respondents every other year, than data from half of respondents every year. This is particularly so for those variables which we want to use in analyses combined with other variables, but applies less in the case of variables like politics where we can’t really do that.
Ah yes, that makes sense and I hadn’t thought of that