I wonder if it is possible to add a check for this in a future survey. Maybe ask them if they intentionally conserve their water usage to save the environment? There should be no reason for that to change from pre- to post- without a change in social desirability or attrition.
My guess is the main problem occurs when it is very clear to students what the instructors want them to say. Since we don’t talk about water usage, students may not change their answer for the water usage question, but still change their answer to the questions relating to the content (whether or not they have been impacted by the program), so there may not be a shift in the water question, but still a shift in the other questions.
We did test out some different social desirability scales in the surveys, which is a common method. The preliminary analysis suggested that social desirability is a factor, but we haven’t finished that analysis on the full data yet.
I wonder if it is possible to add a check for this in a future survey. Maybe ask them if they intentionally conserve their water usage to save the environment? There should be no reason for that to change from pre- to post- without a change in social desirability or attrition.
My guess is the main problem occurs when it is very clear to students what the instructors want them to say. Since we don’t talk about water usage, students may not change their answer for the water usage question, but still change their answer to the questions relating to the content (whether or not they have been impacted by the program), so there may not be a shift in the water question, but still a shift in the other questions.
We did test out some different social desirability scales in the surveys, which is a common method. The preliminary analysis suggested that social desirability is a factor, but we haven’t finished that analysis on the full data yet.