Thanks for pointing out both kinds of biases. These biases can cause a failure of comparability. Concretely, if an intervention causes you to give counterfactually higher scores as a matter of ‘courtesy’ to the researcher, then the intervention changed the meaning of each given response category.
I therefore take it that you don’t think that our particular tests of comparability will cover the two biases you mention. If so, I agree. However, my colleague has given reasons for why we might not be as worried about these sorts of biases.
I don’t think this can be tested in our current survey format, but it might be testable in a different design. We are open to suggestions!
Not only courtesy, but also future hope (which I think may be more important here).
Yeah it’s really hard to test. I think validity of point estimates are pretty reasonable for wellbeing surveys and I agree with most of the reasoning on this post.
It’s very had to test those biases ethically, but probably possible. Not in this kind of survey anyway.
The reasons he gave for not being worried about those biases were not unreasonable, but based on flimsy evidence. Especially future hope bias which may not have been researched at all.
Hi Nick,
Thanks for pointing out both kinds of biases. These biases can cause a failure of comparability. Concretely, if an intervention causes you to give counterfactually higher scores as a matter of ‘courtesy’ to the researcher, then the intervention changed the meaning of each given response category.
I therefore take it that you don’t think that our particular tests of comparability will cover the two biases you mention. If so, I agree. However, my colleague has given reasons for why we might not be as worried about these sorts of biases.
I don’t think this can be tested in our current survey format, but it might be testable in a different design. We are open to suggestions!
Not only courtesy, but also future hope (which I think may be more important here).
Yeah it’s really hard to test. I think validity of point estimates are pretty reasonable for wellbeing surveys and I agree with most of the reasoning on this post.
It’s very had to test those biases ethically, but probably possible. Not in this kind of survey anyway.
The reasons he gave for not being worried about those biases were not unreasonable, but based on flimsy evidence. Especially future hope bias which may not have been researched at all.