How should we think about the 17% response rate to this survey? Is it possible that researchers who are more concerned about alignment are also more likely to complete the survey?
I dream of getting a couple questions added onto a big conference’s attendee application form. But probably not possible unless you’re incredibly well-connected.
Possible, but likely a smaller effect than you might think because:
a) I was very ambiguous about the subject matter until they were taking the survey (e.g. did not mention AGI or risk or timelines)
b) Last time (for the 2016 survey) we checked the demographics of respondents against those for a random subset of non-respondents, and they weren’t very different.
Participants were also mostly offered substantial payment for taking the survey ($50 usually, for a ~15m survey), in part in the hope of making payment a larger motivator than desire to express some particular view, but I don’t think payment actually made a large difference to the response rate, so probably failed have the desired effect on possible response bias.
How should we think about the 17% response rate to this survey? Is it possible that researchers who are more concerned about alignment are also more likely to complete the survey?
I’d guess very slightly, yes. I’m not aware of easy ways to get much evidence, unfortunately.
I dream of getting a couple questions added onto a big conference’s attendee application form. But probably not possible unless you’re incredibly well-connected.
Possible, but likely a smaller effect than you might think because: a) I was very ambiguous about the subject matter until they were taking the survey (e.g. did not mention AGI or risk or timelines) b) Last time (for the 2016 survey) we checked the demographics of respondents against those for a random subset of non-respondents, and they weren’t very different.
Participants were also mostly offered substantial payment for taking the survey ($50 usually, for a ~15m survey), in part in the hope of making payment a larger motivator than desire to express some particular view, but I don’t think payment actually made a large difference to the response rate, so probably failed have the desired effect on possible response bias.