This all seems reasonable to me though I haven’t thought much about my overall take.
I think the details matter a lot for “Even among individual researchers who work independently, or whose org isn’t running surveys, probably relatively few should run their own, relatively publicly advertised individual surveys”
A lot of people might get a lot of the value from a fairly small number of responses, which would minimise costs and negative externalities. I even think it’s often possible to close a survey after a certain number of responses.
A counterargument is that the people who respond earliest might be unrepresentative. But for a lot of purposes, it’s not obvious to me you need a representative sample. “Among the people who are making the most use of my research, how is it useful” can be pretty informative on its own.
A lot of people might get a lot of the value from a fairly small number of responses, which would minimise costs and negative externalities.
Agreed.
This sort of thing is part of why I wrote “relatively publicly advertised”, and added “And maybe it doesn’t hold for surveys sent out in a more targeted manner.” But good point that someone could run a relatively publicly advertised survey and then just close it after a small-ish number of responses; I hadn’t considered that option.
This all seems reasonable to me though I haven’t thought much about my overall take.
I think the details matter a lot for “Even among individual researchers who work independently, or whose org isn’t running surveys, probably relatively few should run their own, relatively publicly advertised individual surveys”
A lot of people might get a lot of the value from a fairly small number of responses, which would minimise costs and negative externalities. I even think it’s often possible to close a survey after a certain number of responses.
A counterargument is that the people who respond earliest might be unrepresentative. But for a lot of purposes, it’s not obvious to me you need a representative sample. “Among the people who are making the most use of my research, how is it useful” can be pretty informative on its own.
Agreed.
This sort of thing is part of why I wrote “relatively publicly advertised”, and added “And maybe it doesn’t hold for surveys sent out in a more targeted manner.” But good point that someone could run a relatively publicly advertised survey and then just close it after a small-ish number of responses; I hadn’t considered that option.