I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
There are also tradeoffs with viewpoints.xyz being accessible (=better sampling) and the data being rich enough. Limitation to short answer stems with a binary response (plus an ambiguous “skip”) lends itself to identifying two major “camps” more easily that clusters within those camps. In general, expanding to five-point Likert scales would help, as would some sort of branching.
For example, I’d want to know—conditional on “Manifest did wrong here” / “the platforming was inappropriate”—what factors were more or less important to the respondent’s judgment. On a 1-5 scale, how important do you find [your view that the organizers did not distance themselves from the problematic viewpoints / the fit between the problematic viewpoints and a conference for the forecasting community / an absence of evidence that special guests with far-left or at least mainstream viewpoints on the topic were solicited / whatever]. And: how much would the following facts or considerations, if true, change your response to a hypothetical situation like the Manifest conference? Again, you can’t get how much on a binary response.
Maybe all that points out to polling being more of a post-dialogue event, and accepting that we would choose discussants based on past history & early reactions. For example, I would have moderately high confidence that user X would represent a stance close to a particular pole on most issues, while I would represent a stance that codes as “~ moderately progressive by EA Forum standards.”
I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
Often it feels like I can never please people on this forum. I think the poll is significantly better than no poll.
I think the poll is significantly better than no poll.
Yeah, I agree with that! I don’t find it inconsistent with the idea that the reasonable trade-offs you made between various characteristics in the data-collection process make the data you got not a good match for the purposes I would like data for. They aregood data for people interested in the answer to certain other questions. No one can build a (practical) poll for all possible use cases, just as no one can build a (reasonably priced) car that is both very energy-efficient and has major towing/hauling chops.
As useful as viewpoints.xyz is, I will mention that for maybe 50% or 60% of the questions, my reaction was “it depends.” I suppose you can’t really get around that unless the person creating the questions spends much more time to carefully craft them (which sort of defeats the purpose of a quick-and-dirty poll), or unless you do interviews (which are of course much more costly). I do think there is value in the quick-and-dirty MVP version, but it’s usefullness has a pretty noticable upper bound.
I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
There are also tradeoffs with viewpoints.xyz being accessible (=better sampling) and the data being rich enough. Limitation to short answer stems with a binary response (plus an ambiguous “skip”) lends itself to identifying two major “camps” more easily that clusters within those camps. In general, expanding to five-point Likert scales would help, as would some sort of branching.
For example, I’d want to know—conditional on “Manifest did wrong here” / “the platforming was inappropriate”—what factors were more or less important to the respondent’s judgment. On a 1-5 scale, how important do you find [your view that the organizers did not distance themselves from the problematic viewpoints / the fit between the problematic viewpoints and a conference for the forecasting community / an absence of evidence that special guests with far-left or at least mainstream viewpoints on the topic were solicited / whatever]. And: how much would the following facts or considerations, if true, change your response to a hypothetical situation like the Manifest conference? Again, you can’t get how much on a binary response.
Maybe all that points out to polling being more of a post-dialogue event, and accepting that we would choose discussants based on past history & early reactions. For example, I would have moderately high confidence that user X would represent a stance close to a particular pole on most issues, while I would represent a stance that codes as “~ moderately progressive by EA Forum standards.”
Often it feels like I can never please people on this forum. I think the poll is significantly better than no poll.
Yeah, I agree with that! I don’t find it inconsistent with the idea that the reasonable trade-offs you made between various characteristics in the data-collection process make the data you got not a good match for the purposes I would like data for. They aregood data for people interested in the answer to certain other questions. No one can build a (practical) poll for all possible use cases, just as no one can build a (reasonably priced) car that is both very energy-efficient and has major towing/hauling chops.
As useful as viewpoints.xyz is, I will mention that for maybe 50% or 60% of the questions, my reaction was “it depends.” I suppose you can’t really get around that unless the person creating the questions spends much more time to carefully craft them (which sort of defeats the purpose of a quick-and-dirty poll), or unless you do interviews (which are of course much more costly). I do think there is value in the quick-and-dirty MVP version, but it’s usefullness has a pretty noticable upper bound.