One limitation here: you have a view about Manifest. Your interlocutor would have a different view. But how do we know if those views are actually representative of major groupings?
My hunch is that, if equipped with a mind probe, we would find at least two major axes with several meaningfully different viewpoints on each axis. Overall, I’d predict that I would find at least four sizable clusters, probably five to seven.
Beyond that, yes you are likely right, but I don’t know how to have that discussion better. I tried using polls and upvoted quotes as a springboard in this post (Truth-seeking vs Influence-seeking—a narrower discussion) but people didn’t really bite there.
Suggestions welcome.
It is kind of exhausting to keep trying to find ways to get better samples of the discourse, without a sense that people will eventually go “oh yeah this convinces me”. If I were more confident I would have more energy for it.
I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
There are also tradeoffs with viewpoints.xyz being accessible (=better sampling) and the data being rich enough. Limitation to short answer stems with a binary response (plus an ambiguous “skip”) lends itself to identifying two major “camps” more easily that clusters within those camps. In general, expanding to five-point Likert scales would help, as would some sort of branching.
For example, I’d want to know—conditional on “Manifest did wrong here” / “the platforming was inappropriate”—what factors were more or less important to the respondent’s judgment. On a 1-5 scale, how important do you find [your view that the organizers did not distance themselves from the problematic viewpoints / the fit between the problematic viewpoints and a conference for the forecasting community / an absence of evidence that special guests with far-left or at least mainstream viewpoints on the topic were solicited / whatever]. And: how much would the following facts or considerations, if true, change your response to a hypothetical situation like the Manifest conference? Again, you can’t get how much on a binary response.
Maybe all that points out to polling being more of a post-dialogue event, and accepting that we would choose discussants based on past history & early reactions. For example, I would have moderately high confidence that user X would represent a stance close to a particular pole on most issues, while I would represent a stance that codes as “~ moderately progressive by EA Forum standards.”
I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
Often it feels like I can never please people on this forum. I think the poll is significantly better than no poll.
I think the poll is significantly better than no poll.
Yeah, I agree with that! I don’t find it inconsistent with the idea that the reasonable trade-offs you made between various characteristics in the data-collection process make the data you got not a good match for the purposes I would like data for. They aregood data for people interested in the answer to certain other questions. No one can build a (practical) poll for all possible use cases, just as no one can build a (reasonably priced) car that is both very energy-efficient and has major towing/hauling chops.
As useful as viewpoints.xyz is, I will mention that for maybe 50% or 60% of the questions, my reaction was “it depends.” I suppose you can’t really get around that unless the person creating the questions spends much more time to carefully craft them (which sort of defeats the purpose of a quick-and-dirty poll), or unless you do interviews (which are of course much more costly). I do think there is value in the quick-and-dirty MVP version, but it’s usefullness has a pretty noticable upper bound.
One limitation here: you have a view about Manifest. Your interlocutor would have a different view. But how do we know if those views are actually representative of major groupings?
My hunch is that, if equipped with a mind probe, we would find at least two major axes with several meaningfully different viewpoints on each axis. Overall, I’d predict that I would find at least four sizable clusters, probably five to seven.
So I ran a poll with 100 ish respondents and if you want to run the k-means analysis you can find those clusters yourself.
The anonymous data is downloadable here.
https://viewpoints.xyz/polls/ea-and-manifest/results
Beyond that, yes you are likely right, but I don’t know how to have that discussion better. I tried using polls and upvoted quotes as a springboard in this post (Truth-seeking vs Influence-seeking—a narrower discussion) but people didn’t really bite there.
Suggestions welcome.
It is kind of exhausting to keep trying to find ways to get better samples of the discourse, without a sense that people will eventually go “oh yeah this convinces me”. If I were more confident I would have more energy for it.
I don’t think those were most of the questions I was looking for, though. This isn’t a criticism: running the poll early risks missing important cruxes and fault lines that haven’t been found yet; running it late means that much of the discussion has already happened.
There are also tradeoffs with viewpoints.xyz being accessible (=better sampling) and the data being rich enough. Limitation to short answer stems with a binary response (plus an ambiguous “skip”) lends itself to identifying two major “camps” more easily that clusters within those camps. In general, expanding to five-point Likert scales would help, as would some sort of branching.
For example, I’d want to know—conditional on “Manifest did wrong here” / “the platforming was inappropriate”—what factors were more or less important to the respondent’s judgment. On a 1-5 scale, how important do you find [your view that the organizers did not distance themselves from the problematic viewpoints / the fit between the problematic viewpoints and a conference for the forecasting community / an absence of evidence that special guests with far-left or at least mainstream viewpoints on the topic were solicited / whatever]. And: how much would the following facts or considerations, if true, change your response to a hypothetical situation like the Manifest conference? Again, you can’t get how much on a binary response.
Maybe all that points out to polling being more of a post-dialogue event, and accepting that we would choose discussants based on past history & early reactions. For example, I would have moderately high confidence that user X would represent a stance close to a particular pole on most issues, while I would represent a stance that codes as “~ moderately progressive by EA Forum standards.”
Often it feels like I can never please people on this forum. I think the poll is significantly better than no poll.
Yeah, I agree with that! I don’t find it inconsistent with the idea that the reasonable trade-offs you made between various characteristics in the data-collection process make the data you got not a good match for the purposes I would like data for. They aregood data for people interested in the answer to certain other questions. No one can build a (practical) poll for all possible use cases, just as no one can build a (reasonably priced) car that is both very energy-efficient and has major towing/hauling chops.
As useful as viewpoints.xyz is, I will mention that for maybe 50% or 60% of the questions, my reaction was “it depends.” I suppose you can’t really get around that unless the person creating the questions spends much more time to carefully craft them (which sort of defeats the purpose of a quick-and-dirty poll), or unless you do interviews (which are of course much more costly). I do think there is value in the quick-and-dirty MVP version, but it’s usefullness has a pretty noticable upper bound.