This post is getting some significant downvotes. I would be interested if someone who has downvoted could explain the reason for that.
Thereâs plenty of room for disagreement on how serious a mistake this is, whether it has introduced a âframingâ bias into other results or not, and what it means for the report as a whole. But it just seems straightforwardly true that this particular question is phrased extremely poorly (it seems disingenuous to suggest that the question using the phrasing âbest matchingâ covers you for not even attempting to include the full range of possibilities in your list).
I assume that people downvoting are objecting to the way that this post is using this mistake to call the entire report into question, with language like âmajor flawâ. They may have a point there. But I think you should have a very high bar for downvoting someone who is politely highlighting a legitimate mistake in a piece of research.
âDisagreeâ react to the âmajor flawâ language if you like, and certainly comment your disagreements, but silently downvoting someone for finding a legitimate methodological problem in some EA research seems like bad EA forum behaviour to me!
Yes, please do not downvote Yarrowâs post just because itâs style is a bit abrasive, and it goes against EA consensus. She has changed my mind quite a lot, as the person who kicked off the dispute, and Connacher who worked on the survey is clearly taking her criticisms seriously.
Thank you very much. I really appreciate your helpful and cooperative approach.
The âbest matchingâ wording of the question doesnât, in my view, change the underlying problem of presenting these as the only three options.
Itâs also a problem, in my view, that the âbest matchingâ wording is dropped on page 38 and the report simply talks about the probability respondents assign to the scenario. I looked at the report in the first place because a Forecasting Research Institute employee just said (on the EA Forum) what the probability assigned to a scenario was, and didnât mention the âbest matchingâ wording (or the three-scenario framing). If you include âbest matchingâ in the question and then drop it when you present the results, what was the point of saying âbest matchingâ in the first place?
[Edited on 2025-11-14 at 6:32 PM Eastern to add: The Forecasting Research Institute also presented the results as expertsâ probabilities for these scenarios in a post on the EA Forum. See Edit #2 added to the post above.]
I didnât intend for the post to come across as more than a criticism of this specific question in the survey â I said that the report contains many questions and said âIâve only looked at the report briefly and there is a lot that could be examined and discussedâ. I meant the title literally and factually. This is a major flaw that I came across in the report.
I would be happy to change the title of the post or change the wording of the post if someone can suggest a better alternative.
If people have qualms with either the tone or the substance of the post, Iâd certainly like to hear them. So, I encourage people to comment.
This post is getting some significant downvotes. I would be interested if someone who has downvoted could explain the reason for that.
Thereâs plenty of room for disagreement on how serious a mistake this is, whether it has introduced a âframingâ bias into other results or not, and what it means for the report as a whole. But it just seems straightforwardly true that this particular question is phrased extremely poorly (it seems disingenuous to suggest that the question using the phrasing âbest matchingâ covers you for not even attempting to include the full range of possibilities in your list).
I assume that people downvoting are objecting to the way that this post is using this mistake to call the entire report into question, with language like âmajor flawâ. They may have a point there. But I think you should have a very high bar for downvoting someone who is politely highlighting a legitimate mistake in a piece of research.
âDisagreeâ react to the âmajor flawâ language if you like, and certainly comment your disagreements, but silently downvoting someone for finding a legitimate methodological problem in some EA research seems like bad EA forum behaviour to me!
Yes, please do not downvote Yarrowâs post just because itâs style is a bit abrasive, and it goes against EA consensus. She has changed my mind quite a lot, as the person who kicked off the dispute, and Connacher who worked on the survey is clearly taking her criticisms seriously.
God bless!
Thank you very much. I really appreciate your helpful and cooperative approach.
The âbest matchingâ wording of the question doesnât, in my view, change the underlying problem of presenting these as the only three options.
Itâs also a problem, in my view, that the âbest matchingâ wording is dropped on page 38 and the report simply talks about the probability respondents assign to the scenario. I looked at the report in the first place because a Forecasting Research Institute employee just said (on the EA Forum) what the probability assigned to a scenario was, and didnât mention the âbest matchingâ wording (or the three-scenario framing). If you include âbest matchingâ in the question and then drop it when you present the results, what was the point of saying âbest matchingâ in the first place?
[Edited on 2025-11-14 at 6:32 PM Eastern to add: The Forecasting Research Institute also presented the results as expertsâ probabilities for these scenarios in a post on the EA Forum. See Edit #2 added to the post above.]
I didnât intend for the post to come across as more than a criticism of this specific question in the survey â I said that the report contains many questions and said âIâve only looked at the report briefly and there is a lot that could be examined and discussedâ. I meant the title literally and factually. This is a major flaw that I came across in the report.
I would be happy to change the title of the post or change the wording of the post if someone can suggest a better alternative.
If people have qualms with either the tone or the substance of the post, Iâd certainly like to hear them. So, I encourage people to comment.