I have no right to ask for this, but for comment sections that really get into the weeds technically (and on issues one would expect to be action-relevant for other members of the community), it would be great to have a fairly short, neutral, accessible writeup once the conversation has died down. I suspect there are a number of readers whose statistical background and abilities are not significantly better than mine (I have some graduate research training in sociology, but it was nearly half a lifetime ago). On the other hand, it’s not reasonable to ask commenters to write their technically-oriented comments in a way that is accessible to people like me.
At present, I think those of us with less technical sophistication are left with something like “There are issues with HLI’s methodology, but the extent to which those issues materially affect the bottom line is a subject of disagreement.” Maybe that’s all that could be said neutrally anyway, and people like me just have to read the comments and draw what conclusions we can?
I think that we can all agree that the analysis was done in an atypical way (perhaps for good reason), that it was not as rigorous as many people expected, and that it had a series of omissions or made atypical analytical moves that (perhaps inadvertently) made SM look better than it will look once that stuff is addressed. I don’t think anyone can speak yet to the magnitude of the adjustment when the analysis is done better or in a standard way.
But I’d welcome especially Joel’s response to this question. It’s a critical question and it’s worth hearing his take.
I have no right to ask for this, but for comment sections that really get into the weeds technically (and on issues one would expect to be action-relevant for other members of the community), it would be great to have a fairly short, neutral, accessible writeup once the conversation has died down. I suspect there are a number of readers whose statistical background and abilities are not significantly better than mine (I have some graduate research training in sociology, but it was nearly half a lifetime ago). On the other hand, it’s not reasonable to ask commenters to write their technically-oriented comments in a way that is accessible to people like me.
At present, I think those of us with less technical sophistication are left with something like “There are issues with HLI’s methodology, but the extent to which those issues materially affect the bottom line is a subject of disagreement.” Maybe that’s all that could be said neutrally anyway, and people like me just have to read the comments and draw what conclusions we can?
That’s so reasonable.
I think that we can all agree that the analysis was done in an atypical way (perhaps for good reason), that it was not as rigorous as many people expected, and that it had a series of omissions or made atypical analytical moves that (perhaps inadvertently) made SM look better than it will look once that stuff is addressed. I don’t think anyone can speak yet to the magnitude of the adjustment when the analysis is done better or in a standard way.
But I’d welcome especially Joel’s response to this question. It’s a critical question and it’s worth hearing his take.
Fair point! I’ll try to to summarize things from my perspective once things have settled a bit more.