It proved hard to get this version published; the apparent subjectivity of the costs, the inclusion of economic methods in an epidemiology paper, and the specific choice of preference elicitation methods, etc, all exposed a large “attack surface” for reviewers. In the end, we just removed the cost-benefit analysis.
Clearly, internal documents of at least some governments will have estimated these costs. But in almost all cases these were not made public. Even then: as far as we know, only economic costs were counted in these private analyses; it is still rare to see estimates of the large direct disutility of lockdown.
I don’t know exactly which papers you’re referring to, but it’s plausible to me that the cost-benefit analysis would be similarly valuable to the rest of the content in the paper. So it really sucks to just lose it.
Did you end up publishing those calculations elsewhere (e.g. as a blog post complement to the paper, or in a non-peer-reviewed verison of the article)? Do you have any thoughts on whether, when, and how we should try to help people escape the peer review game and just publish useful things outside of journals?
The practical tradeoff was between what, where and when to publish. The first version of the preprint which is on medrxive contains those estimates. Some version with them could probably be published in a much worse journal than Science, and would have much less impact.
We could have published them separately, but a paper is a lot of work, and it’s not clear to me whether, for example, to sacrifice some of the”What we tried”and get this done would have been a good call.
It is possible to escape from the game in specific cases—in the case of covid, for example, the advisory body we created in the Czech Republic was able to take into account analyses based on “internal quality”, especially if it was clear peer review game will take months. If such bodies existed in more/more countries, it would be possible. Similarly, it could be done with the help of an ECDC or WHO type institution.
In general, it’s an “inadequate equilibrium” type of problem, I have some thoughts on typical solutions to them, but not in easily shareable written form, at the moment.
This was a bummer to read:
I don’t know exactly which papers you’re referring to, but it’s plausible to me that the cost-benefit analysis would be similarly valuable to the rest of the content in the paper. So it really sucks to just lose it.
Did you end up publishing those calculations elsewhere (e.g. as a blog post complement to the paper, or in a non-peer-reviewed verison of the article)? Do you have any thoughts on whether, when, and how we should try to help people escape the peer review game and just publish useful things outside of journals?
The practical tradeoff was between what, where and when to publish. The first version of the preprint which is on medrxive contains those estimates. Some version with them could probably be published in a much worse journal than Science, and would have much less impact.
We could have published them separately, but a paper is a lot of work, and it’s not clear to me whether, for example, to sacrifice some of the”What we tried”and get this done would have been a good call.
It is possible to escape from the game in specific cases—in the case of covid, for example, the advisory body we created in the Czech Republic was able to take into account analyses based on “internal quality”, especially if it was clear peer review game will take months. If such bodies existed in more/more countries, it would be possible.
Similarly, it could be done with the help of an ECDC or WHO type institution.
In general, it’s an “inadequate equilibrium” type of problem, I have some thoughts on typical solutions to them, but not in easily shareable written form, at the moment.