I’m skeptical of work like this and what you write in note 2. makes me think of some comments. You write:
I’ve come to think that <100pp one-off papers and blog posts… can’t be thorough or deep enough on their own to be persuasive, or solid enough to use as a foundation for later work.
If I’m honest, this seems to me like a large—and slightly odd—claim. Does it really say that research work that can be written up in fewer than one hundred pages cannot be persuasive and cannot be solid enough to build on? Do you really mean to suggest that?
You go on to write:
Instead, I learn more from unusually detailed and thorough “extended effort” analyses that often require 1-5 FTE years of effort, e.g. OpenAI’s series of “scaling laws” papers. Another example is Saif Khan’s series of papers on semiconductor supply chains and policy options: I couldn’t tell from just one paper, or even the first 5 papers, whether genuine strategic progress was being made or not, but around paper #8 (~2 years in) I could see how most of the key premises and potential defeaters had been investigated pretty thoroughly, and how it all made a pretty robustly-considered (but still uncertain) case for a particular set of policy interventions — culminating roughly in The Semiconductor Supply Chain: Assessing National Competitiveness and Securing Semiconductor Supply Chains.
But I don’t really see how all this is much different from saying something a bit like “more research is better than less research” or that although an individual short paper cannot provide what you call ‘strategic’ insight, it may be gained by essentially synthesizing the results contained in lots of smaller contributions. And this latter description sounds a lot like how I would describe the ‘normal’ way that research papers have worked for ages, i.e. something like the fact that each individual contribution tends to be narrow and then as time goes by, the bigger picture emerges from some kind of aggregation and synthesis that the research community somehow effects. (I find it particularly hard to square the example of Saif Khan’s papers with your overall point: Is it not the case that many of Khan’s earlier outputs were much shorter? But each was high-quality enough that he could build on them, even after having put them out into the community? And then later, once the bigger picture had genuinely emerged for him to see, he drew on his experience to produce the longer report?)
I remain unconvinced that it is better for someone to set out from the start to write a single long report, as opposed to—at the very least—an approach similar to Khan’s in which one starts out with shorter, concrete ‘pieces of the puzzle’ and actually puts each of them out in the community where they can be criticized and scrutinized. Then along the way there can be more public discourse, more feedback, and more public variety of opinion. One will be better able to judge which threads really are promising or important and which ideas really are robust and any decision to write a longer, expanded report that draws on the lessons of the many shorter papers, need not be made until later.
It’s a fair question. Technically speaking, of course progress can be more incremental, and some small pieces can be built on with other small pieces. Ultimately that’s what happened with Khan’s series of papers on the semiconductor supply chain and export control options. But in my opinion that kind of thing almost never really happens successfully when it’s different authors building on each other’s MVPs (minimum viable papers) rather than a single author or team building out a sorta-comprehensive picture of the question they’re studying, with all the context and tacit knowledge they’ve built up from the earlier papers carrying over to how they approach the later papers.
I’m skeptical of work like this and what you write in note 2. makes me think of some comments. You write:
If I’m honest, this seems to me like a large—and slightly odd—claim. Does it really say that research work that can be written up in fewer than one hundred pages cannot be persuasive and cannot be solid enough to build on? Do you really mean to suggest that?
You go on to write:
But I don’t really see how all this is much different from saying something a bit like “more research is better than less research” or that although an individual short paper cannot provide what you call ‘strategic’ insight, it may be gained by essentially synthesizing the results contained in lots of smaller contributions. And this latter description sounds a lot like how I would describe the ‘normal’ way that research papers have worked for ages, i.e. something like the fact that each individual contribution tends to be narrow and then as time goes by, the bigger picture emerges from some kind of aggregation and synthesis that the research community somehow effects. (I find it particularly hard to square the example of Saif Khan’s papers with your overall point: Is it not the case that many of Khan’s earlier outputs were much shorter? But each was high-quality enough that he could build on them, even after having put them out into the community? And then later, once the bigger picture had genuinely emerged for him to see, he drew on his experience to produce the longer report?)
I remain unconvinced that it is better for someone to set out from the start to write a single long report, as opposed to—at the very least—an approach similar to Khan’s in which one starts out with shorter, concrete ‘pieces of the puzzle’ and actually puts each of them out in the community where they can be criticized and scrutinized. Then along the way there can be more public discourse, more feedback, and more public variety of opinion. One will be better able to judge which threads really are promising or important and which ideas really are robust and any decision to write a longer, expanded report that draws on the lessons of the many shorter papers, need not be made until later.
It’s a fair question. Technically speaking, of course progress can be more incremental, and some small pieces can be built on with other small pieces. Ultimately that’s what happened with Khan’s series of papers on the semiconductor supply chain and export control options. But in my opinion that kind of thing almost never really happens successfully when it’s different authors building on each other’s MVPs (minimum viable papers) rather than a single author or team building out a sorta-comprehensive picture of the question they’re studying, with all the context and tacit knowledge they’ve built up from the earlier papers carrying over to how they approach the later papers.