I think people have made fair points around my bar maybe being generally unreasonable. I.e. it’s the default for any research field that major insights don’t appear out of nowhere, and that it’s almost always possible to find similar previous ideas [...]
I think this is largely correct, but that it’s still correct to update negatively on the value of research if past progress has been less good on the spectra of majority and novelty.
I don’t understand the last sentence there. In particular, I’m not sure what you mean “less good” in comparison to.
Do you mean if past progress has been less major and novel than expected? If so, then I’d agree that it’s correct to update negatively if that’s the case.
But given the point about “the default for any research field”, it seems unclear to me whether it’s actually been less major and novel than expected. Perhaps instead we’ve had roughly the sort and amount of progress that people would’ve expected ~2015 when thinking that more money and people should flow towards doing longtermist macrostrategy/GPR?
So here are three things I think you might mean:
“Longtermist macrostrategy/GPR’s insights have been even less major and novel than one would typically expect. So we should update negatively about the value of more work in this field in particular (including relative to work in other fields) - perhaps it’s unusually intractable.”
“People who advocated, funded, or did longtermist macrostrategy/GPR had failed to recognise that research fields rarely have major insights out of nowhere, and thus overestimated the value of more research in general. So we should update negatively about the value of more research in general, including in relation to this field.”
“People who advocated, funded, or did longtermist macrostrategy/GPR mistakenly thought that that field would be an exception to the general pattern of fields rarely having major, novel insights. Now that we have evidence that they were probably mistaken, we should update negatively about this field in particular (moving towards thinking it’s more like other fields).”
Is one of those an accurate description of your view?
Among your three points, I believe something like 1 (for an appropriate reference class to determine “typical”, probably something closer to ‘early-stage fields’ than ‘all fields’). Though not by a lot, and I also haven’t thought that much about how much to expect, and could relatively easily be convinced that I expected too much.
I don’t think I believe 2 or 3. I don’t have much specific information about assumptions made by people who advocated for or funded macrostrategy research, but a priori I’d find it surprising if they had made these mistakes to a strong extent.
I also haven’t thought much about how much one should typically expect in a random field, how that should increase or decrease for this field in the last 5 years just because of how many people and dollars it got (compared to other fields), or how what was produced in the last 5 years in this field compares to that.
But one thing that strikes me is that longtermist macrostrategy/GPR researchers over the past 5 years have probably had substantially less training and experience than researchers in most academic fields we’d probably compare this to. (I haven’t really checked this, but I’d guess it’s true.)
So maybe if there was less novel or less major insights from this field than we should typically expect of a field with the same amount of people and dollars, this can be explained by the people having less human capital, rather than by the field being intrinsically harder to make progress on?
(It could also perhaps be explained if the unusual approaches that are decently often taken in this field tend to be less effective—e.g., more generalist/shallow work rather than deeper dives into narrower topics, and more blog post style work.)
Interesting thoughts, thanks :)
I don’t understand the last sentence there. In particular, I’m not sure what you mean “less good” in comparison to.
Do you mean if past progress has been less major and novel than expected? If so, then I’d agree that it’s correct to update negatively if that’s the case.
But given the point about “the default for any research field”, it seems unclear to me whether it’s actually been less major and novel than expected. Perhaps instead we’ve had roughly the sort and amount of progress that people would’ve expected ~2015 when thinking that more money and people should flow towards doing longtermist macrostrategy/GPR?
So here are three things I think you might mean:
“Longtermist macrostrategy/GPR’s insights have been even less major and novel than one would typically expect. So we should update negatively about the value of more work in this field in particular (including relative to work in other fields) - perhaps it’s unusually intractable.”
“People who advocated, funded, or did longtermist macrostrategy/GPR had failed to recognise that research fields rarely have major insights out of nowhere, and thus overestimated the value of more research in general. So we should update negatively about the value of more research in general, including in relation to this field.”
“People who advocated, funded, or did longtermist macrostrategy/GPR mistakenly thought that that field would be an exception to the general pattern of fields rarely having major, novel insights. Now that we have evidence that they were probably mistaken, we should update negatively about this field in particular (moving towards thinking it’s more like other fields).”
Is one of those an accurate description of your view?
Yes, I meant “less than expected”.
Among your three points, I believe something like 1 (for an appropriate reference class to determine “typical”, probably something closer to ‘early-stage fields’ than ‘all fields’). Though not by a lot, and I also haven’t thought that much about how much to expect, and could relatively easily be convinced that I expected too much.
I don’t think I believe 2 or 3. I don’t have much specific information about assumptions made by people who advocated for or funded macrostrategy research, but a priori I’d find it surprising if they had made these mistakes to a strong extent.
I also haven’t thought much about how much one should typically expect in a random field, how that should increase or decrease for this field in the last 5 years just because of how many people and dollars it got (compared to other fields), or how what was produced in the last 5 years in this field compares to that.
But one thing that strikes me is that longtermist macrostrategy/GPR researchers over the past 5 years have probably had substantially less training and experience than researchers in most academic fields we’d probably compare this to. (I haven’t really checked this, but I’d guess it’s true.)
So maybe if there was less novel or less major insights from this field than we should typically expect of a field with the same amount of people and dollars, this can be explained by the people having less human capital, rather than by the field being intrinsically harder to make progress on?
(It could also perhaps be explained if the unusual approaches that are decently often taken in this field tend to be less effective—e.g., more generalist/shallow work rather than deeper dives into narrower topics, and more blog post style work.)