I agree with “beware point estimates”, but it’s not always right that point estimates cause people to overstate the value of working on things. They can also understate it—if your point estimate is that things will fail even with your extra effort, then things look less bleak if you account for the possible worlds where things are easier than you think and you could actually make the difference.
It’s often the case with all-or-nothing problems that the relevant counterfactuals are not shifting it from “will never be solved” to “will be solved”, but shifting forward in time the moment of the solution.
If anyone’s interested in this, I have a largely-complete old paper draft (“on the marginal benefits of research”) that I’m embarrassed to realise we never actually published, which has some simple first-pass economic modelling of the counterfactual value in this case. I could share it privately or check in with my coauthors about the possibility of posting a public link.
I guess the main cases where it’s incorrect to model it as a speedup are where the solution is needed by a particular time.
Although perhaps the obvious such case is differential technological development, but now uncertainty about the timelines to other technologies might smooth things out again, so that in expectation you get a stream of benefits from solving earlier.
I agree with your first point and I should have mentioned it.
On your second point, I am assuming that ‘solving’ the problem means solving it by a date, or before some other event (since there’s no time in my model). But I agree this is often going to be the right way to think, and a case where the value of working on a problem with increasing resources can be smooth, even under certainty.
Shared you on the draft. Yes, it’s related to those old blog posts but extending the analysis by making more assumptions about “default” growth functions for various key variables, and so getting to say something a bit more quantitative about the value of the speed-up.
Nice! Two comments:
I agree with “beware point estimates”, but it’s not always right that point estimates cause people to overstate the value of working on things. They can also understate it—if your point estimate is that things will fail even with your extra effort, then things look less bleak if you account for the possible worlds where things are easier than you think and you could actually make the difference.
It’s often the case with all-or-nothing problems that the relevant counterfactuals are not shifting it from “will never be solved” to “will be solved”, but shifting forward in time the moment of the solution.
If anyone’s interested in this, I have a largely-complete old paper draft (“on the marginal benefits of research”) that I’m embarrassed to realise we never actually published, which has some simple first-pass economic modelling of the counterfactual value in this case. I could share it privately or check in with my coauthors about the possibility of posting a public link.
I guess the main cases where it’s incorrect to model it as a speedup are where the solution is needed by a particular time.
Although perhaps the obvious such case is differential technological development, but now uncertainty about the timelines to other technologies might smooth things out again, so that in expectation you get a stream of benefits from solving earlier.
Thanks for the comment, Owen.
I agree with your first point and I should have mentioned it.
On your second point, I am assuming that ‘solving’ the problem means solving it by a date, or before some other event (since there’s no time in my model). But I agree this is often going to be the right way to think, and a case where the value of working on a problem with increasing resources can be smooth, even under certainty.
Re your draft, I’d be interested in taking a look :)
I thought you’ve written something similar here—https://globalprioritiesproject.org/2015/02/project-overview-problems-of-unknown-difficulty/ (website is currently down 🐛 - archived version)
Shared you on the draft. Yes, it’s related to those old blog posts but extending the analysis by making more assumptions about “default” growth functions for various key variables, and so getting to say something a bit more quantitative about the value of the speed-up.