But when the proposal becomes: “we should not actually study progress or try to accelerate it”, I get lost.
I don’t think anyone is proposing this. The debate I’m interested in is about which priorities are most pressing at the margin (i.e. creates the most value per unit of resources).
The main claim isn’t that speeding up tech progress is bad,* just that it’s not the top priority at the margin vs. reducing x-risk or speeding up moral progress.**
One big reason for this is that lots of institutions are already very focused on increasing economic productivity / discovering new tech (e.g. ~2% of GDP is spent on R&D), whereas almost no-one is focused on reducing x-risk.
If the amount of resources reducing xrisk grows, then it will drop in effectiveness relatively speaking.
In Toby’s book, he roughly suggests that spending 0.1% of GDP on reducing x-risk is a reasonable target to aim for (about what is spent on ice cream). But that would be ~1000x more resources than today.
*Though I also think speeding up tech progress is more likely to be bad than reducing xrisk, my best guess is that it’s net good.
**This assumes resources can be equally well spent on each. If someone has amazing fit with progress studies, that could make them 10-100x more effective in that area, which could outweigh the average difference in pressingness.
Cool to see this thread!
Just a very quick comment on this:
I don’t think anyone is proposing this. The debate I’m interested in is about which priorities are most pressing at the margin (i.e. creates the most value per unit of resources).
The main claim isn’t that speeding up tech progress is bad,* just that it’s not the top priority at the margin vs. reducing x-risk or speeding up moral progress.**
One big reason for this is that lots of institutions are already very focused on increasing economic productivity / discovering new tech (e.g. ~2% of GDP is spent on R&D), whereas almost no-one is focused on reducing x-risk.
If the amount of resources reducing xrisk grows, then it will drop in effectiveness relatively speaking.
In Toby’s book, he roughly suggests that spending 0.1% of GDP on reducing x-risk is a reasonable target to aim for (about what is spent on ice cream). But that would be ~1000x more resources than today.
*Though I also think speeding up tech progress is more likely to be bad than reducing xrisk, my best guess is that it’s net good.
**This assumes resources can be equally well spent on each. If someone has amazing fit with progress studies, that could make them 10-100x more effective in that area, which could outweigh the average difference in pressingness.