This isn’t intended to be a complete response to your post, but for comparison, here are some other things that ambitious tech companies have serious plans to accomplish:
SpaceX has a pretty credible plan to create a revolutionary space-launch system (Starship) and use it to literally colonize another planet. The USA doesn’t seem as excited about this as it rationally ought to be, nor do China or other countries seem as concerned as one might expect.
Various crypto companies and projects have big plans to literally replace fiat currency, undermining one of the fundamental ways that governments wield power. Crypto does seem to be on the radar of most governments, but still less than one might expect given the long-term stakes.
Nuclear fusion companies like Helion, General Fusion, and others have collectively raised billions of dollars to make “energy too cheap to meter” a reality. You’d think governments would be more excited about subsidizing potentially incredible energy sources like this, but maybe they’ve been burned too many times before or maybe they’re just slow.
There is a lot we could straightforwardly do to be better prepared for future pandemics, but we’re mostly not doing that stuff (hugely scaled-up metagenomic sequencing, improved ventilation and PPE, making vacaccines in advance for all potential pandemic viruses, etc) even though it’s totally legible, it doesn’t require any risky breakthroughs, and it’s extremely salient due to covid.
Obviously the consequences of even Mars colonization, full cryptoization of the economy, abundant power from fusion, and significantly mitigated biorisk pale in comparison to the transformative power of AGI. But from a government’s perspective they might all seem to be in the same reference class. Yet it’s surprisingly close to “crickets” on all counts. (I admit that AGI might be especially neglected even here, though—SpaceX at least gets normal NASA contracts, etc.)
I think SpaceX’s regular non-Mars-colonization activities are in fact taken seriously by relevant governments, and the Mars colonization stuff seems like it probably won’t happen and also wouldn’t be that big a deal if it did (in terms of, like, national security; it would definitely affect who gets into the history books). So it doesn’t seem to me like governments are necessarily acting irrationally there.
Same with cryptocurrency; its implications for investor protection, tax evasion, capital controls evasion, and facilitating illicit transactions are indeed taken seriously, and while governments would obviously care quite a lot if it displaced fiat currency, I just don’t think there’s any way that’s happening. If it does, then this is probably because fiat currency itself somehow stopped working and something was needed to fill the void; if governments think this scenario is at all plausible, then presumably their attention would be on the first part where fiat currency fails, since that’s much more within their control and cryptocurrency isn’t really a relevant input.
The scientific and regulatory culture around fusion power seems to be shaped, as you suggest, by the long history of failures in that domain; judging by similar situations in other fields, I wouldn’t be surprised if no one wanted to admit to putting any credence in it, so that they wouldn’t look stupid in case it fails again.
The state of pandemic preparedness does indeed seem like just straight-up government incompetence.
This isn’t intended to be a complete response to your post, but for comparison, here are some other things that ambitious tech companies have serious plans to accomplish:
SpaceX has a pretty credible plan to create a revolutionary space-launch system (Starship) and use it to literally colonize another planet. The USA doesn’t seem as excited about this as it rationally ought to be, nor do China or other countries seem as concerned as one might expect.
Various crypto companies and projects have big plans to literally replace fiat currency, undermining one of the fundamental ways that governments wield power. Crypto does seem to be on the radar of most governments, but still less than one might expect given the long-term stakes.
Nuclear fusion companies like Helion, General Fusion, and others have collectively raised billions of dollars to make “energy too cheap to meter” a reality. You’d think governments would be more excited about subsidizing potentially incredible energy sources like this, but maybe they’ve been burned too many times before or maybe they’re just slow.
There is a lot we could straightforwardly do to be better prepared for future pandemics, but we’re mostly not doing that stuff (hugely scaled-up metagenomic sequencing, improved ventilation and PPE, making vacaccines in advance for all potential pandemic viruses, etc) even though it’s totally legible, it doesn’t require any risky breakthroughs, and it’s extremely salient due to covid.
Obviously the consequences of even Mars colonization, full cryptoization of the economy, abundant power from fusion, and significantly mitigated biorisk pale in comparison to the transformative power of AGI. But from a government’s perspective they might all seem to be in the same reference class. Yet it’s surprisingly close to “crickets” on all counts. (I admit that AGI might be especially neglected even here, though—SpaceX at least gets normal NASA contracts, etc.)
I think SpaceX’s regular non-Mars-colonization activities are in fact taken seriously by relevant governments, and the Mars colonization stuff seems like it probably won’t happen and also wouldn’t be that big a deal if it did (in terms of, like, national security; it would definitely affect who gets into the history books). So it doesn’t seem to me like governments are necessarily acting irrationally there.
Same with cryptocurrency; its implications for investor protection, tax evasion, capital controls evasion, and facilitating illicit transactions are indeed taken seriously, and while governments would obviously care quite a lot if it displaced fiat currency, I just don’t think there’s any way that’s happening. If it does, then this is probably because fiat currency itself somehow stopped working and something was needed to fill the void; if governments think this scenario is at all plausible, then presumably their attention would be on the first part where fiat currency fails, since that’s much more within their control and cryptocurrency isn’t really a relevant input.
The scientific and regulatory culture around fusion power seems to be shaped, as you suggest, by the long history of failures in that domain; judging by similar situations in other fields, I wouldn’t be surprised if no one wanted to admit to putting any credence in it, so that they wouldn’t look stupid in case it fails again.
The state of pandemic preparedness does indeed seem like just straight-up government incompetence.
That’s a good point, and I like the examples, thanks!