Thanks for sharing, Joe. I think the absolute value of the welfare of AI systems will remain much smaller than that of wild animals over at least the next few decades. According to Ege Erdil, āin 2024 we spent ~ $100B on NVIDIA GPUs, and by 2024 Q4 the total computing power of all NVIDIA datacenter chips worldwide was only around 4e21 FLOP/āsā, equal to that of 40 M human brains (= 4*10^(27 ā 15)). So, even if the absolute value of the welfare per FLOP of AI systems was equal to that of humans, their computing power would have to become 200 (= 8*10^9/ā(40*10^6)) times as large as that of NVIDIA datacenter chips for the absolute value of the welfare of AI systems to match that of humans. From Table S1 of Bar-on et al. (2018), there are 10^20 marine arthropods, 12.5 billion (= 10^20/ā(8*10^9)) times as many as humans. If the absolute value of the welfare per second of marine arthropods is 2*10^-4 that of humans, 10 % of Rethink Prioritiesā median welfare range of silkworms, the absolute value of their welfare would be 2.50 M (= 12.5*10^9*2*10^-4) times that of humans. So I estimate the computing power of AI systems would have to become 500 M (= 200*2.50*10^6) times as large as that of NVIDIA datacenter chips for the absolute value of their welfare to match that of marine arthropods assuming the absolute value of the welfare per FLOP of AI systems was equal to that of humans. 500 M is a huge factor considering the spending on NVIDIA GPUs in 2024 was around 0.1 % (= 100*10^9/ā(100*10^12)) of the gross world product in 2024.
Rapid algorithmic improvement (perhaps in the course of a software only singularity) could easily make welfare per FLOP much higher than humans, though this would depend on underlying moral views.
Also, I think we might see extremely fast economic growth leading to vastly more compute (e.g. via harnessing much of the energy of the sun).
Hmm, 10k$ is maybe too small size to be worth it, but I might be down to do:
You transfer $50k to me now.
If AIs arenāt able to automate a typical senior individual contributor software engineer /ā research engineer by 2031 (based on either credible reports about whatās happening inside AI companies or testing of externally deployed systems), I send $75k to you. ($75k = $50k * (1 + 1ā5) * (1.045^5.5) The 1.045 comes from interest rates.)
More precise operationalization: typical in AI development or some other moderate importance sector where the software engineering doesnāt require vision. Also, the AI needs to be able to automate what this job looked like in 2025 (as this job might evolve over time with AI capabilities to be what AIs canāt do).
Iād like to bet on a milestone that triggers before itās too late for human intervention if possible, so Iāve picked this research engineer milestone. Weād presuambly have to operationalize further. Iām not sure if I think itās worth the time to try to operationalize enough that we could do a bet.
Thanks for the suggestion, Ryan! 50 k$ is a large fraction of my savings, so it would not work for me. Moreover, I agree software engineering will be largely automated until 2031. I am just very sceptical that this will lead to a growth explosion. I guess my probability of it being automated more than e.g. 99 % is much lower than yours, but this may be hard to operationalise. The bet does not have to resolve in time for human intervention for it to be financially positive for you if I transfer money to you now? I wonder whether you expect the annual unemployment rate, globally or in the United States (US), to be much higher in 2027. In the US, it was 4 % in 2024, and I would say the probability of it being higher than 8 % in 2027 is lower than 1ā3.
Stocks grew 5 % per year from 1900 to 2022, and I think they will grow faster over the next 5 years, maybe around 10 % per year. So I would want to use an interest rate higher than yours of 4.5 %.
Thanks for sharing, Joe. I think the absolute value of the welfare of AI systems will remain much smaller than that of wild animals over at least the next few decades. According to Ege Erdil, āin 2024 we spent ~ $100B on NVIDIA GPUs, and by 2024 Q4 the total computing power of all NVIDIA datacenter chips worldwide was only around 4e21 FLOP/āsā, equal to that of 40 M human brains (= 4*10^(27 ā 15)). So, even if the absolute value of the welfare per FLOP of AI systems was equal to that of humans, their computing power would have to become 200 (= 8*10^9/ā(40*10^6)) times as large as that of NVIDIA datacenter chips for the absolute value of the welfare of AI systems to match that of humans. From Table S1 of Bar-on et al. (2018), there are 10^20 marine arthropods, 12.5 billion (= 10^20/ā(8*10^9)) times as many as humans. If the absolute value of the welfare per second of marine arthropods is 2*10^-4 that of humans, 10 % of Rethink Prioritiesā median welfare range of silkworms, the absolute value of their welfare would be 2.50 M (= 12.5*10^9*2*10^-4) times that of humans. So I estimate the computing power of AI systems would have to become 500 M (= 200*2.50*10^6) times as large as that of NVIDIA datacenter chips for the absolute value of their welfare to match that of marine arthropods assuming the absolute value of the welfare per FLOP of AI systems was equal to that of humans. 500 M is a huge factor considering the spending on NVIDIA GPUs in 2024 was around 0.1 % (= 100*10^9/ā(100*10^12)) of the gross world product in 2024.
Rapid algorithmic improvement (perhaps in the course of a software only singularity) could easily make welfare per FLOP much higher than humans, though this would depend on underlying moral views.
Also, I think we might see extremely fast economic growth leading to vastly more compute (e.g. via harnessing much of the energy of the sun).
Fair enough if you are skeptical of these views.
How soon would you expect to see extremely fast economic growth? Say >20%/āyear GDP growth.
Conditional on no intentional slow down, maybe median 2035 or something? I donāt have a cached 25th percentile for this, but maybe more like 2031.
Thanks, Ryan.
I remain open to bets against short AI timelines, or what they supposedly imply, up to 10 k$.
Hmm, 10k$ is maybe too small size to be worth it, but I might be down to do:
You transfer $50k to me now.
If AIs arenāt able to automate a typical senior individual contributor software engineer /ā research engineer by 2031 (based on either credible reports about whatās happening inside AI companies or testing of externally deployed systems), I send $75k to you. ($75k = $50k * (1 + 1ā5) * (1.045^5.5) The 1.045 comes from interest rates.)
More precise operationalization: typical in AI development or some other moderate importance sector where the software engineering doesnāt require vision. Also, the AI needs to be able to automate what this job looked like in 2025 (as this job might evolve over time with AI capabilities to be what AIs canāt do).
Iād like to bet on a milestone that triggers before itās too late for human intervention if possible, so Iāve picked this research engineer milestone. Weād presuambly have to operationalize further. Iām not sure if I think itās worth the time to try to operationalize enough that we could do a bet.
Thanks for the suggestion, Ryan! 50 k$ is a large fraction of my savings, so it would not work for me. Moreover, I agree software engineering will be largely automated until 2031. I am just very sceptical that this will lead to a growth explosion. I guess my probability of it being automated more than e.g. 99 % is much lower than yours, but this may be hard to operationalise. The bet does not have to resolve in time for human intervention for it to be financially positive for you if I transfer money to you now? I wonder whether you expect the annual unemployment rate, globally or in the United States (US), to be much higher in 2027. In the US, it was 4 % in 2024, and I would say the probability of it being higher than 8 % in 2027 is lower than 1ā3.
Stocks grew 5 % per year from 1900 to 2022, and I think they will grow faster over the next 5 years, maybe around 10 % per year. So I would want to use an interest rate higher than yours of 4.5 %.