I am a generalist quantitative researcher. I am open to volunteering and paid work. I welcome suggestions for posts. You can give me feedback here (anonymously or not).
Vasco Grilošø
Karma: 11,167
Hi Emile. I see this is your 1st comment on the EA Forum. Welcome.
I think the difference in uncertainty is mostly explained by the surveys covering different people, not by ESPAIās predictions having been made around 2.25 years earlier. ESPAI 2023 involved ā2,778 researchers who had published in top-tier artificial intelligence (AI) venuesā, and ātook place in the fall of 2023ā³. The 2026 Summit on Existential Security (SES) involved āleaders and key thinkers in the x-risk and AI safety communitiesā, and āSurvey data comes from the 59 respondents who consented to their answers being shared publiclyā, and āwas collected in February 2026ā.
Buck, I would be curious to know what is your median time from weak AGI to artificial superintelligence (ASI) in this question from Metaculus, and your best guess for the (unconditional) probability of human extinction in the next 10 years.
Thanks for the helpful clarifications, Melanie. They made sense to me.
Hi Buck. True. I still think the survey underestimates the variance in median AI timelines. Below are the results for the 2023 Expert Survey on Progress in AI (ESPAI). Half of the responses for the median date of full automation of tasks or occupations range from around 2045 to some date after 2120. In the survery of the post, half of the responses for the median date of AGI range from around 2032 to 2037. For the 25th percentile date of full automation, half of ESPAIās responses range from around 2030 to 2100. In the survey of the post, half of the responses for the 25th percentile date of AGI range from around 2028 to 2032. AGI in the survey of the post does not have the exact same meaning as full automation of taks or occupations, but I am pretty confident my broad point stands if I am reading the graph below correctly.
What is your probability of human extinction in the 10 years following the achievement of artificial superintelligence (ASI) as defined by AI Futures?
Hi Michael.
(Or: Why I donāt see how the probability of extinction could be less than 25% on the current trajectory)
Lesss than 25 % from now until when?
Thanks for the comment, titotal. I agree the survey underestimates the variance in AI timelines and risk.
The AI Futures, which is know for AI 2027, had super broad timelines for artificial superintelligence (ASI) timelines on January 26. The difference between the 90th and 10th percentile was 168 years for Daniel Kokotajlo (2027 to 2195), and 137 years for Eli Lifland (2028 to 2165).
There is also huge variation in assessment of AI extinction risk. In the Existential Risk Persuasion Tournament (XPT), among domain experts and superforecasters, the 5th and 95th percentile AI extinction risk from 2023 to 2100 were 9.45*10^-7 and 37.0 % (excluding the 7 people who guessed a risk of exactly 0; here are the results).
Do you have more thoughts on this comment? Feel free to follow up there.
Understood.
Got it.
No, I am not confident that biofuel subsidies decrease the population of invertebrates.
This makes sense to me, but I am not sure I fully understand why you describe biofuel subsidies as āquite appealingā for people who are āsufficiently suffering-focusedā. Maybe you believe that soil microarthropods are the most important to determine the expected change in welfare? In this case, I would agree that biofuel subsidies would be quite appealing because they seem to robustly decrease the population of microarthropods. However, I can easily see the welfare of soil macroarthropods or nematodes being much larger than that of soil microarthropods, and there is significant uncertainty about whether biofuel subsidies increase or decrease the population of soil macroarthropods/ānematodes.
These are shallow investigations and I expect that additional research would change our minds about many of the conclusions that people reached.
This is why I like the intervention āInsecticides and insect welfare: a research agendaā. It is explicitly about doing further research.
Thanks for the useful context, Bob. Is there any grant round on soil animals that you would be willing to run for less than 100 k$? It does not have to be about investigating sentience, or comparing the welfare of soil animals with that of humans, and it could be about soil ants or termites.
By ārobustly increase welfareā, I meant that welfare is expected to increase (under expectational total hedonistic utilitarianism (ETHU); ignoring moral uncertainty), and this conclusion is not sensitive to close to arbitrary empirical assumptions (for example, whether invertebrates of some species have positive or negative lives). You do not think intervention 14 satisfies this?
Are you confident that biofuel subsidies decrease the population of invertebrates? From Table 1 of the report, accounting only for invertebrates with at least ā2mmā (macrofauna), corn with 357 animals per m^2 (= 126 + 231) replaces grassland with 970 animals per m^2 (= 441 + 529), thus leading to 613 fewer animals per m^2 corn (= 970 ā 357). However, from Table S4 of Rosenberg et al. (2023) (in the Supplementary Materials), replacing temperate grasslands, savannas, and shrublands with crops results in 598 more soil ants, termites, and other soil arthropods besides springtails and mites (macroarthropods) per m^2 (= (-1.06 + 1.66)*10^3 + 0.533). The change in the number of animals per m^2 is ā1.06 k for soil ants, 1.66 k for soil termites, and 0.533 for soil arthropods besides springtails and mites. Adding up the lower/āupper bounds of the 95 % confidence intervals (CIs) in Table S4, I conclude there are 900 to 6.4 k macroarthropods per m^2 in crops, and 172 to 7.00 k in temperate grasslands, savannas, and shrublands. There is significant overlap between these ranges. So it is unclear to me whether replacing temperate grasslands, savannas, and shrublands with crops increases or decreases the number of macroarthropods. The same goes for replacing grassland with crops in the United States (US)?
I also think it is worth looking into the effects of increasing cropland on the number of microarthropods and nematodes. I have see macroarthropods, microarthropods, nematodes, or any combination of these being the major driver of total welfare.
Replacing temperate grasslands, savannas, and shrublands with crops robustly decreases the number of soil springtails and mites (microarthropods) according to Table S4 of Rosenberg et al. (2023). Adding up the lower/āupper bounds of the 95 % CIs, there are 11 k to 37 k soil microarthropods per m^2 in crops, and 70 k to 170 k in temperate grasslands, savannas, and shrublands. There is no overlap between these ranges.
However, I believe replacing crops with grassland may increase or decrease the number of soil animals accounting for all animals. The vast majority of soil animals are nematodes, and I am very uncertain about whether replacing crops with grassland increases or decreases the number of soil nematodes.
From Figure 1a of Li et al. (2022), which is below, it is unclear whether cropland has more or fewer soil nematodes than āprimary habitat (undisturbed natural habitat)ā, or āsecondary habitat (recovering, previously disturbed natural habitat)ā. For example, secondary habitat which is āunmanaged (no documented or observed direct human disturbance)ā is estimated to have fewer soil nematodes than cropland and pasture which are unmanaged or āmanaged (more or less disturbed by various human activities like fertilization, tillage, grazing, logging, etc.)ā.
Figure 7 of the meta-analysis of Pothula et al. (2019), which is below, suggests it is very unclear whether agricultural land has more or fewer soil nematodes than natural or disturbed grassland, or forest.
White (2022) concludes ānematode abundance is higher in managed than unmanaged primary and secondary habitatsā, which is compatible with crops having more nematodes than grassland.
That said, our confidence in our own position is not high. So, weād be willing to fund things to challenge our own views: If we had sufficient funding from folks interested in the question, Arthropoda would fund a grant round specifically on soil invertebrate sentience and relevant natural history studies (especially in ways that attempt to capture the likely enormous range of differences between species in this group). Currently, much of our grant-making funds are restricted (at least informally) to farmed insects and shrimp, so itās not an option.
Could you elaborate on what would be āsufficient fundingā for āa grant round specifically on soil invertebrate sentience and relevant natural history studiesā?
Thanks for sharing, Bob. The database seems like a great resource to build interest in increasing the welfare of wild animals. Do you think any of the 28 interventions there robustly increase (total) welfare (in expectation) accounting for all animals (in particular, soil animals)? The only one I feel confident achieves this is āInsecticides and insect welfare: a research agendaā (14; here is the report, and here is scoring sheet).
Hi Michael.
Animal agriculture has huge effects on wild animals (land use, climate change) but with unclear sign, so farmed animal work could backfire spectacularly.
Here is an illustration of the above.
Even chicken welfare reforms have effects on land use.
I think chicken welfare reforms may impact soil ants and termites much more than chickens.
Thanks for the great post.
Life spans of insects, however, are quite variable. Carey (2001) notes that the between-group variation is enormous: for herbivores, this range includes aphids with a lifespan of weeks, to xylem-feeding beetles that take several years to reach maturity, to termite queens that can live for decades. This 5000-fold difference in the life spans of insects
Did you mean 500-fold difference? 1 decade is 522 weeks (= 10*365.25/ā7).
I see. I agree an infinitesimal change to one of 2 exactly identical states could make their expected welfare incomparable under your framework. However, it does not follow that any 2 interventions are incomparable with respect to how much they change expected welfare (across all space and time). I think intervals representing the expected change in welfare are sufficiently narrow for any decision-relevant comparisons to be feasible, although very often with lots of (standard) uncertainty involved.
Hi CB. I would be curious to know your thoughts on my post Chicken welfare reforms may impact soil ants and termites much more than chickens?. You are welcome to comment on the post.
What makes two actions incomparable, under the imprecise EV model, is that the interval of EV differences crosses zero.
What exactly do you mean by āinterval of EV differencesā? Imagine A = [a1, a2], and B = [b1, b2] are intervals representing the imprecise expected welfare of 2 states of the world, and that b2 >= a2. What would be the āinterval of EV differencesā between B and A in terms of a1, a2, b1, and b2? I thought it would be BāA = [b1 - a2, b2 - a1].
Got it. Thanks. Here is what @Ryan Greenblatt says in the piece you linked about fatalities from AI takeover.
Ryan, could you clarify what is the timeline for the 50 % of humans dying in expectation conditional on takeover?
The 25 % chance of human extinction refers to the 300 years following takeover, and excludes voluntary informed extinction, which I like because it would not be obviously bad. Here is footnote 4.