Iâm not defending what you think is a bailey, but as a practical matter, I would say until recently (with Open Phil publishing a few models for AI), longtermists have not been using numbers or models much, or when they do, some of the most important parameters are extremely subjective personal guesses or averages of peopleâs guesses, not based on reference classes, and risks of backfire were not included.
Fair. I should revise my claim to being about the likelihood of a catastrophe and the risk reduction from working on these problems (especially or only in AI; I havenât looked as much at whatâs going on in other x-risks work). AI Impacts looks like they were focused on timelines.
Replied to LinchâTL;DR: I agree this is true compared to global poverty or animal welfare, and I would defend this as simply the correct way to respond to actual differences in the questions asked in longtermism vs. those asked in global poverty or animal welfare.
You could move me by building an explicit quantitative model for a popular question of interest in longtermism that (a) didnât previously have models (so e.g. patient philanthropy or AI racing doesnât count), (b) has an upshot that we didnât previously know via verbal arguments, (c) doesnât involve subjective personal guesses or averages thereof for important parameters, and (d) I couldnât immediately tear a ton of holes in that would call the upshot into question.
You could move me by building an explicit quantitative model for a popular question of interest in longtermism that (a) didnât previously have models (so e.g. patient philanthropy or AI racing doesnât count), (b) has an upshot that we didnât previously know via verbal arguments, (c) doesnât involve subjective personal guesses or averages thereof for important parameters, and (d) I couldnât immediately tear a ton of holes in that would call the upshot into question.
I feel that (b) identifying a new upshot shouldnât be necessary; I think it should be enough to build a model with reasonably well-grounded parameters (or well-grounded ranges for them) in a way that substantially affects the beliefs of those most familiar with or working in the area (and maybe enough to change minds about what to work on, within AI or to AI or away from AI). E.g., more explicitly weighing risks of accelerating AI through (some forms of) technical research vs actually making it safer, better grounded weights of catastrophe from AI, a better-grounded model for the marginal impact of work. Maybe this isnât a realistic goal with currently available information.
Yeah, I agree that would also count (and as you might expect I also agree that it seems quite hard to do).
Basically with (b) I want to get at âthe model does something above and beyond what we already had with verbal argumentsâ; if it substantially affects the beliefs of people most familiar with the field that seems like it meets that criterion.
Iâm not defending what you think is a bailey, but as a practical matter, I would say until recently (with Open Phil publishing a few models for AI), longtermists have not been using numbers or models much, or when they do, some of the most important parameters are extremely subjective personal guesses or averages of peopleâs guesses, not based on reference classes, and risks of backfire were not included.
This seems to me to not be the case. For a very specific counterexample, AI Impacts has existed since 2015.
Fair. I should revise my claim to being about the likelihood of a catastrophe and the risk reduction from working on these problems (especially or only in AI; I havenât looked as much at whatâs going on in other x-risks work). AI Impacts looks like they were focused on timelines.
Replied to LinchâTL;DR: I agree this is true compared to global poverty or animal welfare, and I would defend this as simply the correct way to respond to actual differences in the questions asked in longtermism vs. those asked in global poverty or animal welfare.
You could move me by building an explicit quantitative model for a popular question of interest in longtermism that (a) didnât previously have models (so e.g. patient philanthropy or AI racing doesnât count), (b) has an upshot that we didnât previously know via verbal arguments, (c) doesnât involve subjective personal guesses or averages thereof for important parameters, and (d) I couldnât immediately tear a ton of holes in that would call the upshot into question.
I feel that (b) identifying a new upshot shouldnât be necessary; I think it should be enough to build a model with reasonably well-grounded parameters (or well-grounded ranges for them) in a way that substantially affects the beliefs of those most familiar with or working in the area (and maybe enough to change minds about what to work on, within AI or to AI or away from AI). E.g., more explicitly weighing risks of accelerating AI through (some forms of) technical research vs actually making it safer, better grounded weights of catastrophe from AI, a better-grounded model for the marginal impact of work. Maybe this isnât a realistic goal with currently available information.
Yeah, I agree that would also count (and as you might expect I also agree that it seems quite hard to do).
Basically with (b) I want to get at âthe model does something above and beyond what we already had with verbal argumentsâ; if it substantially affects the beliefs of people most familiar with the field that seems like it meets that criterion.