The insistence that only a certain form of evidence, in this case, empirical evidence, would count is for an argument is something Eliezer anticipated in the sequences, unsurprisingly.
Not this person, but many AI risk arguments are necessarily logical rather than empirical—there are good reasons to believe the relevant behaviors won’t appear, or be trivially easy to counter (at least re: harmful outputs), until you have very capable systems.
Like, if I can construct a deceptive response-to-training strategy (but current models can’t), that’s enough evidence to be concerned future superhuman models might do similar deceptive alignment. Other concerns like inner optimizers (e.g. humans stopped being kid-maxxers at high capability, because our proxy decoupled from evolution’s target) might not show up, or change in character, as models become less limited. And even when you can demonstrate the behavior empirically, people dismiss it as overly-induced or a toy environment—which was the whole point, just to show plausibility not prove it.
More fundamentally: If I argue that a future thing logically implies certain risks arise, responding with “there’s no empirical evidence” is silly. Logical chains and structural arguments are still valid epistemic tools.
I’m not trying to imply it, I’m trying to state it clearly. You dismiss the arguments made in the book as not being empirical. If you haven’t read your post, here are some quotes indicating where you do this explicitly: ”the chapter presents effectively zero empirical research” “one might expect Y&S to substantiate their case with empirical evidence” ″lack of empirical evidence”
I did not write the post, or read the book. However, based on the podcasts with Eliezer Yudkowsky and Nate Soares I have listened to, I would also like them to focus more on empirical evidence.
Do you see another bet we could make about AI risk? I remain open tobetsagainst short AI timelines, or what they supposedly imply, up to 10 k$. I am also open the increasing the stakes of our bet.
The insistence that only a certain form of evidence, in this case, empirical evidence, would count is for an argument is something Eliezer anticipated in the sequences, unsurprisingly.
https://www.lesswrong.com/w/logical-rudeness
Hi David. Are you implying this post is neglecting non-empirical evidence? If so, which type of evidence do you have in mind?
Not this person, but many AI risk arguments are necessarily logical rather than empirical—there are good reasons to believe the relevant behaviors won’t appear, or be trivially easy to counter (at least re: harmful outputs), until you have very capable systems.
Like, if I can construct a deceptive response-to-training strategy (but current models can’t), that’s enough evidence to be concerned future superhuman models might do similar deceptive alignment. Other concerns like inner optimizers (e.g. humans stopped being kid-maxxers at high capability, because our proxy decoupled from evolution’s target) might not show up, or change in character, as models become less limited. And even when you can demonstrate the behavior empirically, people dismiss it as overly-induced or a toy environment—which was the whole point, just to show plausibility not prove it.
More fundamentally: If I argue that a future thing logically implies certain risks arise, responding with “there’s no empirical evidence” is silly. Logical chains and structural arguments are still valid epistemic tools.
I’m not trying to imply it, I’m trying to state it clearly. You dismiss the arguments made in the book as not being empirical. If you haven’t read your post, here are some quotes indicating where you do this explicitly:
”the chapter presents effectively zero empirical research”
“one might expect Y&S to substantiate their case with empirical evidence”
″lack of empirical evidence”
I did not write the post, or read the book. However, based on the podcasts with Eliezer Yudkowsky and Nate Soares I have listened to, I would also like them to focus more on empirical evidence.
Do you see another bet we could make about AI risk? I remain open to bets against short AI timelines, or what they supposedly imply, up to 10 k$. I am also open the increasing the stakes of our bet.