Who are “they”? If you mean Yudkowsky and Soares, “QED” is something that Hanson (the author of this critique) includes in his paraphrase of Yudkowsky and Soares, but I don’t think it’s anything Yudkowsky and Soares wrote in their book. The quoted argument is not actually a quote, but a paraphrase.
Thanks for clarifying, Erich. I believe Falk was referring to Yudkowsky and Soares. I have not read their book. I have just listened to podcasts they have done, and skimmed some of their writings. However, I think the broader point stands that they often use language that implies much greater confidence in and robustness of the possibility of human extinction than warranted by their arguments.
I didn’t realize the quoted text was a paraphrase rather than an exact quote. I only commented on the paraphrase, not on the book itself. I apologize for the oversight.
I completely agree with what you just stated (although I have not read the post you linked), but I do not understand why it would undermine the broader point I mentioned in my comment.
If you thought Yudkowsky and Soares used overly confident language and would have taken the “QED” as further evidence of that, but this particular example turns out not to have been written by Yudkowsky and Soares, that’s some evidence against your hypothesis. But instead of updating away a little, you seemed to dismiss that evidence and double down. (I think you originally replied to the original comment approvingly or at least non-critically, but then deleted that comment after I replied to it, but I could be misremembering that.)
For what it’s worth, I think you’re right that Yudkowsky at least uses overly confident language sometimes—or I should say, is overly confident sometimes, because I think his language generally reflects his beliefs—but I would’ve been surprised to see him use “QED” in that way, which is why I reacted to the original comment here with skepticism and checked whether “QED” actually appeared in the book (it didn’t). I take that to imply I was better calibrated than anyone who did not so react.
If you thought Yudkowsky and Soares used overly confident language and would have taken the “QED” as further evidence of that, but this particular example turns out not to have been written by Yudkowsky and Soares, that’s some evidence against your hypothesis.
I agree.
But instead of updating away a little, you seemed to dismiss that evidence and double down.
I updated away a little, but negligibly so.
I think you originally replied to the original comment approvingly or at least non-critically, but then deleted that comment after I replied to it, but I could be misremembering that.
I deleted a comment which said something like the following. “Thanks, Falk. I very much agree”. I did not remember “QED” was Robin paraphrasing. However, I think the “QED” is still supposed to represent the level of confidence of the authors (in the book) in their arguments for a high risk of human extinction.
I would’ve been surprised to see him use “QED” in that way, which is why I reacted to the original comment here with skepticism and checked whether “QED” actually appeared in the book (it didn’t).
Interesting. I would not have found the use of “QED” surprising. To me it seems that Yudkowsky is often overly confident.
I remain open tobetsagainst short AI timelines, or what they supposedly imply, up to 10 k$. Do you see any that we could make that is good for both of us considering we could invest our money, and that you could take loans?
Who are “they”? If you mean Yudkowsky and Soares, “QED” is something that Hanson (the author of this critique) includes in his paraphrase of Yudkowsky and Soares, but I don’t think it’s anything Yudkowsky and Soares wrote in their book. The quoted argument is not actually a quote, but a paraphrase.
Thanks for clarifying, Erich. I believe Falk was referring to Yudkowsky and Soares. I have not read their book. I have just listened to podcasts they have done, and skimmed some of their writings. However, I think the broader point stands that they often use language that implies much greater confidence in and robustness of the possibility of human extinction than warranted by their arguments.
I didn’t realize the quoted text was a paraphrase rather than an exact quote. I only commented on the paraphrase, not on the book itself. I apologize for the oversight.
If the evidence doesn’t change one’s mind if it goes the other way, it wasn’t really supporting one’s belief in the first place.
I completely agree with what you just stated (although I have not read the post you linked), but I do not understand why it would undermine the broader point I mentioned in my comment.
If you thought Yudkowsky and Soares used overly confident language and would have taken the “QED” as further evidence of that, but this particular example turns out not to have been written by Yudkowsky and Soares, that’s some evidence against your hypothesis. But instead of updating away a little, you seemed to dismiss that evidence and double down. (I think you originally replied to the original comment approvingly or at least non-critically, but then deleted that comment after I replied to it, but I could be misremembering that.)
For what it’s worth, I think you’re right that Yudkowsky at least uses overly confident language sometimes—or I should say, is overly confident sometimes, because I think his language generally reflects his beliefs—but I would’ve been surprised to see him use “QED” in that way, which is why I reacted to the original comment here with skepticism and checked whether “QED” actually appeared in the book (it didn’t). I take that to imply I was better calibrated than anyone who did not so react.
I agree.
I updated away a little, but negligibly so.
I deleted a comment which said something like the following. “Thanks, Falk. I very much agree”. I did not remember “QED” was Robin paraphrasing. However, I think the “QED” is still supposed to represent the level of confidence of the authors (in the book) in their arguments for a high risk of human extinction.
Interesting. I would not have found the use of “QED” surprising. To me it seems that Yudkowsky is often overly confident.
I remain open to bets against short AI timelines, or what they supposedly imply, up to 10 k$. Do you see any that we could make that is good for both of us considering we could invest our money, and that you could take loans?