Note: I’m being a bit adversarial with these questions, probably because the book launch advertisements are annoying me a bit. Still, an answer to my questions could tip me over to pre-ordering/not pre-ordering the book.
Would you say that this book meaningfully moves the frontier of the public discussion on AI x-risk forward?
As in, if someone’s read much to ~all of the publicly available MIRI material (including all of the arbital alignment domain, the 2021 dialogues, the LW sequences, and even some of the older papers), plus a bunch of writing from detractors (e.g. Pope, Belrose, Turner, Barnett, Thornley, 1a3orn), will they find updated defenses/elaborations on the evolution analogy, why automated alignment isn’t possible, why to expect expected utility maximizers, why optimization will be “infectious”, and some more on things linked here?
Additionally, would any of the long-time MIRI-debaters (as mentioned above, also including Christiano, the OpenPhil/Constellation cluster of people) plausibly give a positive endorsement of the book as to not just being a good distillation, but moving the frontier of the public discussion forward?
Thanks for your comment :) sorry you finding all the book posts annoying, I decided to post here after seeing that there hadn’t been a post on the EA Forum
I’m not actually sure what book content I’m allowed to talk about publicly before the launch. Overall the book is written much more for an audience who are new to the AI x-risk arguments (e.g., policymakers and the general public), and it is less focused on providing new arguments to people who have been thinking/reading about this for years (although I do think they’ll find it an enjoyable and clarifying read). I don’t think it’s trying to go 15 arguments deep in a LessWrong argument chain. That said, I think there is new stuff in there; the arguments are clearer than previously, there are novel framings on things, and I would guess that there’s at least some things in there that you would find new. I don’t know if I would expect people from the “Pope, Belrose, Turner, Barnett, Thornley, 1a3orn” crowd to be convinced, but they might appreciate the new framings. There will also be related online resources, which I think will cover more of the argument tree, although again, I don’t know how convincing this will be to people who are already in deep.
If you’re a LessWrong regular, you might wonder whether the book contains anything new for you personally. The content won’t come as a shock to folks who have read or listened to a bunch of what Eliezer and I have to say, but it nevertheless contains some new articulations of our arguments, that I think are better articulations than we’ve ever managed before.
I would guess many people from the OpenPhil/Constellation cluster would give endorsements as the book being a good distillation. But insofar as it’s moving the frontier of online arguments about AI x-risk forward, it will mainly be by saying arguments more clearly (which imo is still progress).
Thanks! That’s the kind of answer I was looking for. I’ll sleep a night about pre-ordering, and then definitely look forward more to the online appendices. (I also should’ve specified it’s the other Barnett ;-)
Note: I’m being a bit adversarial with these questions, probably because the book launch advertisements are annoying me a bit. Still, an answer to my questions could tip me over to pre-ordering/not pre-ordering the book.
Would you say that this book meaningfully moves the frontier of the public discussion on AI x-risk forward?
As in, if someone’s read much to ~all of the publicly available MIRI material (including all of the arbital alignment domain, the 2021 dialogues, the LW sequences, and even some of the older papers), plus a bunch of writing from detractors (e.g. Pope, Belrose, Turner, Barnett, Thornley, 1a3orn), will they find updated defenses/elaborations on the evolution analogy, why automated alignment isn’t possible, why to expect expected utility maximizers, why optimization will be “infectious”, and some more on things linked here?
Additionally, would any of the long-time MIRI-debaters (as mentioned above, also including Christiano, the OpenPhil/Constellation cluster of people) plausibly give a positive endorsement of the book as to not just being a good distillation, but moving the frontier of the public discussion forward?
Thanks for your comment :) sorry you finding all the book posts annoying, I decided to post here after seeing that there hadn’t been a post on the EA Forum
I’m not actually sure what book content I’m allowed to talk about publicly before the launch. Overall the book is written much more for an audience who are new to the AI x-risk arguments (e.g., policymakers and the general public), and it is less focused on providing new arguments to people who have been thinking/reading about this for years (although I do think they’ll find it an enjoyable and clarifying read). I don’t think it’s trying to go 15 arguments deep in a LessWrong argument chain. That said, I think there is new stuff in there; the arguments are clearer than previously, there are novel framings on things, and I would guess that there’s at least some things in there that you would find new. I don’t know if I would expect people from the “Pope, Belrose, Turner, Barnett, Thornley, 1a3orn” crowd to be convinced, but they might appreciate the new framings. There will also be related online resources, which I think will cover more of the argument tree, although again, I don’t know how convincing this will be to people who are already in deep.
Here’s what Nate said in the LW announcement post:
I would guess many people from the OpenPhil/Constellation cluster would give endorsements as the book being a good distillation. But insofar as it’s moving the frontier of online arguments about AI x-risk forward, it will mainly be by saying arguments more clearly (which imo is still progress).
Thanks! That’s the kind of answer I was looking for. I’ll sleep a night about pre-ordering, and then definitely look forward more to the online appendices. (I also should’ve specified it’s the other Barnett ;-)