Off-the-cuff reflection on communication (mainly triggered by the appendix):
It is very difficult to figure out what is most useful to say to an individual person, let alone a given group of people. We should expect to face great uncertainty and huge room for reasonable disagreement about many, perhaps most of the big decisions here.
While strict adherence to “say what you believe is true” is nearly always the right approach in spirit, you can’t avoid big decisions about emphasis, sequencing, premises you make explicit or implicit, ideas you argue for or take as given, the terminology you use, the tone, use of metaphor or example or logical notation, and so on.
There are surely some things that most would agree on as a big mistake (e.g. tasteless or unnecessarily upsetting examples), and lots of general rules of thumb people agree on too. But for something like how to talk about AI risk and X-risk estimates and timelines… that’s always going to be hard.
My impression is that Will did an unusually large amount of work to test arguments, examples and messages with different audiences while writing the book (think: lots of seminars and talks, not just asking contractors to message test using Amazon Turk).
Even if authors are occasionally willing to put serious effort into this kind of “user research”, my guess is that we need to heavily rely upon a portfolio approach.
On a portfolio approach, the stuff that we expect to get the biggest audience should probably be the most risk-averse, i.e. the bigger the reach, the more concerned it should be with not hard-bouncing people, and leaving most of the readers with at least a somewhat positive impression of the ideas and the people working on them.
Thanks for the post.
Off-the-cuff reflection on communication (mainly triggered by the appendix):
It is very difficult to figure out what is most useful to say to an individual person, let alone a given group of people. We should expect to face great uncertainty and huge room for reasonable disagreement about many, perhaps most of the big decisions here.
While strict adherence to “say what you believe is true” is nearly always the right approach in spirit, you can’t avoid big decisions about emphasis, sequencing, premises you make explicit or implicit, ideas you argue for or take as given, the terminology you use, the tone, use of metaphor or example or logical notation, and so on.
There are surely some things that most would agree on as a big mistake (e.g. tasteless or unnecessarily upsetting examples), and lots of general rules of thumb people agree on too. But for something like how to talk about AI risk and X-risk estimates and timelines… that’s always going to be hard.
My impression is that Will did an unusually large amount of work to test arguments, examples and messages with different audiences while writing the book (think: lots of seminars and talks, not just asking contractors to message test using Amazon Turk).
Even if authors are occasionally willing to put serious effort into this kind of “user research”, my guess is that we need to heavily rely upon a portfolio approach.
On a portfolio approach, the stuff that we expect to get the biggest audience should probably be the most risk-averse, i.e. the bigger the reach, the more concerned it should be with not hard-bouncing people, and leaving most of the readers with at least a somewhat positive impression of the ideas and the people working on them.
Oh cool I just saw that Will MacAskill wrote a long comment on this topic in reply to Scott Alexander.