tl;dr: I now think that EA community-builders should present ideas in a less weird way when it doesn’t come at the expense of clarity, but maybe the advice “be less weird” is not good advice because it might make community-builders avoid communicating weird ideas that are worth communicating.
You probably leave some false impressions either way
In some sense (in the sense I actually care about), both statements are misleading.
I think that community builders are going to convey more information, on average, if they start with the less weird statement.
Often inferential gaps can’t be crossed in a conversation.
That missing understanding will always get filled in with something inaccurate (if they had an accurate impression, then there is no inferential gap here). The question is, which misconceptions are better to leave someone with?
You’ve outlined how “an AI system might be hard to control and because of that, some experts think it could be really dangerous” could be misunderstood. I agree that people are unlikely to think you mean “the AI system will kill us all” without further elaboration. They will attach more accessible examples to the vaguer statement in the meantime. It is unlikely they will attach one really specific wrong example though, there is ambiguity there and the uncertainty left from that ambiguity is much better than a strongly held false impression (if those are the two choices, ideally, the inferential gap gets closed and you get a strongly held true impression).
People who are hearing the statement “the AI system will kill us all” without further context will still try and attach the most accessible examples they have to make the phrase make as much sense as possible to them. This tends to mean Skynet style walking robots. They’ll also probably hypothesize that you don’t have very good epistemics (even if this is not the language they’d use to describe it). They won’t trust you to have good reasons to believe what you do because you’ve made an extraordinary claim without having laid out the case for it yet. These are false impressions too. They are also likely to stick more because the extra weirdness makes these first impressions much more memorable.
Which impression do I prefer community builders leave newcomers with?
I value community builders conveying the reasoning processes much more than the bottom line. I want newcomers to have the tools to come to reasonable conclusions for themselves (and I think giving newcomers the reasons why the EA community has its current conclusions is a good start).
Giving newcomers a more accurate impression of a conclusion without giving them much context on that conclusion seems often worse than nothing. Especially since you often lose the trust of reasonable people when you make very surprising claims and can’t back them up in the conversation (because that’s too much of an inferential distance to cross in the time you have).
Giving them an accurate impression of one of the reasons for a conclusion seems neutral (unaligned AI seems like it could be an x-risk because AI systems are hard to control). That isolated reason without further elaboration doesn’t actually say that much, but I think it does lay the groundwork for a deeper understanding of the final conclusion “AI might kill us all” if future conversations happen down the road.
My takeaways
After this discussion, I’ve changed my mind on “be less weird” being the right advice to get what I want. I can see how trying to avoid being weird might make community builders avoid getting the point across.
Something like “aim to be as accurate as possible using language and examples the person you are talking to can understand” still probably will result in less weird. I’d be surprised if it resulted in community builders obscuring their point.
Fair enough.
tl;dr: I now think that EA community-builders should present ideas in a less weird way when it doesn’t come at the expense of clarity, but maybe the advice “be less weird” is not good advice because it might make community-builders avoid communicating weird ideas that are worth communicating.
You probably leave some false impressions either way
In some sense (in the sense I actually care about), both statements are misleading.
I think that community builders are going to convey more information, on average, if they start with the less weird statement.
Often inferential gaps can’t be crossed in a conversation.
That missing understanding will always get filled in with something inaccurate (if they had an accurate impression, then there is no inferential gap here). The question is, which misconceptions are better to leave someone with?
You’ve outlined how “an AI system might be hard to control and because of that, some experts think it could be really dangerous” could be misunderstood. I agree that people are unlikely to think you mean “the AI system will kill us all” without further elaboration. They will attach more accessible examples to the vaguer statement in the meantime. It is unlikely they will attach one really specific wrong example though, there is ambiguity there and the uncertainty left from that ambiguity is much better than a strongly held false impression (if those are the two choices, ideally, the inferential gap gets closed and you get a strongly held true impression).
People who are hearing the statement “the AI system will kill us all” without further context will still try and attach the most accessible examples they have to make the phrase make as much sense as possible to them. This tends to mean Skynet style walking robots. They’ll also probably hypothesize that you don’t have very good epistemics (even if this is not the language they’d use to describe it). They won’t trust you to have good reasons to believe what you do because you’ve made an extraordinary claim without having laid out the case for it yet. These are false impressions too. They are also likely to stick more because the extra weirdness makes these first impressions much more memorable.
Which impression do I prefer community builders leave newcomers with?
I value community builders conveying the reasoning processes much more than the bottom line. I want newcomers to have the tools to come to reasonable conclusions for themselves (and I think giving newcomers the reasons why the EA community has its current conclusions is a good start).
Giving newcomers a more accurate impression of a conclusion without giving them much context on that conclusion seems often worse than nothing. Especially since you often lose the trust of reasonable people when you make very surprising claims and can’t back them up in the conversation (because that’s too much of an inferential distance to cross in the time you have).
Giving them an accurate impression of one of the reasons for a conclusion seems neutral (unaligned AI seems like it could be an x-risk because AI systems are hard to control). That isolated reason without further elaboration doesn’t actually say that much, but I think it does lay the groundwork for a deeper understanding of the final conclusion “AI might kill us all” if future conversations happen down the road.
My takeaways
After this discussion, I’ve changed my mind on “be less weird” being the right advice to get what I want. I can see how trying to avoid being weird might make community builders avoid getting the point across.
Something like “aim to be as accurate as possible using language and examples the person you are talking to can understand” still probably will result in less weird. I’d be surprised if it resulted in community builders obscuring their point.