We could potentially survey the EA community on this later this year. Please feel free to reach out if you have specific requests/suggestions for the formulation of the question.
I’ve heard versions of the claim multiple times, including from people i’d expect to know better, so having the survey data to back it up might be helpful even if we’re confident we know. the answer.
Where I think most EAs would strongly disagree with is that they would find pursuing SAI “at all costs” to be abhorrent and counter to their fundamental goals. But I also suspect that showing survey data about EA’s professed beliefs wouldn’t be entirely convincing to some people given the close connections between EAs and rationalists in AI.
I feel a bit uneasy that EAs should put in a lot of effort into a survey (both the survey designers and takers) just because someone made up something at some point. Maybe asking the people who you’d expect to know better, why they believe what they believe?
>”Despite the potential risks, EAs broadly believe super-intelligent AI should be pursued at all costs.” This is inaccurate imo.
Could we get a survey on a few versions of this question? I think it’s actually super-rare in EA.
e.g.
“i believe super-intelligent AI should be pursued at all costs”
“I believe the benefits outweigh the risks of pursuing superintelligent AI”
“I believe if risk of doom can be agreed to be <0.2, then the benefits of AI outweight the risks”
“I believe even if misalignment risk can be reduced to near 0, pursuing superintelligence is undesirable”
We could potentially survey the EA community on this later this year. Please feel free to reach out if you have specific requests/suggestions for the formulation of the question.
Yeah it’s incredibly inaccurate,
I don’t think it even needs to be surveyed.I’ve heard versions of the claim multiple times, including from people i’d expect to know better, so having the survey data to back it up might be helpful even if we’re confident we know. the answer.
I think there are truths that are not so far from it. Some rationalists believe Superintelligent AI is necessary for an amazing future. Strong versions of AI Safety and AI capabilities are complementary memes that start from similar assumptions.
Where I think most EAs would strongly disagree with is that they would find pursuing SAI “at all costs” to be abhorrent and counter to their fundamental goals. But I also suspect that showing survey data about EA’s professed beliefs wouldn’t be entirely convincing to some people given the close connections between EAs and rationalists in AI.
Good point! You’re right
I feel a bit uneasy that EAs should put in a lot of effort into a survey (both the survey designers and takers) just because someone made up something at some point. Maybe asking the people who you’d expect to know better, why they believe what they believe?