Not really. EA really shouldn’t presume a moral consequentialism approach, just real-world effectiveness.
Sharmake
Karma: 37
There are two factions working to prevent AI dangers. Here’s why they’re deeply divided.
Thank you, I’ll remove the downvotes. On priors a lot of discussion on this topic tends to go bad places fast, so I wanted to head that off.
Not exactly what I meant. Specifically, when the amount of people are the bottleneck, then yeah, don’t be elitist, you just hire people. It’s in the higher and more-variance impact that intelligence matters far more.
Basically: The higher the variance of impact, the more intelligence matters. The lower the variance, the less intelligence matters.
Returns to intelligence are very large, thanks to the heavy-tail nature of most tasks due to being log-normal or power law distribution, not a normal distribution. From Gwern on human intelligence:
Turning to human intelligence, the absolute range of human intelligence is very small: differences in reaction times are small, backwards digit spans range from 3–7, brain imaging studies have difficulty spotting neurological differences, the absolute genetic influence on intelligence is on net minimal, and this narrow range may be a general phenomenon about humans (Wechsler 1935); and yet, in human society, how critical are these tiny absolute differences in determining who will become rich or poor, who will become a criminal, who will do cutting-edge scientific research, who will get into the Ivy Leagues, who will be a successful politician, and this holds true as high as IQ can be measured reliably (see TIP / SMPY etc).
Something similar has been noted about human intelligence—while any particular day-to-day decision has little to do with intelligence, the effects of intelligence are consistently beneficial and accumulate over a lifetime, so the random noise starts to cancel out, and intelligence is seen to have strong correlations with long-term outcomes (eg. Gottfredson 1997). More abstractly, many career or intellectual outcomes have been noticed to follow a roughly log-normal distribution; a log-normal distributed can be generated when an outcome is the end result of a ‘leaky pipeline’ (scientific output might be due to motivation times intelligence times creativity…), in which case a small improvement on each variable can yield a large improvement in the output. Such a leaky pipeline might be simply a long sequence of actions, where advantage can build up (eg. if there is a small chance of making a blunder with each action).
I want to make 2 seemly contradictory claims:
-
EA longterm needs to be less elitist, and it needs to stop trying to do so much recruiting at elite universities.
-
EA originally being elitist made sense.
One of the biggest reasons EA has gotten so powerful, so quickly is billionaires injecting a lot of money, but also because of elites. Specifically, the biggest constraint at the the 2010s era was even making a movement that could actually be effective, and since they didn’t have much money, effectiveness mattered most. And intelligence has very large effects because of heavy tails: the most intelligent people are disproportionately much more effective.
However, this constraint started to change once more people got introduced to Effective Altruism, and more money got into the ecosystem.
Also base rate neglect applies here.
That stated, the next decade of EA outreach needs to be less elitist than the last decade.
-
Unfortunately, caring about even one generation into the future, let alone several thousand or more, is already a big shift in Western Culture. If you can get Westerners to care about one generation in the future, you’ve mostly won. Also, I’ll give some people credit for thinking about 100-200 years in the future, but how many truly think like this? That’s the question. There’s a large inferential gap that needs to be bridged here.
Basically, what I mean here is that EA/LessWrong has wisely not fallen for shiny political discussions and are able to stay focused on the original goal, for EA that is doing the most good, and for LessWrong it’s rationality. And there is a massive lesson for people that try to do good in the world from Robin Hanson: Pull sideways, rather than forwards or backwards in politics.
EA’s comparative advantage is in noticing and effectively dealing with important, less-discussed causes like AI, global poverty, bio-risk, x-risk more generally, and more. Another advantage EA has is that it roughly tracks the truth epistemically (within some error bars), and this matters because while you do need to, at the end, be able to put your vision into practice, you also need to be able to notice how the world actually looks like in order to do any goal well, like do-gooding. Intelligence and rationality helps every cause. And without significant changes to how humans work, politics is where we are at our most irrational, and EA has rightly found political power to be low tractability, medium-high importance, and low neglectedness. Political discussion is much much lower importance than political power. It’s a perfectly anti-effective cause area.
As for my ranking of causes, I’d say the top 5 causes of impact are:
-
AI Alignment
-
X-risk/Longtermism
-
Global Health
-
Global Poverty
-
Cause X like inequality.
-
Finally, returning to x-risk. The examples above were also chosen to illustrate a different ‘vibe’ that could apply to x-risk besides ‘impeding disaster and heroic drama’. Safety engineering is non-heroic by design: a saviour snatching affairs from the jaws of disaster indicates an intolerable single point of failure. Rather, success is a team effort which is resilient to an individual’s mistake, and their excellence only slightly notches down the risk even further. Yet this work remains both laudable and worthwhile: a career spent investigating not-so-near misses to tease out human factors to make them even more distant has much to celebrate, even if not much of a highlight reel.
‘Existential safety’ could be something similar. Risks of AI, nukes, pandemics etc. should be at least as remote of those of a building collapsing, a plane crashing, or a nuclear power plant melting down. Hopefully these risks are similarly remote, and hopefully one’s contribution amounts to a slight incremental reduction. Only the vainglorious would wish otherwise.
Unfortunately, for the biggest risk, AI, the fact that we depend on EA, MIRI and other organizations to be heroes is a dangerous deficiency in civilizational competence. A lot of that issue is AGI is weird to outsiders, and politicization is strong enough to make civilizational competence negative. Hell, everything in existential risks could be like this.
Basically, William MacAskill’s longtermism, or EA longtermism is trying to solve the distributional shift issue. Most cultures that have long-term thinking assume that there’s no distributional shift such that no key assumptions of the present are wrong. Now if this assumption is correct, we shouldn’t interfere with cultures, as they will go to local optimums. But it isn’t and thus longtermism from has to deal with weird scenarios like AI or x-risk.
Thus the form of EA longtermism is not obvious, as it can’t assume that there’s no distributional shift into out of distribution behavior. In fact, we have good reasons of thinking that there will be massive distributional shifts. That’s the key difference between EA and other culture’s longtermism.
It’s also because, to put it in Eliezer’s words, politics is the mind killer. EA has just accepted the fact that trying to systemically change the world politically is almost impossible and mind kills people (I’m looking at you, Communists.)
The biggest reason EA/LessWrong has good epistemics and is actually effective is because it abhors politics and politics discussion basically doesn’t exist here, compared to other groups. It just focuses on the non-political stuff, where we aren’t mind killed.
To be sort of blunt, I suspect a lot of the reason why AI safety memes are resisted by companies like Deepmind is because taking them seriously would kill their business models. It would force them to fire all their AI capabilities groups, expand their safety groups, and at least try to not be advancing AI.
When billions of dollars are on the line, potentially trillions if the first AGI controls the world for Google, it’s not surprising that any facts that would reconsider that goal are doomed to fail. Essentially it’s a fight between a human’s status-seeking, ambition and drive towards money vs ethics and safety, and with massive incentives for the first group of motivations means the second group loses.
On pivotal acts, a lot of it comes from MIRI who believe that a hard takeoff is likely, and to quote Eli Tyre, hard vs soft takeoff matters for whether pivotal acts need to be done:
On the face of it, this seems true, and it seems like a pretty big clarification to my thinking. You can buy more time or more safety, at little bit at a time, instead of all at once, in sort of the way that you want to achieve life extension escape velocity.
But it seems like this largely depends on whether you expect takeoff to be hard or soft. If AI takeoff is hard, you need pretty severe interventions, because they either need to prevent the deployment of AGI or be sufficient to counter the actions of a superintelligece. Generally, it seems like the sharper takeoff is, the more good outcomes flow through pivotal acts, and the smoother takeoff is the more we should expect good outcomes to flow through incremental improvements.
Are there any incremental actions that add up to a “pivotal shift” in a hard takeoff world?
My usual answer to VWH concerns is that it unreasonably assumes that we can align global states and make sure they stay aligned. The default state is a narrow interest group usually takes it over. Also, states have more reason to pursue black-ball technologies like nukes, pandemics and more, and it has no incentive to pursue gray or white-ball technologies, so a global surveillance state will in time try to kill billions based on narrow interest groups.
My answers for them (From an EA perspective) is generally the following:
Same values is 30% chance
Worse values 69% chance
Better Values 1% chance
Agree, and hiring PR people does matter. You usually want a positive view, at least weakly, so they don’t become adversarial to your goals.
I agree, though I suspect that one was simply designed badly.
I really want to ask the question: Why do people think that doom is inevitable or that things usually get worse, not better? Is it actually founded on reality, or is a lot of the doomerism psychological and social?
I also understand that outside view 1 entirely, given voters have very bad moods about both the economy and inflation, when they’re actually doing okayish.
Sorry to come in late, but I do have a link for a near-utopia here:
https://orionsarm.com/ is essentially as utopian as it can be, barring the 3 powers of time travel, reality warping or thermodynamics breaking.
Which is essentially in his perspective, we are utterly doomed barring technological stagnation.
We can’t win: Either we get a bang x-risk from individuals, or we get a stable x-risk from states.
Hi: I’m Sharmake Farah that has been in the forum since February.
I got to EA by getting a free book called The Precipice, which introduced genuinely novel concepts to me like existential risks, longtermism and our potentially very long future ahead. While I got used to the scales, it’s still large enough to matter.
I made my first post today, focusing on AI ethics and alignment fields fighting, and why that’s bad. Here it is: https://forum.effectivealtruism.org/posts/AARnvz99hiEytnA9k/there-are-two-factions-working-to-prevent-ai-dangers-here-s
My biggest impact so far is in reframing what matters the most in a healthy way. I got much less worried about politics and got to see a broader perspective than my own country’s near term politics.