Currently doing the Stanford Existential Risk Summer Fellowship in Field-Building and local AI safety Movement Building in Australia and NZ.
Chris Leong
Karma: 2,439
Standards can help with security b/c that’s more of a standard problem, but I suspect it’ll be a distraction for aligning AGI.
I guess the success of those standards for the web doesn’t feel very relevant to the problem of aligning AI. For a start, the design of the protocols has led to countless security flaws, hardly seems robust?
In addition, the technology has often evolved by messing up and then being patched later.
Sorry, that’s not ever I meant. See the article here for a definition of cached thoughts: https://www.lesswrong.com/tag/cached-thoughts#:~:text=Cached Thoughts are ideas%2C attitudes,t re-evaluated since then.
My point is that I’ve found people who come from leftist spaces sometimes have a tendency of saying a space is full of “straight white males” because that’s a catchy phrase without checking all three of the predicates to see if they match the particular context.
This is a bit of a hot-take, but I’m somewhat skeptical of the ability of standards to effectively regulate TAI. I suspect that in order to be safe, an actor will have to be willing to take measures beyond any standards, in which case implementing paragraph 23 subsection d will only be a distraction. On the other hand, standards could very easily slow the most responsible actors and cause one of the least responsible actors who doesn’t care about them at all to win the AGI race.
To be honest “straight white male” feels a bit like a cached thought to me. Are you sure that the number of EAs who are gay or bi would be less than the population distribution or less than the population distribution among the educated classes? It wouldn’t surprise me if it were more.
“ EU citizens can travel and work freely and citizens of many countries outside of the EU do not need visas to come and work here independently for 3 months”
Anyone have any details on what kinds of visas are b available and for what countries?
“Who is actually working on infinite ethics?”- I’m actually very interested in this question myself. If you know anyone working on infinite ethics, please connect me :-).
I think the current focus on one-on-ones adds a lot of value, but it also makes to tricky to engage in spontaneous conversations b/c a large percent of people are either waiting for one-on-ones are engaging in them. I would really like to see conferences specifically have an area for spontaneous conversation.
I’m happy to hear you have been able to make some worthwhile connections.
I’m expecting Berlin to really take off as a hub.
I’m curious. Do you only think it’s worth being connected to the community if your ideas get funded by the community? You’ve got a cool idea, but a lot of other people have cool ideas too and not everything can be funded.
I guess I see the community offering a host of other potential benefits as well, such as feedback, frameworks, access to talent, ect. I know it must be frustrating, but sometimes the negatives cloud out the positives. And there’s nothing wrong if you disagree with me here, just thought I’d offer another way of seeing things.
Great suggestion. I would love to see someone diving deeper into these topics.
This sounds like a really cool idea and I’d love to see someone experiment with it. I’m curious, what feedback on these ideas have you received from talking with Effective Altruists?
I also noticed that your post title was “EA’s Culture and Thinking are Severely Limiting Its Impact” and while there’s nothing wrong with criticism, I would have personally chosen a title that would have promoted the idea of developing an ecosystem of leaders.
I would be surprised if we could do much to slow AI, but I agree that at least a few people should look into this approach.
I think it could be a highly valuable project for someone to form a community around this as long as they were careful not to allow the discussion of extreme options within the group.
Sorry, I don’t have time to respond to all your points, but I agree that the EA movement can’t be demeaning. That isn’t the same though as optimising for a certain audience.
(Fixed typo: wrote demanding instead of demeaning).
Thanks for writing this article. You make a number of good points, however I don’t think you quite grapple with the strongest arguments against this.
The more you prioritise near-termist causes the more you’ll be tempted to grow the movement to maximise the donations to charity, whilst the more you prioritise long-termism, the more you‘ll prioritise high-skilled recruitment for specific skills we need (Actually, even near-termists may want to prioritise niche outreach as donations are heavy tailed too)
We face a dilemma—if everything counts as EA, then EA will lose its distinctiveness—but at the same time we don’t want to come off as narrow minded. I think part of the solution is acknowledging there are ways of doing altruism effectively outside of Effective Altruism. Then it makes sense for us to decide that we don’t need to do everything (this originally said ‘anything’ as a typo)
I really wish it weren’t the case, but broadening the movement too much would reduce the nuance in discussions by default.
I think it makes sense for Giving What We Can to focus on mass outreach and Effective Altruism to remain narrower rather than to do everything under the same banner as an event for everyone is an event for no-one.
Interesting idea, but to be honest, I suspect the cure worse than the disease.
That said, it might make sense for some people who have built up career capital within EA to use this capital to pursue opportunities outside of the movement so as to open up an opportunity for an upcoming EA, but this would only make sense in particular contexts.
Footnotes are your friend here—they allow you to add detail for those who need or want it—whilst not wasting everyone else’s times.
One other idea would be to invite some prominent skeptics and proponents to synthesize the best of their arguments and debate them, live or in writing, with an emphasis on clear, jargon-free language (maybe such a project already exists?).
This is a pretty good idea!
I thought this post raised many points worth pondering, but I am skeptical of the actual suggestions largely because it underrates the benefits of the current setup and neglects the costs. I’ll list my thoughts below:
a) Yeah, the community aspect is worrying in terms of how it distorts people’s incentives, but I believe that we also have to be willing to back ourselves and not risk crippling our effectiveness by optimising too much on minimising downside in the case where we are wrong.
b) Ways of thinking and frameworks of the world are much more than merely a “vibe” or “intellectual aesthetic”.
c) Groupthink can be addressed by other less costly interventions, such the Criticism Red Teaming and Competition. I imagine that we could run other projects in this space, such as providing longer-term funding for people who bring a different perspective to the table. These aren’t perfect, but achieving goals is much easier when you have many like-minded people.
d) I disagree with the EA style being hard to acquire, but I agree that it’s important for us to be able to appreciate criticisms not written in this style as otherwise we’ll learn from others at a much slower rate.
e) I feel that EA is somewhat dropping the ball on special responsibilities at the moment. With our current resources and community size, I think that we could address this without substantially impacting our mission although this might change over the longer-term.
f) I feel that the advantages of a community are vastly underrated by this post. For one, the community provides a vital talent pool. Many people would never have gotten involved in direct work if there wasn’t a lower commitment step than redirected their career that they could take first or a local community running events to help them understand why the cause was important. I suppose we could structure events and activities to minimise the extent to which people become friends, but that would just be a bad community.
g) That said, we should dedicate more effort towards recruiting people with the specific skills that we need. We need more programs like the Legal Priorities Summer Institute or the EA Communicators Fellowship. I’m also bullish on cause-specific movement building to attract people who might not vibe with EA.
h) Giving What We Can and GiveWell are spreading the meme ‘when donating keep in mind that some charities are more effective than others’ without positioning in a broader EA framework.
i) “We should not try to get people who care a lot about animal welfare or decreasing global inequality to care about x-risk”—I agree that we want to limit the amount of effort/resources spend trying to poach people from other cause areas, but shifting from one cause are to another could potential lead to orders of magnitude in the impact someone has.
j) EA was pretty skeptical about being too meta-focused at first due to worries that we might lose our focus on direct work, but in retrospect, I suspect that were wrong not to have spent more money on community-building.
k) I’d like to see the section on less internal recruiting engage with the argument that value alignment is actually important. I think that the inevitable result of hiring people from society at large would be a watering down of EA ideas and rather than a marginally less impactful project being pursued, I expect that in many cases this could reduce impact by an order of magnitude from having people pursue the highest impact project that is high status rather than the highest impact project. There’s also significant risk in that once you bring in people who aren’t value-aligned, they bring in more people who aren’t value-aligned and then the whole culture changes. I’m guessing you might not think this is important given that you’ve described it as a “vibe”, but I’d suggest that having the right culture is a key part of achieving high performance.
l) “So for example, AI risk would probably be cut off around now”—While this would free up resources to incubate more causes, this could also be a major blunder if AI risk is an immediate, short-term priority and we need to be moving on this ASAP.
Most of what I’ve written here is criticism, so I wanted to emphasise again that I found the ideas here fascinating and I definitely think reading your post was worth my time :-).