Currently doing local AI safety Movement Building in Australia and NZ.
Chris Leong
Karma: 7,442
I expect it’d be possible to build some good ops experience in blue collar roles.
I would like to suggest that folk not downvote this post below zero. I’m generally in favour of allowing people to defend themselves, unless their response is clearly in bad faith. I’m sure many folk strongly disagree with the OP’s desired social norms, but this is different from bad faith.
Additionally, I suspect most of us have very little insight into how community health operates and this post provides some much needed visibility. Regardless of whether you think their response was just right, too harsh or too lenient, this post opens up a rare opportunity for the community to weigh in.I suspect people are downvoting this post either because they think the author is a bad person or they don’t want the author at EA events. I would suggest that neither of these are good reasons to downvote this specific post into the negative.
When will this be running?
This is a shame to hear. Do you have any thoughts on why this wasn’t appealing to funders despite the positive feedback/survey results?
Thinking this through: what’s novel is not so much the idea that the path AI takes affects non-human welfare, but that it’s worth developing this as its own subfield.
And the argument for this is much stronger in the current context: the arguments for rapid AI progress, AI companies not being responsible by default and AI not being aligned by default are much more legible these days.
And that makes it much easier to build energy around this as there seem to be folks in the EA animal welfare crowd who were skeptical about AI/AI risk before, but now see that this seems like it is going to be a big deal. Compared to standard AI alignment/governance, the explicit inclusion of animals makes it resonate more with their current interests, in addition being an area where their existing skills/knowledge are likely to be more applicable.
So I suspect what matters is not just having the idea, but deciding to promote the idea in the right context.
I agree with this problem, that there are many folk who lack courage, particularly within AI governance, but I don’t really see these kinds of comments as being that likely to change things.
There are times when it can make sense to shout at people, but it normally works better when there’s a clear and realistic path that’s been sketched out[1]. And when this isn’t the case, it tends to simply create strife for very little gain.
- ^
Someone really needs to write an article along the lines of “The case for being bold”.
- ^
If your complaint is that the default assumption is that these are more or less true, well, my claim is that even though normies tend to see this as a negative signal, it’s actually a positive signal for those with good epistemics.
That said, it’s important to keep in mind that these aren’t directly talking about actual systems (the Orthogonality Thesis is about possible systems and Instrumental Convergence is about incentives).
Interesting post.
I think it did a good job of explaining why the metacrisis might be relevant from an EA standpoint. I made a similar (but different!) argument—that Less Wrong should be paying attention to the sensemaking space—back in 2021[1] and it may still be helpful for readers who want to get better sense of the scene[2].Unfortunately, I’m with Amina. Short AI timelines are looking increasingly likely and culture change tends to take a long time, so the argument for prioritising this isn’t looking as promising as it previously did[3]. It’s a shame that some of these conversations didn’t start happening a decade or two earlier than they did. Some of these conversations could have been great for preparing the (intellectual) soil and it could have provided motiviation for working on generally useful infrastructure at the point in time when it made sense to be doing that.
Another worry I have about the metacrisis framing is that it, by default, it seems to imply that we should think of all these threats as being on par, when that increasingly doesn’t seem to actually be the case.
Amina: This all feels impenetrable to me. They use a bunch of jargon and a lot of it sounds floaty, vague or mental.
Diego: Yea I get that too. It reminds me of how I felt the first time I encountered EA.I felt this response leaned a bit popularist. I think it’s pretty clear that conversation in sensemaking space is much less precise than EA/rationality on average. The flip side of the coin is that the sensemaking space is open to ideas that would be less likely to resonate in EA/rationality. Whether this trade-off is worth it comes down to factors like how valuable these ideas tend to be, how valuable it is to avoid incorrectly adopting confused beliefs vs. incorrectly rejecting fruitful ideas and the purpose of the movement.
FWIW, I was a lot more positive on the sensemaking space back in the day, now I’m a lot more uncertain. I think there’s a lot of fruitful ideas there, but I’m not convinced that the scene has the tools that it needs to identify which ideas are or aren’t fruitful.
As soon as you start charging a fee to be a member, people will become suspicious that you’re trying to sign them up because you want their cash, rather than being purely dedicated to charity. It’ll also cost you members because they’ll have the choice to either spend the money or not feel fully part of the community. This effect will be worse in some countries than others.
If you don’t get enough members signing up, then the organisation becomes vulnerable to capture by someone who signs up a bunch of straw members who never show up to meetings except to vote once. You can reduce this by adding rules around attendance at meetings, but then this makes it more formal and may cost you more members, especially if people are used to groups being much more casual.
Counterpoint: This seems to work all right for university clubs.
An Easily Overlooked Post on the Automation of Wisdom and Philosophy
I personally haven’t updated too much based off this example, as I suspect this model works better in Norway than it would elsewhere.
Nothing specific. It just seems like it would make sense for them to write an explanation of why they’re doing this.
I’m a bit more skeptical, but this sounds fascinating. Would love to hear more.
I haven’t made a list of existing projects yet, but I hope to do this at some point.
Summary: “Imagining and building wise machines: The centrality of AI metacognition” by Johnson, Karimi, Bengio, et al.
Reflections on AI Wisdom, plus announcing Wise AI Wednesdays
Potentially Useful Projects in Wise AI
EA used to lean more into moral arguments/criticism back in the day, but most folks, even those who were part of the movement back in the day, seem to have leaned away from this.
It’s hard to say why exactly, but being confrontational is unpleasant and it’s not clear that it was actually more effective. OGTutzauer makes a good point that a movement trying to raise donations has more incentive to leverage guilt, whilst a movement trying to shift people’s careers has more incentive to focus on being appealing to be part of.
It might also be partly due to the influence of rationalist culture norms, whilst Moral Ambition seems to have been influenced by both EA and progressivism. (My experience has been that the animal welfare folks, who tend to lean more into progressivism, are most likely to lean into confrontationalism).
“Prominence” isn’t static.
My impression is that Holly has intentionally sacrificed a significant amount of influence within EA because she feels that EA is too constraining in terms of what needs to be done to save humanity from AI.So that term would have been much more accurate in the past.
You might want to try AI Safety Quest.