Ex particle physicist and AI safety guy.
Now I run the integral altruism network.
EuanMcLean
Karma: 353
I suspect part of what is happening is that systems change advocates are not judging their interventions purely on an individualist consequentialist calculus. If they were purely motivated by a belief that, say, starting a proto-B or intentional community is going to Solve The Metacrisis, I would agree that this is extremely unlikely making the intervention weak AF.
But seeing it as part of a correlated ecosystem of interventions might make more sense. I’m modelling systems change folks as taking a bet that the general direction they’re going in is correct enough that many others will independently (or somewhat dependently through engaging with metacrisis literature) reach similar conclusions and do similar things, resulting in emergent larger-scale changes. (there’s a good chance I am modelling metacrisis folk wrong)
FWIW this doesn’t feel extremely different to longtermist cause areas like AI safety to me. AI safety is also an ecosystem of interventions (technical work + advocacy + governance + education + philosophy + …), if it works it’ll likely be due to some complicated combination of these that the individual theories of change didn’t fully capture. If an individual or group tell me that they are going to single-handedly save the world from unaligned AI, that is a red flag for me, because the system of AI development is more complex than an individual/group can reckon with IMO.
Thanks Tandena, I appreciate this poke! The following is my counter-poke.
They mostly obfuscate them by disallowing comparison and avoid failure by never choosing between options.
We are not interested in disallowing comparison and never choosing between options. Our move is to be aware of more options to choose from. To integrate x and y is not to never chose between them, but to have both at your disposal so you can apply either x or y depending on which works better in a given context.
One way we are distinct from EA is that we are explicitly striving to be metarational, which means recognizing that the best frame, idea, tool, or action is context-dependent. Dependent on the context of the individual altruist, the problem they are working on, and a plethora of other factors.
I claim that the current explicit tools & implicit approaches of EA are not the best suited for all contexts that changemakers today can find themselves in.
For example, take the ‘Decoupling & Recoupling’ principle. Taking the most zoomed-out frames (e.g. metacrisis) benefits understanding but hurts tractability, and zoomed-in frames has the opposite tradeoff. We want to consciously choose this tradeoff depending on a judgement of which is better given what we’re doing, rather than only being aware of the zoomed in frame.
int/a as a superstructure is not committing to x or y, because there is no universal truth of whether x or y is correct, it depends on the person and the problem. But this does not prevent individuals or projects within int/a from committing to x or y, we want to empower individuals & projects with x and y so they can discern for themselves whether x or y is right for their context.
And it is not preventing the possibility of a synthesis of x and y leading to novel concrete directions for solving the world’s most complex & pressing problems.
So yea, I appreciate that “staying in the ideation phase” is a possible failure mode, but I don’t consider it intrinsic to the project.
(edit: looking at this again, this is not the only way to interpret ‘integrate’, and probably it would be cool to get more clear about what we mean by the word. For example, another angle on integrating x and y could be to find a specific action z that is robust to both x and y being true)
I wouldn’t consider the metacrisis movement to have any wins yet (according to the EA standard of wins at least), and its hard to point at very concrete proposed solutions yet. It’s mostly in ‘diagnosis mode’.
The main directions of solutions I’m aware of are things like Perspectiva’s antidebates, which are meant to improve collective epistemics (a layer of the metacrisis) (which was also adapted recently by Liv Bouree in her recent AI safety antidebate), or Life Itself’s Developmental Spaces, which is trying to catalyze culture change (another layer of the metacrisis)(Life Itself’s more broad proposed strategy is here). You could consider Schmachtenberger’s third attractor the beginnings of a proposed solution.
Most/all of the metacrisis discourse hasn’t taken short AI timelines into account, so the paths to impact of the proposed solutions tend to assume we have a bunch of time.
All the metacrisis stuff could do with more direction, urgency and clarity IMO, and part of my motivation for building this bridge between them and EA is that hopefully injecting some EA energy might make concrete progress more likely to happen.
It’s worth restating what I said in the post here:
We intend the presentation here to be descriptive rather than convincing—arguing for the merits of these principles is beyond the scope of this post.
And
The principles are also currently somewhat abstract, in the future we hope to translate these to be more concrete & action-guiding.
Getting into more detailed arguments of what EA is missing and precisely what they should do differently is quite a big project due to the inferential difference between EA and liminal land, and one that I hope int/a can attempt in the future.
Thanks Richard!
On the ‘hippies have too much agreeableness’ point—yes, you are totally right!!
On the ‘pinning down core int/a claims’ point. I agree that in general getting more precise about claims is good. But I have some caution around pushing to generate precise object-level claims that “define int/a”, in that you have to believe these claims to be part of it. One thing I feel towards EA is that it used to be about “the question” (how to do the most good), and created room for people to generate new answers to that question, but more recently it has become about “the answer” (this short list of career paths is how to do the most good). But I don’t think the cultural/structural locking in of those answers is good because we might be missing crucial considerations that will only become clear in the future.
Thanks for the thoughtful response Elliot!
On point 2 - yes, it is a fair criticism of both int/a and the sensemaking folks that what we’re saying feel indirect. The challenge as I see it is that the things the sensemaking world are pointing at are just a lot harder to put in very explicit terms. That doesn’t necessarily mean stuff like the metacrisis doesn’t exist, it could just mean that its harder to point at/analyze/get traction on.
I’ve heard metacrisis people describe EA as ‘searching for the keys under the lamppost’, in that EA focuses on the things that can be explicitly stated and modeled, which is not necessarily the same as the set of problems that exist. They would argue that instead of continuing to search under the lamppost, maybe we should build new lampposts, or buy a torch, or whatever. I don’t fully condone this but it’s a good intuition pump for where they’re coming from.
Part of int/a’s ambition in building this bridge is to try to caste sensemaking ideas in more direct EA-brained terms (like this rough first attempt), but it’s tough and a work in progress!
On point 3 - sure, a lot of what we talk about here is already in the EA discourse to varying degrees. I think the distinction is the degree in which the value is emphasized and practiced. For example, the element of ‘personal fit’ is a meme that exists within EA, but in the 80k guide feels like a footnote. In contrast, in int/a we have an intention for personal fit to be quite central and have it inform the structure and emergent behavior of the movement.
On point 4 - yea great point. Ultimately it would be cool to examine whether int/a makes sense on both EA territory and other territories.
Thanks Tobias, some good threads to pull here!
Yes, the question of whether int/a is a subset of EA, overlapping, or something totally different has been a big point of discussion, and we haven’t found a clean answer.
You are right that EA in some sense already contains a lot of the things int/a is excited about (especially in terms of the official written principles being quite broad), but perhaps the real difference is what is emphasized in practice.
For example:
Effective altruism doesn’t take a position on whether we are in conflict with the natural unfolding of the universe.
Yea, EA doesn’t explicitly say anything about that, but what we’re pointing at is perhaps a cultural or semi-conscious current that pervades a lot of EA work (possibly this is more relevant to rationalism than EA). This line was inspired by in part by Joe Carlsmith’s An Even Deeper Atheism, which points out a current underlying a lot of EA/rat/AI safety that is born out of a deep mistrust of everything (might not be doing the essay justice but that’s the general direction).
I’m not necessarily saying this current is bad, rather that we should have an awareness of it and be able to step outside of that frame of mind when it is not helping us, and integrate different frames. The hope is that int/a can more explicitly/consciously find the right balance between the yang-y mistrusting the universe vibe and the yin-y trusting the universe vibe.
Obviously a line by line response to your line by line response to my line by line response to your article would be somewhat over the top. So I’ll refrain!
Yes we could waste our lives falling down a hole here.
But then you also respond to most of (not all) my points without actually giving a counter-argument, just claiming that I’m clearly mistaken.
Huh. I must have messed up the tone of my last message because that wasn’t the intention at all. For some of my responses I thought I was basically agreeing with you, and others I was clarifying what I (or rather Diego) was trying to say rather than saying you are wrong.
Thanks for your comment Thom.
In my response to this, I really want to avoid this vibe of “you just don’t understand” or “you need to wake up” and all that kind of thing. I know how annoying and unproductive that kind of response can be. In what I write below I’m not trying to just assert that my position is obviously more right, if only you could see through the matrix. I’m interested in clarifying the metacrisis position/how metacrisis folk think, as maybe if it becomes clear enough it will no longer seem obviously stupid to you!
That being said, the standard metacrisis-y response to most of what you’ve said here would be to try and pick apart the (modernist-y) assumptions that might lie underneath it. Being contextualizing isn’t just about awareness of the context of the system you’re studying, but also the context of your own mind/rationality/beliefs – all the quite deep assumptions driving those beliefs that work well in a number of contexts (like building bridges, training LLMs, or saving kids from malaria) but might not generalize as well as you think to statements about a rapidly evolving and highly interconnected world.
I’ll pick out a couple of examples to pick on from your comment:
> Modernity has led to the mental health crisis: I’m just not sure this empirically stacks up. It is really hard to measure mental health over time, given that its measurement is so culturally contingent
I want to pick on this because I think it’s a good example of what the vibey crowd like to call “scientism”, which in my head means something like the view that “the only possible way to know something to be true is if it’s published in a peer-reviewed journal”. That is obviously one of the most reliable ways to know something is true, but it limits your toolkit for making sense of the world.
In the case of modernity leading to a mental health crisis: yes, you’re right that it’s a very hard thing to measure, and therefore no signal has been found by any study. But when you include lived experience… I don’t know man, it just feels so true, at least in some sense, from my own life and the lives of so many around me. For example, there’s no signal that shows that social media can mess up teenagers’ mental health, but this exchange we’re having in the comments of the EA forum is making my muscles tense and my mind race, and when I extrapolate this to being a teenager and dealing with high stakes things like how I look or who fancies me, this is some good evidence for that causation in my book.
Sometimes I think of EA as the “analytical philosophy of changemaking” while metacrisis is the “continental philosophy of changemaking”, since in analytical philosophy something is true because you can prove it to be true, while in continental philosophy something is true because it feels true. We need both.
> Growth has historically been the single biggest driver of human welbeing.
So… how are you defining human wellbeing here? In terms of stuff you can measure (life expectancy, economic prosperity, etc), yea there’s no argument, you’re right. But all the other things that contribute to a broader definition of wellbeing? Community, meaning, connection to nature, etc? Wellbeing is a complex beast. I don’t have the arguments or the data to say you’re wrong, but you’re saying it here without any argument as if this is obviously right.
You may well be right even in the broader sense of the word wellbeing, but a metacrisis person might also say that once you’ve optimised hard enough for economic growth, goodhart’s law bites you in the arse and growth starts to decorrelate from wellbeing. Some might argue that that is starting already (see, for example, Scandinavia scoring best on various happiness metrics while having a smaller economy than, say, the US).
To reiterate something I said in the post, I’m aware that if you’re constantly questioning everything and constantly having to keep all the mystery of the definitions of all your terms you’re using in your attention, you’ll never get anything done. Getting things done is good. But if you commit to some metric and then never revisit the assumptions behind it, you might find yourself getting the wrong things done when your optimisation pushes the world out of the regime in which your metric makes sense.
> The idea that rivalry (caused by human nature) is a background assumption and not necessarily the case: the point here surely is that, yes of course humans can be more or less cooperative at times and given different cultural assumptions, but this kind of game theory describes dynamics that are independent of how most people behave.
This feels like a statement about the strength & generality of game theory when applied to humans on various scales. The metacrisis nerds would probably try to poke at how much confidence you have in game theory to support the claim that rivalry will always arise. This kind of background acceptance in game theory has a modernist vibe.
Anyway, I don’t expect I’ve changed your mind on any of this, which is fine! Even if we don’t agree it’s good to more deeply understand each other’s position. Ok bye
fixed, thanks!
This qualitative survey of AI safety experts I just finished, I think it might be a useful resource for people just starting their career in AI safety! https://www.lesswrong.com/s/xCmj2w2ZrcwxdH9z3
Should we be as confident in the ‘answers that EA has settled on’ as a science teacher is in how a cell works?