aka G Gordon Worley III
Gordon Seidoh Worley
Karma: 1,813
Would there be enough activity to justify an SE? Seems like an area where we might quickly run out of questions but want to spend a lot of time finding better answers to old questions, which I’m not sure fits the SE format.
Reading between the lines here, are you saying ACE may not be living up to EA standards given this and other recommendations it has made?
I think the challenge with a project like this is that it is not ‘neutral’ in the way most EA causes are.
Most EA causes I can think of are focused on some version of saving lives or reducing suffering. Although there may be disagreement about how to best save lives or reduce suffering (and what things suffer), there is almost no disagreement that we should save lives and reduce suffering. Although this is not a philosophically neutral position, it’s ‘neutral’ in that you will find a vanishingly small number of people who disagree with the goal of saving lives and reducing suffering.
To put it another way, it’s ‘neutral’ because everyone values saving lives and reducing suffering so everyone feels like EA promotes their values.
Specific books, unless they are complete milk-toast, are not neutral in this way and implicitly promote particular ideas. Much of introductory EA literature, if nothing else, assumes positive act utilitarianism (although within the community there are many notable voices opposed to this position). And if we move away from EA books to other books we think are valuable, they are also going to drift further from ‘neutral’ values everyone can get behind.
This is not necessarily bad, but it is a project that doesn’t seem to fit well to me with much of the EA brand because whatever impact it has will have to be measured in terms of values not everyone agrees with.
For example, lots of people in the comments list HPMOR, The Sequences, or GEB. I like all of these a lot and would like to see more people read them, but that’s because I value the ideas and behaviors they encourage. You don’t have to look very far in EA though to find people who don’t agree with the rationalist project and wouldn’t like to see money spent on sending people copies of these books.
In a position like that, how do you rate the effectiveness of such a project? The impact will be measured in terms of value transmission around values that not everyone agrees on spreading. Unless you limit yourself to books that just promote the idea that we can save lives, reduce suffering, and be a little smarter about how we go about that, I think you’ll necessarily attract a lot of controversy in terms of evaluation.
I’m not saying I’m not in favor of people taking on projects like this. I just want to make sure we’re aware it’s not a normal EA project because the immediate outcome seems to be idea transmission and it’s going to be hard to evaluate what ideas are even worth spreading.
One thing I find meta-interesting about s-risk is that s-risk is included in the sort of thing we were pointing at in the late 90s before we started talking about x-risk, and so to my mind s-risk has always been part of the x-risk mitigation program but, as you make clear, that’s not how it’s been communicated.
I wonder if there are types of risks for the long-term future we implicitly would like to avoid but have accidentally explicitly excluded from both x-risk and s-risk definitions.
My memory is somewhat fuzzy here because it was almost 20 years ago, but I seem to recall discussions on Extropians about far future “bad” outcomes. In those early days much of the discussion focused around salient outcomes like “robots wipe out humans” that we picked up from fiction or outcomes that let people grind their particular axes (capitalist dystopian future! ecoterrorist dystopian future! ___ dystopian future!), but there was definitely more serious focus around some particular issues.
I remember we worried a lot about grey goo, AIs, extraterrestrial aliens, pandemics, nuclear weapons, etc. A lot of it was focused on getting wiped out (existential threats), but some of it was about undesirable outcomes we wouldn’t want to live in. Some of this was about s-risks I’m sure, but I feel like a lot of it was really more about worries over value drift.
I’m not sure there’s much else there, though. We knew bad outcomes were possible, but we were mostly optimistic and hadn’t developed anything like the risk-avoidance mindset that’s become relatively more prevalent today.
I’m going to try to explain here why I am suspicious of the need for this.
People who do things are not, in general, idea constrained from what I can tell. Lots of people have lots of ideas about what they could do and there are already people making arguments for and against these ideas in public forums. People who choose to act do so based in part of how these discussions of ideas influence their thinking, but filtered through the lens of experience at making stuff happen.
Additionally, we already have a lot of ideas people recognize as being worth implementing that no one is working on or work being done on them has not yet come to fruition. It doesn’t take long, relative to the effort that will be invested to do something, to read and think enough to decide what to do, so it seems more likely to me that on the margin we need more desire to do than more curation of ideas about what to do.
All this said, if you want to do something I think there is something to be done in terms of curating the list of ideas/projects you want to see people know about and promoting the existence of that list. Or writing about specific ideas/projects you think people should work on and trying to convince folks they should work on those. But an idea directory of the sort you propose sounds to me like a lot of make-work to see only slightly more clearly the landscape doers are already navigating.
Maybe I’m arguing that we should develop recruiting ideas?
Yep :-)
Also- any suggestions for good formal discussions of the philosophy and sociology of ideas (beyond the slightly nauseating pop business literature)? “Where Good Ideas Come From” by Steven Johnson is excellent, but not philosophically rigorous.
I don’t but I suspect some folks around here do. Talk to Malcolm Ocean maybe?
I recently wrote about if generic feedback processes might produce suffering. I’m working on a follow up post now so interested especially in things I didn’t address that people would like to see addressed.
My guess is that it probably makes sense to keep the brands separate even if they are intertwined. That does mean there may be a lot of cross posting or posting things in one location when they would have done better in the other. Unless LW 2.0 has some plans I’m unaware of to support branded sub-groups so EA could have its own identity on the LW platform.
I like this phrasing, but maybe not for the reason you propose it.
“Doing the most good” leaves implicit what is good, but still uses a referent (“good”) that everyone thinks they know what it means. I think this issue is made even clearer if we talk about “optimizing Earth” instead since optimization must always be optimizing for something. That is, optimization is inherently measured and is about maximization/minimization of some measure. Even when we try to have a generic notion of optimal we still really mean something like effective or efficient as in optimizing for effectiveness or optimizing for efficiency.
But if EA is about optimizing Earth or doing the most good, we must still tackle the problem of what is worth optimizing for and what is good. You mention impact, which also sounds a lot to me like some combination of effectiveness and productivity multiplied by effect size, yet when we are this vague that makes EA more of a productivity movement and less of a good doing movement, whatever we may think good is. The trouble is that, exposing the hollowness of ethical content in the message, it makes it unclear what things would not benefit from being part of EA.
To take a repugnant example, if I thought maximizing suffering were good, would I still be part of EA since I want to optimize the Earth (for suffering)?
The best attempt at dealing with this issue has, for me, been Brian Tomasik’s looks at dealing with moral multiplicity and compromise.
Perhaps the problem is not enough people take the idea seriously, which seems strange to me because so many people take anthropics seriously and, from my perspective, they both work by the same mechanism.
I’ll add that considerations here should probably focus on measure since given that we don’t find ourselves in the a timeline where impossible things have happened we should expect that a thing’s measure can and does go to zero so that “quantum death” can occur.
I also suspect that most people are not familiar enough with the physics to have much to say here since much of it depends on developing a metaphysical interpretation of evidence that is still being actively debated.
Thanks for writing this. I don’t think I’ve seen anyone tell the story quite so well, and I was there for all of it!
Prioritization Consequences of “Formally Stating the AI Alignment Problem”
I think the ELI5 on AI alignment is the same as it has been: make nice AI. Being a little more specific I like Russell’s slightly more precise formulation of this as “align AI with human values”, and being even more specific (without jumping to mathematical notation), I’d say we want to design AI that value what humans value and for us to believe these AI share our values.
Maybe the key thing I’m trying to get at though is that alignable AI will be phenomenally conscious, or in ELI5 terms as much people as anything else (humans, animals, etc.). So then my position is not just “make nice AI” but “make nice AI people we can believe are nice”.
For what it’s worth I started out being very much in the analytic philosophy camp and thought qualia sounded like nonsense for a long time because much of the discussion of the idea avoids giving a precise description of what qualia are. But over time I switched sides, if you will, because I was forced into it by trying to parsimoniously explain reality with empiricist epistemology. For this reason I generally prefer to talk about noemata (a term I gave technical meaning to avoid confusion with existing ideas) rather than qualia for this reason: it avoids the way “qualia” has become associated with all kinds of confusion.
Avoiding AI Races Through Self-Regulation
I think you are conflating EA with utilitarianism/consequentialism. To be fair this is totally understandable since many EAs are consequentialists and consequentialist EAs may not be careful to make or even see such a distinction, but as someone who is closest to being a virtue ethicist (although my actual metaethics are way more complicated) I see EA as being mainly about intentionally focusing on effectiveness rather than just doing what feels good in our altruistic endeavors.
An SRO might incentivize participation in several ways. One is idea sharing, be it via patent agreements or sharing of trade secrets among members on a secure forum. Another is via social and possibly legal penalties for non-participation, or by acting as a cartel to lock non-participants out of the market the way many professional groups do.
That said it does seem a step in the direction of legal oversight, but moves us towards a model similar to so-called technocratic regulatory bodies rather than one where legislation tries to directly control actions. Creating an SRO would give us an already-existing organization that could step in to serve this role in an official capacity if governments or inter-governmental organizations choose to regulate AI.
I’ve so far not seen it, but is anyone taking a broader axiological approach to AI alignment rather than a decision theory specific approach? Obviously the decision theory approach is a more bounded problem that is likely easier to solve since it is a special case of dealing with processes we can apply decision theory to, but I wonder if we might not gain insights and better intuitions from studying more general cases.