Mau

Karma: 2,794

Mau Aug 19, 2022, 12:50 AM
8 points
0 ∶ 0
on: What does moral progress consist of?
Thanks for the thoughtful post!

Some of the disconnect here might be semantic—my sense is people here often use “moral progress” to refer to “progress in people’s moral views,” while you seem to be using the term to mean both that and also other kinds of progress.

Other than that, I’d guess people might not yet be sold on how tractable and high-leverage these interventions are, especially in comparison to other interventions this community has identified. If you or others have more detailed cases to make on the tractability of any of these important problems, I’d be curious to see them, and I imagine others would be, too. (As you might have guessed, you might find more ears if you argue for relevance to x-risks, since the risk aversion of global health and development parts of EA seems to leave them with little interest in hard-to-measure interventions.)

Mau Aug 15, 2022, 12:42 AM
4 points
0 ∶ 0
in reply to: Michael St Jules 🔸’s comment on: Prioritizing x-risks may require caring about future people
Good points!

Some GiveWell charities largely benefit young children, too, but if I recall correctly, I think donations have been aimed at uses for the next year or two, so maybe only very young children would not benefit on such a person-affecting view, and this wouldn’t make much difference.

Agreed that this wouldn’t make much of a difference for donations, although maybe it matters a lot for some career decisions. E.g. if future people weren’t ethically important, then there might be little value in starting a 4+ year academic degree to then donate to these charities.

(Tangentially, the time inconsistency of presentists’ preferences seems pretty inconvenient for career planning.)

Mau Aug 11, 2022, 7:30 AM
12 points
0 ∶ 0
on: Against longtermism
Thanks for writing—I skimmed so may have missed things, but I think these arguments have significant weaknesses, e.g.:
- They draw a strong conclusion about major historical patterns just based on guesswork about ~12 examples (including 3 that are explicitly taken from the author’s imagination).
- They do not consider examples which suggest long-term thinking has been very beneficial.
  - E.g. some sources suggest that Lincoln had long-term motivations for permanently abolishing slavery, saying, “The abolition of slavery by constitutional provision settles the fate, for all coming time, not only of the millions now in bondage, but of unborn millions to come—a measure of such importance that these two votes must be procured.”
- As another comment suggests, the argument does not consider ways in which our time might be different (e.g. unusually many people are trying to have long-term impacts, people are less ignorant, tech advances may create rare opportunities for long-term impact).

Mau Aug 8, 2022, 11:47 PM
5 points
0 ∶ 0
on: Classifying sources of AI x-risk
Thanks for posting! Tentative idea for tweaks: my intuition would be to modify the middle two branches into the following:
- Long-term AI misuse
  - Stable authoritarianism
  - Value erosion from competition
  - “Lame” future
- AI exacerbates other x-risk factors
  - (Great power / nuclear) conflict
  - Degraded epistemics
  - Other dangerous tech
Rationale:
- “Other (AI-enabled) dangerous tech” feels to me like it clearly falls under “exacerbating other x-risk factors”
- “AI-enabled dystopia” sounds broad enough to cover ~everything on the chart; “long-term AI misuse” might more precisely capture the thought there
- “Authoritarianism” rather than “totalitarianism,” because the latter is often/historically defined as involving features more specific than the extent of the risks here (e.g. large political parties)
- Stable authoritarianism admittedly sorta falls under both “long-term AI misue” and “AI exacerbates other x-risk factors.” My intuition is to put it in the former bucket, because extremely stable authoritarianism doesn’t seem very plausible if it’s not AI-enabled (in that sense, stable authoritarianism doesn’t seem like an “other” x-risk factor).
+1 for not including ~”misaligned, non-power-seeking AI”; that seems to be a somewhat common misinterpretation of some AI concerns.

Edit: good point in the below response!

Mau Aug 6, 2022, 6:01 AM
46 points
0 ∶ 0
on: Being the person who doesn’t launch nukes: new EA cause?
I’d consider tweaking (3) to something like, “Make sure you don’t start a nuclear war based on a false alarm.” The current version has imo some serious downsides:
- Association with planned disloyalty could hurt the efforts of everyone else in this community who’s trying to contribute to policy through dedicated service.
- A (US) community’s reputation for planned non-retaliation might increase risk of nuclear war, because nuclear peace depends partly on countries’ perceptions that other countries are committed to retaliating against nuclear attack.

Mau Jul 28, 2022, 5:19 AM
3 points
0 ∶ 0
on: How long does it take to undersrand AI X-Risk from scratch so that I have a confident, clear mental model of it from first principles?
I don’t have a great sense of how long what you’re describing would take, but here is a collection of relevant resources.

Mau Jul 22, 2022, 2:51 AM
6 points
0 ∶ 0
in reply to: iamasockpuppet’s comment on: EA Probably Shouldn’t Try to Exercise Direct Political Power
- Is the idea that there isn’t already opposition to EA stances, so creating it is extra bad?
- On the flip side of one point, obscurity of these cause areas also reduces cause-area-motivated opponents.

Mau Jul 22, 2022, 12:29 AM
10 points
0 ∶ 0
in reply to: Charlie_Guthmann’s comment on: Leaning into EA Disillusionment
I agree with a lot of this, although I’m not sure I see why standardized cost benefit analysis would be necessary for legitimate epistemic progress to be made? There are many empirical questions that seem important from a wide range of ethical views, and people with shared interest in these questions can work together to figure these out, while drawing their own normative conclusions. (This seems to line up with what most organizations affiliated with this community actually do—my impression is that lots more research goes into empirical questions than into drawing ethical conclusions.)

And even if having a big community were not ideal for epistemic progress, it could be worth it on other grounds, e.g. community size being helpful for connecting people to employers, funders, and cofounders.

Mau Jul 21, 2022, 10:03 PM
8 points
0 ∶ 0
on: EA Probably Shouldn’t Try to Exercise Direct Political Power
I think you have good points around partisanship, comparative advantage, and weaknesses of some arguments on the forum. Two other thoughts:
- Your analysis of the limited influence of individual elected officials focuses on the House of Representatives. These arguments are grounded in the House’s unique features (e.g., influence of leadership, limits on members’ ability to propose amendments), so it doesn’t make sense to generalize their conclusions to the Senate.
- Concluding one of your sections, you write, “they inevitably will end up campaigning on, and dealing with, primarily non-EA topics. This means that most opposition that EA politicians would face isn’t due to their EA stances.” So? I’m not seeing how this is an argument for your main claim.

Mau Jul 18, 2022, 9:29 AM
2 points
0 ∶ 0
in reply to: Ofer’s comment on: Slowing down AI progress is an underexplored alignment strategy
I’m still figuring out how I want to engage on this forum; for now, I generally, tentatively prefer to not disclose personal information on here. I’d encourage readers to conservatively assume I have conflicts of interest, and to assess my comments and posts based on their merits. (My vague sense is that this is a common approach to this forum—common enough that non-disclosure doesn’t imply an absence of conflicts of interest—but maybe I’ve misread? I’m not confident about the approach I’m taking—feel free to message me on this forum if you’d like to discuss this further.)

On your other point, I agree that suspicion toward self-regulation is often warranted; I think my earlier point was sufficiently hedged (“plausible”; “some”) to be compatible with such suspicion.

Mau Jul 17, 2022, 8:58 PM
2 points
0 ∶ 0
in reply to: Ofer’s comment on: Slowing down AI progress is an underexplored alignment strategy
I’m mostly sympathetic—I’d add a few caveats:
- Research has to slow down enough for an AI developer to fall behind; an AI developer that has some lead over their competition would have some slack, potentially enabling safety-concerned people to contribute. (That doesn’t necessarily mean companies should try to get a lead though.)
- It seems plausible for some useful regulation to take the form of industry self-regulation (which safety-concerned people at these companies could help advance).

Mau Jul 17, 2022, 7:52 AM
2 points
0 ∶ 0
in reply to: Michael St Jules 🔸’s comment on: Confused about “making people happy” vs. “making happy people”
Good points, and thanks for the example! That all seems right. I’ve been assuming that it didn’t matter whether the option sets were all available at once, but now I see that amounts to assuming IIA.

Mau Jul 17, 2022, 5:50 AM
2 points
0 ∶ 0
in reply to: Michael St Jules 🔸’s comment on: Confused about “making people happy” vs. “making happy people”
Wide person-affecting views that solve the nonidentity problem can address points 2 and 3.

That seems right. My comment mainly wasn’t intended as a response to these views, although I could have made that clearer. (If I understand them correctly, wide person-affecting views are not always indifferent to creating happy people, so they’re outside the scope of what the original post was discussing.) (Edit: Still, I don’t yet see how wide person-affecting views can address point 2. If you feel like continuing this thread, I’d be curious to hear an example of a wide person-affecting view that does this.)

Re: relation R, good point, we can do that, and that seems much less bad than self-contradiction. (Editing my earlier comment accordingly.) Still, I think the extra complexity of this view loses points via simplicity priors (especially if we tack on more complexity to get relation R to exclude inheritance relations that intuitively “increase population,” like reproduction—without excluding those, plausibly we’ve gone back to valuing making happy people). (The extra complexity of valuing consciousness / happiness also loses points by the same reasoning, but I’m more willing to bite that bullet.)

On your last point, I’m not sure I see yet how person-affecting views can avoid intransitivity. Let’s say we have:
- World A: Happy Bob
- World B: Sad Bob
- World C: No one
Wouldn’t ~all person-affecting views hold that A ~ C and C ~ B, but A > B, violating transitivity? (I guess transitivity is usually discussed in the context of inequalities, while here it’s the indifference relation that’s intransitive.)

Mau Jul 16, 2022, 7:54 PM
12 points
0 ∶ 0
on: Confused about “making people happy” vs. “making happy people”
Thanks for posting! I agree this is far from obvious at first glance. Here are most of the main intuitions that together make me very skeptical of “care about making people happy, not happy people”:
- The reasoning you use would also support conclusions that to me feel very unintuitive. You could just as easily argue, “creating more humans doesn’t really matter morally, even if they would be utterly miserable, because the unborn can’t care about being born.”
- Indifference to the well-being of potential people would imply that many people don’t matter (in certain decisions). My starting assumption is heavy skepticism toward any view that advocates indifference to the well-being of many people, since historically, such views have horrific track records.
- Arguably, personal identity does not persist over time. For example, I will not exist in a decade or even in a second; instead, I will grow into someone who is similar to me, but not quite the same person. So whenever we “make people happy,” we are creating new happy people who are slightly different from whoever would have existed otherwise. If that’s the case, then making people happy is a way of making happy people. So it’s contradictory [edit: or requires additional, complex assumptions] to value making people happy if we never value making happy people.
- If we care about making people happy but are indifferent to making happy people, this implies intransitive preferences [edit: or non-independence of irrelevant alternatives], which I find very unintuitive.
So the view that making people happy matters but making happy people doesn’t is asymmetric (or unintuitive), frequently indifferent, self-contradictory [edit: or additionally complex], and intransitive [edit: or not independent of irrelevant alternatives].

Mau Jul 13, 2022, 10:37 AM
24 points
0 ∶ 0
on: Slowing down AI progress is an underexplored alignment strategy
(Crossposting, with tweaks)

Thanks for the post—I think (unoriginally) there are some ways heavy regulation of AI could be very counterproductive or ineffective for safety:
- If AI progress slows down enough in countries were safety-concerned people are especially influential, then these countries (and their companies) will fall behind internationally in AI development. This would eliminate much/most of safety-concerned people’s opportunities for impacting AI’s trajectory.
- If China “catches up” to the US in AI (due to US over-regulation) when AI is looking increasingly economically and militarily important, that could motivate US policymakers to hit the gas on AI (which would at least undo some of the earlier slowing down of AI, and might spark an international race to the bottom on AI).
Also, you mention,

The community strategy (insofar as there even is one) is to bet everything on getting a couple of technical alignment folks onto the team at top research labs in the hopes that they will miraculously solve alignment before the mad scientists in the office next door turn on the doomsday machine.

From conversation, my understanding is some governance/policy folks fortunately have (somewhat) more promising ideas than that. (This doesn’t show up much on this site, partly because these professionals tend to be busy and the ideas are fairly rough.) I hear there’s some work aimed at posting about some of these ideas—until then, chatting with people (e.g., by reaching out to people) might be the best way to learn about these ideas.

Mau Jul 8, 2022, 1:15 AM
7 points
0 ∶ 0
in reply to: Jacy’s comment on: The Future Might Not Be So Great.
I think we’re on a similar page regarding double-counting—the approach you describe seems like roughly what I was going for. (My last comment was admittedly phrased in an overly all-or-nothing way, but I think the numbers I attached suggest that I wasn’t totally eliminating the weight on history.)

On whether we see “reasons for negative weight” differently, I think that might be semantic—I had in mind the net weight, as you suggest (I was claiming this net weight was 0). The suggestion that digital minds might be affected just by their being different is a good point that I hadn’t been thinking about. (I could imagine some people speculating that this won’t be much of a problem because influential minds will also eventually tend to be digital.) I tentatively think that does justify a mildly negative weight on digital minds, with the other factors you mention seeming to be fully accounted for in other weights.

Mau Jul 1, 2022, 1:16 AM
8 points
0 ∶ 0
in reply to: UwU’s comment on: The Future Might Not Be So Great.
More specifically, I think there’s a good case to be made* that most of the expected disvalue of suffering risks comes from cooperation failures, so I’d especially encourage people who are interested in suffering risks and AI to look into cooperative AI and cooperation on AI. (These are areas mentioned in the paper you cite and in related writing.)

(*Large-scale efforts to create disvalue seem like they would be much more harmful than smaller-scale or unintentional actions, especially as technology advances. And the most plausible reason I’ve heard for why such efforts might happen is that: various actors might commit to creating disvalue under certain conditions, as a way to coerce other agents, and they would then carry out these threats if the conditions come about. This would leave everyone worse off than they could have been, so it is a sort of cooperation failure. Sadism seems like less of a big deal in expectation, because many agents have incentives to engage in coercion, while relatively few agents are sadists.)

(More closely related to my own interest in them, cooperation failures also seem like one of the main types of things that may prevent humanity from creating thriving futures, so this seems like an area that people with a wide range of perspectives on the value of the future can work together on :)

Mau Jul 1, 2022, 12:38 AM
7 points
0 ∶ 0
in reply to: Jacy’s comment on: The Future Might Not Be So Great.
Thanks! Responding on the points where we may have different intuitions:
- Regarding your second bullet point, I agree there are a bunch of things that we can imagine having gone differently historically, where each would have been enough to make things go better. These other factors are all already accounted for, so putting the weight on historical harms/progress again still seems to be double-counting (even if which thing it’s double-counting isn’t well-defined).
- Regarding your third bullet point, thanks for flagging those points—I don’t think I buy that any of them are reasons for negative weight.
  - Intrusions could be harmful, but there could also be positive analogues.
  - Duplication, instrumental usefulness, and nested minds are just reasons to think there might be more of these minds, so these considerations only seem net negative if we already have other reasons to assume these minds’ well-being would be net negative (we may have such reasons, but I think these are already covered by other factors, so counting them here seems like double-counting)
    (As long as we’re speculating about nested minds: should we expect them to be especially vulnerable because others wouldn’t recognize them as minds? I’m skeptical; it seems odd to assume we’ll be at that level of scientific progress without having learned how experiences work.)
- On interpretation of the spreadsheet:
  - I think (as you might agree) that results should be taken as suggestive but far from definitive. Adding things up fails to capture many important dynamics of how these things work (e.g., cooperation might not just create good things but also separately counteract bad things).
  - Still, insofar as we’re looking at these results, I think we should mostly look at the logarithmic sum (because some dynamics of the future could easily be far more important than others).
  - As I suggested, I have a few smaller quibbles, so these aren’t quite my numbers (although these quibbles don’t really matter if we’re looking at the logarithmic sum).

Mau Jun 30, 2022, 9:25 PM
31 points
0 ∶ 0
on: The Future Might Not Be So Great.
Thanks for this! I agree this is a very important question, I sympathize with the view that people overweight some arguments for historical optimism, and I’m mostly on board with the list of considerations. Still, I think your associated EV calculation has significant weaknesses, and correcting for these seems to produce much more optimistic results.
- You put the most weight on historical harms, and you also put a lot of weight on empirical utility asymmetry. But arguably, the future will be deeply different from the past (including through reduced influence of biological evolution), so simple extrapolation from the past or present should not receive very high weight. (For the same reason, we should also downweight historical progress.)
- Arguably, historical harms have occurred largely through the divergence of agency and patiency, so counting both is mostly double-counting. (Similarly, historical progress has largely occurred through the other mechanisms that are already covered.) So we should further downweight these.
- I don’t see why we should put negative weight on “The Nature of Digital Minds, People, and Sentience.”
- Reasoned cooperation should arguably receive significantly more weight as an argument for optimism; moral trade should allow altruists to substantially mitigate suffering and increase well-being, especially as human tools for shaping the world and efficiently coordinating continue to improve.
- Editing the calculation to account for all of the above (and ignoring my other, more minor quibbles), we reach a fairly optimistic result (especially if we’re looking at the more relevant logarithmic sum).
(Edited to account for how downweighting history should mean downweighting historical progress, not just downweighting historical harms, and for phrasing tweaks.)

Mau Jun 27, 2022, 7:34 PM
5 points
0 ∶ 0
in reply to: constructive’s comment on: US Policy Careers Speaker Series—Summer 2022
Hi, thanks for your question! Unfortunately, there are tricky tradeoffs here—to respect the privacy of speakers and to allow them to speak freely, the talks will not be recorded.