Mau

Karma: 2,791

Mau Sep 13, 2022, 8:05 AM
3 points
1 ∶ 0
in reply to: Fai’s comment on: Could it be a (bad) lock-in to replace factory farming with alternative protein?

my point is that, within the FAW and altpro movements, A is mentioned

Oh interesting, I wasn’t aware this point came up much. Taking your word for it, I agree then that (A) shouldn’t get more weight than (B) (except insofar as we have separate, non-speculative reasons to be more bullish about economic interventions).

I think you kind of changed the “latter argument” a bit here from what we were discussing before.

Sorry for the confusion—I was trying to say that alt-pro advocates often have an argument that’s different (and better-grounded) than (A) and (B).

In other words, my current view is that (A) and (B) roughly “cancel out” due to being similarly speculative, while the separate view that “good, lasting value change is more likely when it’s convenient” is better-grounded than its opposite.

Mau Sep 12, 2022, 12:18 AM
8 points
1 ∶ 1
in reply to: Fai’s comment on: Could it be a (bad) lock-in to replace factory farming with alternative protein?
Thanks for the thoughtful response!

I actually think this paragraph you created is worth presenting and considering. The thing is, it’s pretty much been presented already. This is, for example, roughly the story of Bruce Friedrich (founder and CEO of GFI), and maybe pretty much GFI too. And that was my story too, and might be the story of a lot of EA animal/alt-pro advocates. So if this argument is presented, why not also consider its counterpart? (what I did)

I think this is subtly off. The story I’ve heard from alt-pro advocates is that we should focus on making it easier for people to drop factory farming because that would get people to do so, while generations of moral advocacy against factory farming have failed to achieve mass consumer change. That’s a historical argument about tractability—it’s not a speculative argument about how we might inspire or mislead future advocates.

(To be fair, the above is still not an argument about long-term impacts. But I think the related long-term argument that “good, lasting value change is more likely when it’s convenient” is a much better-grounded claim than “good, lasting value change is more likely when advocates have historical examples of entirely morality-driven change”; the latter claim seems entirely speculative, while the former is at least in line with various historical examples and psychological findings.)

Mau Sep 10, 2022, 10:17 PM
15 points
8 ∶ 0
on: Agree/disagree voting (& other new features September 2022)
I’m not sure how much of a pain this would be implementation-wise (or stylistically), but I’d be curious to see agree/disagree voting for posts (rather than just comments). After all, arguments for having this type of voting for comments seem to roughly generalize to posts, e.g. it seems useful for readers to be able to quickly distinguish between (i) critical posts that the community tends to appreciate and agree with, and (ii) critical posts that the community tends to appreciate but disagree with.

Mau Sep 10, 2022, 10:08 PM
6 points
4 ∶ 1
on: Could it be a (bad) lock-in to replace factory farming with alternative protein?
Thanks for writing! I’m skeptical that a non-morally-motivated ban would create bad value lock-in. Most of this post’s arguments for that premise seem to be just the author’s speculative intuitions, given with no evidence or argument (e.g. ” I also worry that using laws to capture our abolition of moral catastrophes after they become economically inviable, can create a false sense of progress [...] Always waiting for technological changes might mislead us to think that we have less obligation to improve our moral values or actions when the technological/economic incentives are lower.”) But I don’t think ungrounded intuitions about how society might work are good ways to make predictions; there’s too many complications and alternatives that approach might miss.
- As a reason why this kind of argument isn’t reliable, we could just as easily come up with intuitive stories that point to the opposite conclusion, e.g. “economic changes that drive moral progress will inspire and inform future advocates to take pragmatic approaches that actually work well rather than engaging in endless but ineffective moral grandstanding; always waiting for moral progress might mislead us to think we have less obligation to improve economic incentives when the tractability of moral advocacy is lower.”
- Also, I think the historical importance of economic and military motives for the abolition of slavery are understated.
(Edited within a few mins to delete a weaker argument.)

Mau Sep 9, 2022, 3:53 AM
5 points
2 ∶ 0
in reply to: Geoffrey Miller’s comment on: AI alignment with humans… but with which humans?

But it seems like such a narrow notion of alignment that it glosses over almost all of the really hard problems in real AI safety—which concern the very real conflicts between the humans who will be using AI.

I very much agree these these political questions matter, and that alignment to multiple humans is conceptually pretty shaky; thanks for bringing up these issues. Still, I think some important context is that many AI safety researchers think that it’s a hard, unsolved problem to just keep future powerful AI systems from causing many deaths (or doing other unambiguously terrible things). They’re often worried that CIRL and every other approach that’s been proposed will completely fail. From that perspective, it no longer looks like almost all of the really hard problems are about conflicts between humans.

(On CIRL, here’s a thread and a longer writeup on why some think that “it almost entirely fails to address the core problems” of AI safety.)

Mau Sep 7, 2022, 10:59 PM
4 points
0 ∶ 0
in reply to: tamgent’s comment on: AI Governance Needs Technical Work
Thanks for the comment! I agree these are important considerations and that there’s plenty my post doesn’t cover. (Part of that is because I assumed the target audience of this post—technical readers of this forum—would have limited interest in governance issues and would already be inclined to think about the impacts of their work. Though maybe I’m being too optimistic with the latter assumption.)

Were there any specific misuse risks involving the tools discussed in the post that stood out to you as being especially important to consider?

AI Governance Needs Technical Work

MauSep 5, 2022, 10:25 PM

121 points

3 comments8 min readEA link

Mau Sep 5, 2022, 8:56 PM
38 points
0 ∶ 0
on: The Base Rate of Longtermism Is Bad
Thanks for writing this. I think there are actually some pretty compelling examples of people/movements being quite successful at helping future generations (while partly trying to do so):
- Some sources suggest that Lincoln had long-term motivations for permanently abolishing slavery, saying, “The abolition of slavery by constitutional provision settles the fate, for all coming time, not only of the millions now in bondage, but of unborn millions to come—a measure of such importance that these two votes must be procured.” Looking back now, abolition still looks like a great move for future generations.
  - I don’t know how accurate those sources are, but at least a U.S. constitutional amendment is structured to have very long-lasting impacts, given the extreme difficulty of undoing it.
- The U.S. constitution appears to have been partly aiming to create a long-lasting democracy, citing “our posterity” in its preamble. It seems to have largely worked.
- Proponents of measures to avoid nuclear war and reduce nuclear weapons testing often cited future generations as one motivation. (For example, in a famous speech he gave before launching U.S.-Soviet cooperation on limiting nuclear testing and nonproliferation, Kennedy appealed to the importance of “not merely peace in our time but peace for all time” and to “the right of future generations to a healthy existence.”) These efforts have been quite successful; we’ve had about 77 years with no wartime use of nuclear weapons, nuclear testing has plummeted, and far few states than once feared now have nuclear weapons.
[Edited to add] All this looks to me like a mixed (and maybe fairly good overall) track record, not a terrible one. (Though a deeper problem is that we can’t justifiably draw almost any conclusions about base rates from these or the post’s examples, since we’ve made no serious efforts to find a representative sample of historical longtermist efforts.)

Mau Sep 1, 2022, 11:10 PM
4 points
0 ∶ 0
in reply to: titotal’s comment on: Chaining the evil genie: why “outer” AI safety is probably easy
Maybe, I’m not sure though. Future applications that do long-term, large-scale planning seem hard to constrain much while still letting them do what they’re supposed to do. (Bounded goals—if they’re bounded to small-scale objectives—seem like they’d break large-scale planning, time limits seem like they’d break long-term planning, and as you mention the “don’t kill people” counter would be much trickier to implement.)

Mau Sep 1, 2022, 9:31 AM
22 points
0 ∶ 0
on: Reasons for my negative feelings towards the AI risk discussion
I also used to be pretty skeptical about the credibility of the field. I was surprised to learn about how much mainstream, credible support AI safety concerns have received:
- Multiple leading AI labs have large (e.g. 30-person) teams of researchers dedicated to AI alignment.
  - They sometimes publish statements like, “Unaligned AGI could pose substantial risks to humanity and solving the AGI alignment problem could be so difficult that it will require all of humanity to work together. ”
- Key findings that are central to concerns over AI risk have been accepted (with peer review) into top ML conferences.
- A top ML conference is hosting a workshop on ML safety (with a description that emphasizes “long-term and long-tail safety risks”).
- Reports and declarations from some major governments have endorsed AI risk worries.
  - The UK’s National AI Strategy states, “The government takes the long term risk of non-aligned Artificial General Intelligence, and the unforeseeable changes that it would mean for the UK and the world, seriously.”
- There are AI faculty at universities including MIT, UC Berkeley, and Cambridge who endorse AI risk worries.
To be fair, AI risk worries are far from a consensus view. But in light of the above, the idea that all respected AI researchers find AI risk laughable seems plainly mistaken. Instead, it seems clear that a significant fraction of respected AI researchers and institutions are worried. Maybe these concerns are misguided, but probably not for any reason that’s obvious to whoever has basic knowledge of AI—or these worried AI experts would have noticed.

(Also, in case you haven’t seen it yet, you might find this discussion on whether there are any experts on these questions interesting.)

Mau Aug 30, 2022, 10:46 PM
8 points
1 ∶ 0
on: Chaining the evil genie: why “outer” AI safety is probably easy

To counter that, let me emphasize the aspects of AI risk that are not disproven here.

Adding to this list, much of the field thinks a core challenge is making highly capable, agentic AI systems safe. But (ignoring inner alignment issues) severe constraints create safe AI systems that aren’t very capable agents. (For example, if you make an AI that only considers what will happen within a time limit of 1 minute, it probably won’t be very good at long-term planning. Or if you make an AI system that only pursues very small-scale goals, it won’t be able to solve problems that you don’t know how to break up into small-scale goals.) So on its own, this doesn’t seem to solve outer alignment for highly capable agents.

(See e.g. the “2. Competitive” section of this article by Paul Christiano for some more discussion of why a core desiderata for safety solutions is their performance competitiveness.)
What links here?
- elifland's comment on My take on What We Owe the Future by elifland (Sep 3, 2022, 1:10 PM; 6 points)

Mau Aug 30, 2022, 8:01 AM
3 points
0 ∶ 0
on: The Happiness Maximizer: Why EA is an x-risk
(I skimmed; apologies if I missed relevant things.)

no one can bring about a dystopian future unless their ability to accomplish their goals is significantly more advanced than everyone else’s

[...] the EA community [...] is itself a substantial existential risk

This post seems to rely on the assumption that, in the absence of extremely unusual self-limits, EA’s ability to accomplish its goals will somehow become significantly more advanced than those of the rest of the world combined. That’s quite a strong, unusual assumption to make about any social movement—I think it’d take lots more argument to make a convincing case for it.

Mau Aug 29, 2022, 6:31 PM
2 points
0 ∶ 0
on: Sacred Cow: has the Effective Altruism community had its view on animal husbandry challenged?
I’m not sure how much I agree with this / its applicability, but one argument I’ve heard is that, for individual decision-making and social norm-setting,

total abstinence is easier than perfect moderation

(Kind of a stretch, but I enjoyed this speech on the cultural and coordinating power of simple norms, which can be seen as a case against nuanced norms. Maybe the simplicity of some standards as individuals’ principles, advocacy goals, and social norms makes them more resilient to pressure, whereas more nuanced standards might more easily fall down slippery slopes, incrementally succumbing to pressure to expand the scope of their exceptions until they’re far too broad.)

Mau Aug 29, 2022, 1:43 AM
2 points
0 ∶ 0
in reply to: Lukas_Gloor’s comment on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Fair points!

I think preference-based views fit neatly into the asymmetry.

Here I’m moving on from the original topic, but if you’re interested in following this tangent—I’m not quite getting how preference-based views (specifically, person-affecting preference utilitarianism) maintain the asymmetry while avoiding (a slightly/somewhat weaker version of) “killing happy people is good.”

Under “pure” person-affecting preference utilitarianism (ignoring broader pluralistic views of which this view is just one component, and also ignoring instrumental justifications), clearly one reason why it’s bad to kill people is that this would frustrate some of their preferences. Under this view, is another (pro tanto) reason why it’s bad to kill (not-entirely-satisfied) people that their satisfaction/fulfillment is worth preserving (i.e. is good in a way that outweighs associated frustration)?

My intuition is that one answer to the above question breaks the asymmetry, while the other revives some very counterintuitive implications.
- If we answer “Yes,” then, through that answer, we’ve accepted a concept of “actively good things” into our ethics, rejecting the view that ethics is just about fixing states of affairs that are actively problematic. Now we’re back in (or much closer to?) a framework of “maximize goods minus bads” / “there are intrinsically good things,” which seems to (severely) undermines the asymmetry.
- If we answer “No,” on the grounds that fulfillment can’t outweigh frustration, this would seem to imply that one should kill people, whenever their being killed would frustrate them less than their continued living. Problematically, that seems like it would probably apply to many people, including many pretty happy people.
  - After all, suppose someone is fairly happy (though not entirely, constantly fulfilled), is quite myopic, and only has a moderate intrinsic preference against being killed. Then, the preference utilitarianism we’re considering seems to endorse killing them (since killing them would “only” frustrate their preferences for a short while, while continued living would leave them with decades of frustration, amid their general happiness).
  - There seem to be additional bizarre implications, like “if someone suddenly gets an unrealizable preference, even if they mistakenly think it’s being satisfied and are happy about that, this gives one stronger reasons to kill them.” (Since killing them means the preference won’t go unsatisfied as long.)
  - (I’m assuming that frustration matters (roughly) in proportion to its duration, since e.g. long-lasting suffering seems especially bad.)
  - (Of course, hedonic utilitarianism also endorses some non-instrumental killing, but only under what seem to be much more restrictive conditions—never killing happy people.)

Mau Aug 28, 2022, 12:59 AM
5 points
0 ∶ 0
in reply to: Lukas_Gloor’s comment on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Of the experience-based asymmetric views discussed in the OP, my posts on tranquilism and suffering-focused ethics mention value pluralism and the idea that things other than experiences (i.e., preferences mostly) could also be valuable. Given these explicit mentions it seems false to claim that “these views don’t easily fit into a preference-focused framework.” [...] I’m not sure why you think [a certain] argument would have to be translated into a preference-focused framework.

I think this misunderstands the point I was making. I meant to highlight how, if you’re adopting a pluralistic view, then to defend a strong population asymmetry (the view emphasized in the post’s title), you need reasons why none of the components of your pluralistic view value making happy people.* This gets harder the more pluralistic you are, especially if you can’t easily generalize hedonic arguments to other values. As you suggest, you can get the needed reasons by introducing additional assumptions/frameworks, like rejecting the principle that it’s better for there to be more good things. But I wouldn’t call that an “easy fit”; that’s substantial additional argument, sometimes involving arguing against views that many readers of this forum find axiomatically appealing (like that it’s better for there to be more good things).

(* Technically you don’t need reasons why none of the views consider the making of happy people valuable, just reasons why overall they don’t. Still, I’d guess those two claims are roughly equivalent, since I’m not aware of any prominent views which hold the creation of purely happy people to be actively bad.)

Besides that, I think at this point we’re largely in agreement on the main points we’ve been discussing?
- I’ve mainly meant to argue that some of the ethical frameworks that the original post draws on and emphasizes, in arguing for a population asymmetry, have implications that many find very counterintuitive. You seem to agree.
- If I’ve understood, you’ve mainly been arguing that there are many other views (including some that the original post draws on) which support a population asymmetry while avoiding certain counterintuitive implications. I agree.
  - Your most recent comment seems to frame several arguments for this point as arguments against the first bullet point above, but I don’t think they’re actually arguments against the above, since the views you’re defending aren’t the ones my most-discussed criticism applies to (though that does limit the applicability of the criticism).

Mau Aug 26, 2022, 10:26 PM
14 points
0 ∶ 0
on: I’m interviewing sometimes EA critic Jeffrey Lewis (AKA Arms Control Wonk) about what we get right and wrong when it comes to nuclear weapons and nuclear security. What should I ask him?
On deterrence:
- In the context of massive nuclear attacks, why isn’t the danger of nuclear winter widely seen as making nuclear retaliation redundant (as Ellsberg suggested on another 80K podcast episode)?
- The US threatens to retaliate (with nukes) against anyone who nukes certain US allies—how credible is this threat, and why?
  - Part of how the US tries to make this threat more credible is by sharing nukes with some of its allies. How does this sharing work? Does the US share nukes in such a way that, in a crisis, a non-nuclear host country could easily seize and launch a nuke?
- Why has the US relied on mutually assured destruction instead of minimal deterrence?
- What are some ways in which recent developments in cybersecurity and machine learning interact with nuclear deterrence?
- Why have the US and the USSR/Russia invested so much in making lots of land-based nuclear missiles? (If they were mainly trying to ensure their nuclear weapons would survive a first strike, wouldn’t it have been better to put that money toward making even more nuclear-armed submarines?)
Other questions on nuclear politics:
- Why did it take decades after the first nuclear weapons for countries to sign nuclear nonproliferation and arms control agreements?
On verification of compliance with arms control agreements:
- The US uses satellites to check whether Russia is following nuclear arms control agreements—as far as is publicly known, just how good are US spy satellites? What about commercial satellites?
- The UN’s nuclear watchdog extensively monitors a bunch of nuclear power facilities, to make sure they’re not using uranium to make nukes. One might think it’d be helpful to also do these things at uranium mines, but they usually don’t do this—why not?
- How good do you think current systems are at detecting secret nuclear facilities?
On what else to check out:
- In addition to your own blog and podcast as well as this podcast, what are some of your favorite blogs or podcasts on nuclear security or international security?
(Edited to add a few.)

(I don’t know that there’s much of an EA consensus on nuclear weapons issues—and if there is, I don’t know what the consensus is—so these aren’t quite questions about what EA gets right/wrong on this.)

Mau Aug 24, 2022, 10:21 PM
7 points
0 ∶ 0
in reply to: Miranda_Zhang’s comment on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Thanks for the thoughtful reply; I’ve replied to many of these points here.

In short, I think you’re right that Magnus doesn’t explicitly assume consequentialism or hedonism. I understood him to be implicitly assuming these things because of the post’s focus on creating happiness and suffering, as well as the apparent prevalence of these assumptions in the suffering-focused ethics community (e.g. the fact that it’s called “suffering-focused ethics” rather than “frustration-focused ethics”). But I should have more explicitly recognized those assumptions and how my arguments are limited to them.

Mau Aug 24, 2022, 10:11 PM
9 points
0 ∶ 0
in reply to: Magnus Vinding’s comment on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Thanks for the thoughtful reply; I’ve replied to many of these points here.

On a few other ends:
- I agree that strong negative utilitarian views can be highly purposeful and compassionate. By “semi-nihilistic” I was referring to how some of these views also devalue much (by some counts, half) of what others value. [Edit: Admittedly, many pluralists could say the same to pure classical utilitarians.]
- I agree classical utilitarianism also has bullets to bite (though many of these look like they’re appealing to our intuitions in scenarios where we should expect to have bad intuitions, due to scope insensitivity).

Mau Aug 24, 2022, 9:54 PM
6 points
0 ∶ 0
in reply to: Lukas_Gloor’s comment on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Thanks for the thoughtful reply. You’re right, you can avoid the implications I mentioned by adopting a preference/goal-focused framework. (I’ve edited my original comment to flag this; thanks for helping me recognize it.) That does resolve some problems, but I think it also breaks most of the original post’s arguments, since they weren’t made in (and don’t easily fit into) a preference-focused framework. For example:
- The post argues that making happy people isn’t good and making miserable people is bad, because creating happiness isn’t good and creating suffering is bad. But it’s unclear how this argument can be translated into a preference-focused framework.
  - Could it be that “satisfying preferences isn’t good, and frustrating preferences is bad”? That doesn’t make sense to me; it’s not clear to me there’s a meaningful distinction between satisfying a preference and keeping it from being frustrated.
  - Could it be that “satisfying positive preferences isn’t good, and satisfying negative preferences is good?” But that seems pretty arbitrary, since whether we call some preference positive or negative seems pretty arbitrary (e.g. do I have a positive preference to eat or a negative preference to not be hungry? Is there a meaningful difference?).
- The second section of the original post emphasizes extreme suffering and how it might not be outweighable. But what does this mean in a preference-focused context? Extreme preference frustration? I suspect, for many, that doesn’t have the intuitive horribleness that extreme suffering does.
- The third section of the post focuses on surveys that ask questions about happiness and suffering, so we can’t easily generalize these results to a preference-focused framework.
(I also agree—as I tried to note in my original comment’s first bullet point—that pluralistic or “all-things-considered” views avoid the implications I mentioned. But I think ethical views should be partly judged based on the implications they have on their own. The original post also seems to assume this, since it highlights the implications total utilitarianism has on its own rather than as a part of some broader pluralistic framework.)
What links here?
- Mau's comment on Critique of MacAskill’s “Is It Good to Make Happy People?” by Magnus Vinding (Aug 24, 2022, 10:11 PM; 9 points)
- Mau's comment on Critique of MacAskill’s “Is It Good to Make Happy People?” by Magnus Vinding (Aug 24, 2022, 10:21 PM; 7 points)

Mau Aug 23, 2022, 6:41 PM
7 points
0 ∶ 0
on: Critique of MacAskill’s “Is It Good to Make Happy People?”
Thanks for writing. You’re right that MacAskill doesn’t address these non-obvious points, though I want to push back a bit. Several of your arguments are arguments for the view that “intrinsically positive lives do not exist,” and more generally that intrinsically positive moments do not exist. Since we’re talking about repugnant conclusions, readers should note that this view has some repugnant conclusions of its own.

[Edit: I stated the following criticism too generally; it only applies when one makes an additional assumption: that experiences matter, while things that don’t affect anyone’s experiences don’t matter. As I argue in the below comment thread, that strong focus on experiences seems necessary for some of the original post’s main arguments to work.]

It implies that there wouldn’t be anything wrong with immediately killing everyone reading this, their families, and everyone else, since this supposedly wouldn’t be destroying anything positive. It also implies that there was nothing good (in an absolute sense) in the best moment of every reader’s life—nothing actively good about laughing with friends, or watching a sunset, or hugging a loved one. To me, that’s deeply and obviously counter-intuitive. (And, as the survey you brought up shows, the large majority of people don’t hold these semi-nihilistic views.) Still, to caveat:
- In practice, to their credit, people sympathetic with these views tend to appreciate that, all things considered, they have many reasons to be cooperative toward others and avoid violence.
- None of the above criticisms apply to weak versions of the asymmetry. People can think that reducing suffering is somewhat more important than increasing happiness—rather than infinitely so—and then they avoid most of these criticisms. But they can’t ground these views on the premise that good experiences don’t exist.
Also, as long as we’re reviving old debates, readers may be interested in Toby Ord’s arguments against many of these views (and e.g. this response).

Mau

AI Gover­nance Needs Tech­ni­cal Work

AI Governance Needs Technical Work