EliezerYudkowsky

Karma: 3,633

EliezerYudkowsky Nov 12, 2022, 2:32 AM
20 points
0 ∶ 0
on: Crypto markets, EA funding and optics
Word to @SaraAzukuike. That aged well.

EliezerYudkowsky Nov 12, 2022, 2:25 AM
38 points
20 ∶ 4
in reply to: Stuart Buck’s comment on: The FTX Future Fund team has resigned
I think that what FTX is accused of this comment is legitimately way more something where a charitable recipient is not morally obliged to demand this level of careful checking of everything, because our civilization is just not actually able to support this level of competency pornography.
Stealing your customers’ funds is a very different matter from “some of the people who use our services are criminals”. Why, MIRI has in the past accepted matching funds from Google, which I’m sure profits a whole lot off criminals using their services! And some of those criminals may even be bad people!
But you can’t, actually, run a post-agricultural civilization on the principle of everybody who engages in every transaction checking out the full moral character of everybody who transacts with them. If you did try to build clever infrastructure for that, its first use on the margin would be by the right to hunt down sex workers (as already occurs with Visa) and by the left to hunt down people who said bad things on Twitter.
In a hunter-gatherer tribe it maybe makes sense to demand that people not transact with that bad guy over there; it scales as far as it needs to scale. And MIRI would not take money from somebody what we knew had stolen in charity’s name. But to figure it all out—if you want to read about Civilizations that have the basic infrastructure and competence to run those kind of traces, go read science fiction; here on Earth you’ve got VC firms trying to run six months of due diligence and then they invest in FTX.

EliezerYudkowsky Nov 12, 2022, 2:18 AM
37 points
15 ∶ 0
in reply to: Guy Raveh’s comment on: The FTX Future Fund team has resigned
Not everybody maintains their FB friends list carefully or at all, really. I’ve seen way nuttier people on friends lists.

EliezerYudkowsky Nov 12, 2022, 2:16 AM
121 points
20 ∶ 3
in reply to: Kerry_Vaughan’s comment on: The FTX Future Fund team has resigned
In 2021 I tried asking about SBF among what I suppose you could call “EA leadership”, trying to distinguish whether to put SBF into the column of “keeps compacts but compact very carefully” versus “un-Lawful oathbreaker”, based on having heard that early Alameda was a hard breakup. I did not get a neatly itemized list resembling this one on either points 1 or 2, just heard back basically “yeah early Alameda was a hard breakup and the ones who left think they got screwed” (but not that there’d been a compact that got broken) (and definitely not that they’d had poor capital controls), and I tentatively put SBF into column 1. If “EA leadership” had common knowledge of what you list under items 1 or 2, they didn’t tell me about it when I asked. I suppose in principle that I could’ve expended some of my limited time and stamina to go and inquire directly among the breakup victims looking for one who hadn’t signed an NDA, but that’s just a folly of perfect hindsight.
My own guess is that you are mischaracterizing what EA leadership knew.
What links here?

EliezerYudkowsky Sep 6, 2022, 5:37 PM
33 points
0 ∶ 0
on: Open EA Global
Add my voice to the others who’d support an EACon with open registration (maybe with particular persona-non-grata being excluded, I would not support an event that made a big deal about ‘excluding nobody’ or some such). Get some of the people who run successful science-fiction conventions, pick a relatively accessible and cheap location (Las Vegas, maybe?), have panels, invite merchants to bring by EA-relevant or just weird merchandise.

EliezerYudkowsky Mar 20, 2022, 6:24 PM
25 points
0 ∶ 0
in reply to: Milan Griffes’s comment on: Milan Griffes on EA blindspots
Smart things are not dangerous because they have access to human-built legacy nukes. Smart things are dangerous because they are smarter than you.
I expect that the most efficient way to kill everyone is via the biotech->nanotech->tiny diamondoid bacteria hopping the jetstream and replicating using CHON and sunlight->everybody falling over dead 3 days after it gets smart. I don’t expect it would use nukes if they were there.
Smart AIs are not dangerous because somebody built guns for them, smart AIs are not dangerous because cars are connected to the Internet, smart AIs are not dangerous because they can steal existing legacy weapons infrastructure, smart AIs are dangerous because they are smarter than you and can think of better stuff to do.

EliezerYudkowsky Nov 20, 2021, 1:36 AM
17 points
0 ∶ 0
on: FTX EA Fellowships
If I ended up spending some time in the Bahamas this year, do you have a guess as to when would be the optimal time for that?

EliezerYudkowsky Oct 27, 2021, 6:45 PM
28 points
1 ∶ 0
on: List of EA funding opportunities
Can you put something on here to the effect of: “Eliezer Yudkowsky continues to claim that anybody who comes to him with a really good AGI alignment idea can and will be funded.”

EliezerYudkowsky Sep 23, 2021, 8:27 PM
10 points
0 ∶ 0
in reply to: Wei Dai’s comment on: Towards a Weaker Longtermism
It strikes me as a fine internal bargain for some nonhuman but human-adjacent species; I would not expect the internal parts of a human to able to abide well by that bargain.

EliezerYudkowsky Aug 8, 2021, 6:21 PM
19 points
0 ∶ 0
in reply to: zdgroff’s comment on: Towards a Weaker Longtermism
There’s nothing convoluted about it! We just observe that historical experience shows that the supposed benefits never actually appear, leaving just the atrocity! That’s it! That’s the actual reason you know the real result would be net bad and therefore you need to find a reason to argue against it! If historically it worked great and exactly as promised every time, you would have different heuristics about it now!

EliezerYudkowsky Aug 8, 2021, 4:26 PM
71 points
0 ∶ 0
on: Towards a Weaker Longtermism
The final conclusion here strikes me as just the sort of conclusion that you might arrive at as your real bottom line, if in fact you had an arrived at an inner equilibrium between some inner parts of you that enjoy doing something other than longtermism, and your longtermist parts. This inner equilibrium, in my opinion, is fine; and in fact, it is so fine that we ought not to need to search desperately for a utilitarian defense of it. It is wildly unlikely that our utilitarian parts ought to arrive at the conclusion that the present weighs about 50% as much as our long-term future, or 25% or 75%; it is, on the other hand, entirely reasonable that the balance of what our inner parts vote on will end up that way. I am broadly fine with people devoting 50%, 25% or 75% of themselves to longtermism, in that case, as opposed to tearing themselves apart with guilt and ending up doing nothing much, which seems to be the main alternative. But you’re just not going to end up with a utilitarian defense of that bottom line; if the future can matter at all, to the parts of us that care abstractly and according to numbers, it’s going to end up mattering much more than the present; equivalently, any rationalization like exponential discounting that can imply averting this, is going to imply that it is better to eat an ice cream today and destroy a galaxy of happy sapient beings in ten million years. This is crazy, and I think it makes a lot more sense to just admit that part of you cares about galaxies and part of you cares about ice cream and say that neither of these parts are going to be suppressed and beaten down inside you.
I think this is what actually yields an appeal of “regular longtermism”, and since that’s what actually produces the bottom line, I think that what produces this bottom line should just be directly called the justification for it—there’s no point in reaching for a different argument for justification than for conclusion-production.
What links here?
- Simplify EA Pitches to “Holy Shit, X-Risk” by Neel Nanda (Feb 11, 2022, 1:57 AM; 186 points)

EliezerYudkowsky Aug 8, 2021, 2:39 PM
94 points
0 ∶ 0
on: Towards a Weaker Longtermism
The reason we have a deontological taboo against “let’s commit atrocities for a brighter tomorrow” is not that people have repeatedly done this, it worked exactly like they said it would, and millions of people received better lives in exchange for thousands of people dying unpleasant deaths exactly as promised.

The reason we have this deontological taboo is that the atrocities almost never work to produce the promised benefits. Period. That’s it. That’s why we normatively should have a taboo like that.

(And as always in a case like that, we have historical exceptions that people don’t like to talk about because they worked, eg, Knut Haukelid, or the American Revolution. And these examples are distinguished among other factors by a found mood (the opposite of a missing mood) which doesn’t happily jump on the controversial wagon for controversy points, nor gain power and benefit from the atrocity; but quietly and regretfully kills the innocent night watchman who helped you, to prevent the much much larger issue of Nazis getting nuclear weapons.)

This logic applies without any obvious changes to “let’s commit atrocities in pursuit of a brighter tomorrow a million years away” just like it applies to “let’s commit atrocities in pursuit of a brighter tomorrow in 2 years”. Literally any nice thing somebody says you could get would “justify atrocities”, in exactly the same way, if you forgot this rule. If you admit the existence of thousands of American schoolchildren getting suboptimally nutritious lunches, it could, oh no, justify abducting and torturing businessmen into using their ATM cards so you could get more money for the schoolchildren. Obviously then those children must not exist, or maybe they don’t have qualia so their suffering won’t be important, because if they existed and mattered that could justify atrocities, couldn’t it?

There is nothing special about longtermism compared to any other big desideratum in this regard. It is 100% unjustified special attention because people don’t like the desideratum itself. The same way that people ask “How can we spend money on AI safety when children are starving now?” but their mind doesn’t make the same leap about “How can we spend money on fighting global warming when children are starving now?” or say “Hey maybe we should critique total spending on lipstick advertising before we critique spending on rockets.”

As always, transhumanism done correctly is just humanism.

EliezerYudkowsky Jun 17, 2021, 5:23 PM
47 points
0 ∶ 0
on: Taboo “Outside View”
I worriedly predict that anyone who followed your advice here would just switch to describing whatever they’re doing as “reference class forecasting” since this captures the key dynamic that makes describing what they’re doing as “outside viewing” appealing: namely, they get to pick a choice of “reference class” whose samples yield the answer they want, claim that their point is in the reference class, and then claiming that what they’re doing is what superforecasters do and what Philip Tetlock told them to do and super epistemically virtuous and anyone who argues with them gets all the burden of proof and is probably a bad person but we get to virtuously listen to them and then reject them for having used the “inside view”.
My own take: Rule One of invoking “the outside view” or “reference class forecasting” is that if a point is more dissimilar to examples in your choice of “reference class” than the examples in the “reference class” are dissimilar to each other, what you’re doing is “analogy”, not “outside viewing”.
All those experimental results on people doing well by using the outside view are results on people drawing a new sample from the same bag as previous samples. Not “arguably the same bag” or “well it’s the same bag if you look at this way”, really actually the same bag: how late you’ll be getting Christmas presents this year, based on how late you were in previous years. Superforecasters doing well by extrapolating are extrapolating a time-series over 20 years, which was a straight line over those 20 years, to another 5 years out along the same line with the same error bars, and then using that as the baseline for further adjustments with due epistemic humility about how sometimes straight lines just get interrupted some year. Not by them picking a class of 5 “relevant” historical events that all had the same outcome, and arguing that some 6th historical event goes in the same class and will have that same outcome.
What links here?
- Taboo “Outside View” by Daniel Kokotajlo (LessWrong; Jun 17, 2021, 9:36 AM; 351 points)
- Taboo “Outside View” by kokotajlod (Jun 17, 2021, 9:39 AM; 177 points)

EliezerYudkowsky Sep 28, 2016, 10:10 PM
18 points
0 ∶ 0
on: Two Strange Things About AI Safety Policy
The idea of running an event in particular seems misguided. Conventions come after conversations. Real progress toward understanding, or conveying understanding, does not happen through speakers going On Stage at big events. If speakers On Stage ever say anything sensible, it’s because an edifice of knowledge was built in the background out of people having real, engaged, and constructive arguments with each other, in private where constructive conversations can actually happen, and the speaker On Stage is quoting from that edifice.

(This is also true of journal publications about anything strategic-ish—most journal publications about AI alignment come from the void and are shouting into the void, neither aware of past work nor feeling obliged to engage with any criticism. Lesser (or greater) versions of this phenomenon occur in many fields; part of where the great replication crisis comes from is that people can go on citing refuted studies and nothing embarrassing happens to them, because god forbid there be a real comments section or an email reply that goes out to the whole mailing list.)

If there’s something to be gained from having national-security higher-ups understanding the AGI alignment strategic landscape, or from having alignment people understand the national security landscape, then put Nate Soares in a room with somebody in national security who has a computer science background, and let them have a real conversation. Until that real progress has already been made in in-person conversations happening in the background where people are actually trying to say sensible things and justify their reasoning to one another, having a Big Event with people On Stage is just a giant opportunity for a bunch of people new to the problem to spout out whatever errors they thought up in the first five seconds of thinking, neither aware of past work nor expecting to engage with detailed criticism, words coming from the void and falling into the void. This seems net counterproductive.

EliezerYudkowsky May 31, 2015, 5:02 AM
59 points
0 ∶ 0
in reply to: Dale’s comment on: The history of the term ‘effective altruism’
There’s only so many things you can call it, and accidental namespace collisions / phrase reinventions aren’t surprising. I was surprised when I looked back myself and noticed the phrase was there, so it would be more surprising if Toby Ord remembered than if he didn’t. I’m proud to have used the term “effective altruist” once in 2007, but to say that this means I coined the term, especially when it was re-output by the more careful process described above, might be giving me too much credit—but it’s still nice to have this not-quite-coincidental mention be remembered, so thank you for that!
What links here?