EliezerYudkowsky

Karma: 3,587

EliezerYudkowsky 31 May 2015 5:02 UTC
54 points
0 ∶ 0
in reply to: Dale’s comment on: The history of the term ‘effective altruism’
There’s only so many things you can call it, and accidental namespace collisions / phrase reinventions aren’t surprising. I was surprised when I looked back myself and noticed the phrase was there, so it would be more surprising if Toby Ord remembered than if he didn’t. I’m proud to have used the term “effective altruist” once in 2007, but to say that this means I coined the term, especially when it was re-output by the more careful process described above, might be giving me too much credit—but it’s still nice to have this not-quite-coincidental mention be remembered, so thank you for that!
What links here?
- When Did EA Start? by Jeff Kaufman (25 Jan 2023 14:51 UTC; 117 points)
- When Did EA Start? by jefftk (LessWrong; 25 Jan 2023 14:30 UTC; 37 points)

EliezerYudkowsky 28 Sep 2016 22:10 UTC
18 points
0 ∶ 0
on: Two Strange Things About AI Safety Policy
The idea of running an event in particular seems misguided. Conventions come after conversations. Real progress toward understanding, or conveying understanding, does not happen through speakers going On Stage at big events. If speakers On Stage ever say anything sensible, it’s because an edifice of knowledge was built in the background out of people having real, engaged, and constructive arguments with each other, in private where constructive conversations can actually happen, and the speaker On Stage is quoting from that edifice.

(This is also true of journal publications about anything strategic-ish—most journal publications about AI alignment come from the void and are shouting into the void, neither aware of past work nor feeling obliged to engage with any criticism. Lesser (or greater) versions of this phenomenon occur in many fields; part of where the great replication crisis comes from is that people can go on citing refuted studies and nothing embarrassing happens to them, because god forbid there be a real comments section or an email reply that goes out to the whole mailing list.)

If there’s something to be gained from having national-security higher-ups understanding the AGI alignment strategic landscape, or from having alignment people understand the national security landscape, then put Nate Soares in a room with somebody in national security who has a computer science background, and let them have a real conversation. Until that real progress has already been made in in-person conversations happening in the background where people are actually trying to say sensible things and justify their reasoning to one another, having a Big Event with people On Stage is just a giant opportunity for a bunch of people new to the problem to spout out whatever errors they thought up in the first five seconds of thinking, neither aware of past work nor expecting to engage with detailed criticism, words coming from the void and falling into the void. This seems net counterproductive.

EliezerYudkowsky 17 Jun 2021 17:23 UTC
47 points
0 ∶ 0
on: Taboo “Outside View”
I worriedly predict that anyone who followed your advice here would just switch to describing whatever they’re doing as “reference class forecasting” since this captures the key dynamic that makes describing what they’re doing as “outside viewing” appealing: namely, they get to pick a choice of “reference class” whose samples yield the answer they want, claim that their point is in the reference class, and then claiming that what they’re doing is what superforecasters do and what Philip Tetlock told them to do and super epistemically virtuous and anyone who argues with them gets all the burden of proof and is probably a bad person but we get to virtuously listen to them and then reject them for having used the “inside view”.
My own take: Rule One of invoking “the outside view” or “reference class forecasting” is that if a point is more dissimilar to examples in your choice of “reference class” than the examples in the “reference class” are dissimilar to each other, what you’re doing is “analogy”, not “outside viewing”.
All those experimental results on people doing well by using the outside view are results on people drawing a new sample from the same bag as previous samples. Not “arguably the same bag” or “well it’s the same bag if you look at this way”, really actually the same bag: how late you’ll be getting Christmas presents this year, based on how late you were in previous years. Superforecasters doing well by extrapolating are extrapolating a time-series over 20 years, which was a straight line over those 20 years, to another 5 years out along the same line with the same error bars, and then using that as the baseline for further adjustments with due epistemic humility about how sometimes straight lines just get interrupted some year. Not by them picking a class of 5 “relevant” historical events that all had the same outcome, and arguing that some 6th historical event goes in the same class and will have that same outcome.
What links here?

EliezerYudkowsky 8 Aug 2021 14:39 UTC
94 points
0 ∶ 0
on: Towards a Weaker Longtermism
The reason we have a deontological taboo against “let’s commit atrocities for a brighter tomorrow” is not that people have repeatedly done this, it worked exactly like they said it would, and millions of people received better lives in exchange for thousands of people dying unpleasant deaths exactly as promised.

The reason we have this deontological taboo is that the atrocities almost never work to produce the promised benefits. Period. That’s it. That’s why we normatively should have a taboo like that.

(And as always in a case like that, we have historical exceptions that people don’t like to talk about because they worked, eg, Knut Haukelid, or the American Revolution. And these examples are distinguished among other factors by a found mood (the opposite of a missing mood) which doesn’t happily jump on the controversial wagon for controversy points, nor gain power and benefit from the atrocity; but quietly and regretfully kills the innocent night watchman who helped you, to prevent the much much larger issue of Nazis getting nuclear weapons.)

This logic applies without any obvious changes to “let’s commit atrocities in pursuit of a brighter tomorrow a million years away” just like it applies to “let’s commit atrocities in pursuit of a brighter tomorrow in 2 years”. Literally any nice thing somebody says you could get would “justify atrocities”, in exactly the same way, if you forgot this rule. If you admit the existence of thousands of American schoolchildren getting suboptimally nutritious lunches, it could, oh no, justify abducting and torturing businessmen into using their ATM cards so you could get more money for the schoolchildren. Obviously then those children must not exist, or maybe they don’t have qualia so their suffering won’t be important, because if they existed and mattered that could justify atrocities, couldn’t it?

There is nothing special about longtermism compared to any other big desideratum in this regard. It is 100% unjustified special attention because people don’t like the desideratum itself. The same way that people ask “How can we spend money on AI safety when children are starving now?” but their mind doesn’t make the same leap about “How can we spend money on fighting global warming when children are starving now?” or say “Hey maybe we should critique total spending on lipstick advertising before we critique spending on rockets.”

As always, transhumanism done correctly is just humanism.

EliezerYudkowsky 8 Aug 2021 16:26 UTC
71 points
0 ∶ 0
on: Towards a Weaker Longtermism
The final conclusion here strikes me as just the sort of conclusion that you might arrive at as your real bottom line, if in fact you had an arrived at an inner equilibrium between some inner parts of you that enjoy doing something other than longtermism, and your longtermist parts. This inner equilibrium, in my opinion, is fine; and in fact, it is so fine that we ought not to need to search desperately for a utilitarian defense of it. It is wildly unlikely that our utilitarian parts ought to arrive at the conclusion that the present weighs about 50% as much as our long-term future, or 25% or 75%; it is, on the other hand, entirely reasonable that the balance of what our inner parts vote on will end up that way. I am broadly fine with people devoting 50%, 25% or 75% of themselves to longtermism, in that case, as opposed to tearing themselves apart with guilt and ending up doing nothing much, which seems to be the main alternative. But you’re just not going to end up with a utilitarian defense of that bottom line; if the future can matter at all, to the parts of us that care abstractly and according to numbers, it’s going to end up mattering much more than the present; equivalently, any rationalization like exponential discounting that can imply averting this, is going to imply that it is better to eat an ice cream today and destroy a galaxy of happy sapient beings in ten million years. This is crazy, and I think it makes a lot more sense to just admit that part of you cares about galaxies and part of you cares about ice cream and say that neither of these parts are going to be suppressed and beaten down inside you.
I think this is what actually yields an appeal of “regular longtermism”, and since that’s what actually produces the bottom line, I think that what produces this bottom line should just be directly called the justification for it—there’s no point in reaching for a different argument for justification than for conclusion-production.
What links here?
- Simplify EA Pitches to “Holy Shit, X-Risk” by Neel Nanda (11 Feb 2022 1:57 UTC; 184 points)

EliezerYudkowsky 8 Aug 2021 18:21 UTC
19 points
0 ∶ 0
in reply to: zdgroff’s comment on: Towards a Weaker Longtermism
There’s nothing convoluted about it! We just observe that historical experience shows that the supposed benefits never actually appear, leaving just the atrocity! That’s it! That’s the actual reason you know the real result would be net bad and therefore you need to find a reason to argue against it! If historically it worked great and exactly as promised every time, you would have different heuristics about it now!

EliezerYudkowsky 23 Sep 2021 20:27 UTC
10 points
0 ∶ 0
in reply to: Wei Dai’s comment on: Towards a Weaker Longtermism
It strikes me as a fine internal bargain for some nonhuman but human-adjacent species; I would not expect the internal parts of a human to able to abide well by that bargain.

EliezerYudkowsky 27 Oct 2021 18:45 UTC
28 points
1 ∶ 0
on: List of EA funding opportunities
Can you put something on here to the effect of: “Eliezer Yudkowsky continues to claim that anybody who comes to him with a really good AGI alignment idea can and will be funded.”

EliezerYudkowsky 20 Nov 2021 1:36 UTC
17 points
0 ∶ 0
on: FTX EA Fellowships
If I ended up spending some time in the Bahamas this year, do you have a guess as to when would be the optimal time for that?

EliezerYudkowsky 20 Mar 2022 18:24 UTC
25 points
0 ∶ 0
in reply to: Milan_Griffes’s comment on: Milan Griffes on EA blindspots
Smart things are not dangerous because they have access to human-built legacy nukes. Smart things are dangerous because they are smarter than you.
I expect that the most efficient way to kill everyone is via the biotech->nanotech->tiny diamondoid bacteria hopping the jetstream and replicating using CHON and sunlight->everybody falling over dead 3 days after it gets smart. I don’t expect it would use nukes if they were there.
Smart AIs are not dangerous because somebody built guns for them, smart AIs are not dangerous because cars are connected to the Internet, smart AIs are not dangerous because they can steal existing legacy weapons infrastructure, smart AIs are dangerous because they are smarter than you and can think of better stuff to do.

EliezerYudkowsky 6 Sep 2022 17:37 UTC
33 points
0 ∶ 0
on: Open EA Global
Add my voice to the others who’d support an EACon with open registration (maybe with particular persona-non-grata being excluded, I would not support an event that made a big deal about ‘excluding nobody’ or some such). Get some of the people who run successful science-fiction conventions, pick a relatively accessible and cheap location (Las Vegas, maybe?), have panels, invite merchants to bring by EA-relevant or just weird merchandise.

EliezerYudkowsky 12 Nov 2022 2:16 UTC
121 points
20 ∶ 3
in reply to: Kerry_Vaughan’s comment on: The FTX Future Fund team has resigned
In 2021 I tried asking about SBF among what I suppose you could call “EA leadership”, trying to distinguish whether to put SBF into the column of “keeps compacts but compact very carefully” versus “un-Lawful oathbreaker”, based on having heard that early Alameda was a hard breakup. I did not get a neatly itemized list resembling this one on either points 1 or 2, just heard back basically “yeah early Alameda was a hard breakup and the ones who left think they got screwed” (but not that there’d been a compact that got broken) (and definitely not that they’d had poor capital controls), and I tentatively put SBF into column 1. If “EA leadership” had common knowledge of what you list under items 1 or 2, they didn’t tell me about it when I asked. I suppose in principle that I could’ve expended some of my limited time and stamina to go and inquire directly among the breakup victims looking for one who hadn’t signed an NDA, but that’s just a folly of perfect hindsight.
My own guess is that you are mischaracterizing what EA leadership knew.
What links here?

EliezerYudkowsky 12 Nov 2022 2:18 UTC
37 points
15 ∶ 0
in reply to: Guy Raveh’s comment on: The FTX Future Fund team has resigned
Not everybody maintains their FB friends list carefully or at all, really. I’ve seen way nuttier people on friends lists.

EliezerYudkowsky 12 Nov 2022 2:25 UTC
38 points
20 ∶ 4
in reply to: Stuart Buck’s comment on: The FTX Future Fund team has resigned
I think that what FTX is accused of this comment is legitimately way more something where a charitable recipient is not morally obliged to demand this level of careful checking of everything, because our civilization is just not actually able to support this level of competency pornography.
Stealing your customers’ funds is a very different matter from “some of the people who use our services are criminals”. Why, MIRI has in the past accepted matching funds from Google, which I’m sure profits a whole lot off criminals using their services! And some of those criminals may even be bad people!
But you can’t, actually, run a post-agricultural civilization on the principle of everybody who engages in every transaction checking out the full moral character of everybody who transacts with them. If you did try to build clever infrastructure for that, its first use on the margin would be by the right to hunt down sex workers (as already occurs with Visa) and by the left to hunt down people who said bad things on Twitter.
In a hunter-gatherer tribe it maybe makes sense to demand that people not transact with that bad guy over there; it scales as far as it needs to scale. And MIRI would not take money from somebody what we knew had stolen in charity’s name. But to figure it all out—if you want to read about Civilizations that have the basic infrastructure and competence to run those kind of traces, go read science fiction; here on Earth you’ve got VC firms trying to run six months of due diligence and then they invest in FTX.

EliezerYudkowsky 12 Nov 2022 2:32 UTC
20 points
0 ∶ 0
on: Crypto markets, EA funding and optics
Word to @SaraAzukuike. That aged well.

EliezerYudkowsky 12 Nov 2022 3:46 UTC
45 points
19 ∶ 3
in reply to: Kerry_Vaughan’s comment on: The FTX Future Fund team has resigned
Why are you confident of that? In general, I think there’s just less time and competence and careful checking to go around, in this world, than people would want to believe. This isn’t Hieronym’s To The Stars or the partially Hieronym-inspired world of dath ilan.

EliezerYudkowsky 12 Nov 2022 7:00 UTC
50 points
29 ∶ 0
in reply to: Arno’s comment on: IMPCO, don’t injure yourself by returning FTXFF money for services you already provided
(Separately from all issues above, if you hold yourself responsible for returning money, you definitely should not be trying to return money now, while the FTX accounts are still being looted, first by people buying NFTs from Bahamian accounts that could still transfer out money, and now by an outright hack or “hack”.)

EliezerYudkowsky 13 Nov 2022 1:37 UTC
35 points
16 ∶ 17
in reply to: Matthew Yglesias’s comment on: CEA/EV + OP + RP should engage an independent investigator to determine whether key figures in EA knew about the (likely) fraud at FTX
...I for one didn’t take the self-criticism stuff particularly seriously, or consider EA to have scored any points by running the contest? And I thought that seemed too obvious to mention, I guess, that of course scoring diligent self-criticism points is harder than that? Zvi had a longer writeup on a similar take.
I have my own sense of what EA is doing all wrong, but it didn’t particularly occur to me to try to write it up for the criticism contest. That was obviously going to be an in-frame thing, and was obviously not going to be able to reward anything out-of-frame; doing that is really hard if you don’t want to just completely throw the contest over to wild randos. Like, the reason MIRI doesn’t run a contest like that is not that our underlying reality contains nothing worth criticizing, but that a contest like that obviously does not work to highlight your real actual problems.
There’s nothing particularly unhealthy about a movement where a contest like that fails to turn up any real issues—the contest is obviously doomed in that sense from the start. The only failure is the naivety required to imagine that a contest like that could possibly work, against a semiefficient background where smart people have already been saying various things and there are already reasons why those critiques have not been accepted. The best you can say about a contest like that is that maybe it’s worth $100K to get better centralized writeups of in-frame criticisms that your community already knows how to accept but hasn’t finished acting on yet.

EliezerYudkowsky 13 Nov 2022 1:42 UTC
66 points
14 ∶ 3
on: IMPCO, don’t injure yourself by returning FTXFF money for services you already provided
Amendment for the sake of thoroughness: Looking back, I think at one point I was passing through the Bahamas anyways, and asked FTX if they wanted me to stay over a week and talk to a bunch of visiting EAs they had in a Nassau hotel. I paid for the hotel (and later an Airbnb when the hotel ran out) out of pocket, and was reimbursed for that by FTX or FTX FF or something. I hardly consider this ‘being funded’, I didn’t go to any beaches or try to turn it into a vacation, more like I did some free work for them and had a place to stay while doing it; but there was nonetheless at one point a reimbursement that went from FTX wallet to Yudkowsky wallet. I myself consider this unimportant, but in matters like these it is important to be thorough.

EliezerYudkowsky 13 Nov 2022 1:54 UTC
18 points
13 ∶ 0
in reply to: Lukas_Gloor’s comment on: A personal statement on FTX
I’d currently wildly guess that Coinbase is not a fraud.