EliezerYudkowsky

Karma: 3,583

IMPCO, don’t injure yourself by returning FTXFF money for services you already provided

EliezerYudkowsky12 Nov 2022 4:51 UTC

466 points

43 comments8 min readEA link

Who’s at fault for FTX’s wrongdoing

EliezerYudkowsky16 Nov 2022 4:47 UTC

177 points

88 comments6 min readEA link

AGI Ruin: A List of Lethalities

EliezerYudkowsky6 Jun 2022 23:28 UTC

162 points

53 comments30 min readEA link

(www.lesswrong.com)

EliezerYudkowsky 14 Nov 2022 4:10 UTC
158 points
51 ∶ 9
on: NY Times on the FTX implosion’s impact on EA
I think EAs could stand to learn something from non-EAs here, about how not to blame the victim even when the victim is you.

EliezerYudkowsky 12 Nov 2022 2:16 UTC
121 points
20 ∶ 3
in reply to: Kerry_Vaughan’s comment on: The FTX Future Fund team has resigned
In 2021 I tried asking about SBF among what I suppose you could call “EA leadership”, trying to distinguish whether to put SBF into the column of “keeps compacts but compact very carefully” versus “un-Lawful oathbreaker”, based on having heard that early Alameda was a hard breakup. I did not get a neatly itemized list resembling this one on either points 1 or 2, just heard back basically “yeah early Alameda was a hard breakup and the ones who left think they got screwed” (but not that there’d been a compact that got broken) (and definitely not that they’d had poor capital controls), and I tentatively put SBF into column 1. If “EA leadership” had common knowledge of what you list under items 1 or 2, they didn’t tell me about it when I asked. I suppose in principle that I could’ve expended some of my limited time and stamina to go and inquire directly among the breakup victims looking for one who hadn’t signed an NDA, but that’s just a folly of perfect hindsight.
My own guess is that you are mischaracterizing what EA leadership knew.
What links here?

Purchase fuzzies and utilons separately

EliezerYudkowsky27 Dec 2019 2:21 UTC

110 points

3 comments5 min readEA link

(www.lesswrong.com)

EliezerYudkowsky 18 Feb 2023 17:43 UTC
109 points
28 ∶ 4
in reply to: Duncan Sabien’s comment on: People Will Sometimes Just Lie About You
I also attest that Aella is, if anything, severely underconveying the extent to which this central thesis is true. It’s really really hard to convey until you’ve lived that experience yourself. I also don’t know how to convey this to people who haven’t lived through it. My experience was also of having been warned about it, but not having integrated the warnings or really actually understood how bad the misrepresentation actually was in practice, until I lived through it.

EliezerYudkowsky 8 Aug 2021 14:39 UTC
94 points
0 ∶ 0
on: Towards a Weaker Longtermism
The reason we have a deontological taboo against “let’s commit atrocities for a brighter tomorrow” is not that people have repeatedly done this, it worked exactly like they said it would, and millions of people received better lives in exchange for thousands of people dying unpleasant deaths exactly as promised.

The reason we have this deontological taboo is that the atrocities almost never work to produce the promised benefits. Period. That’s it. That’s why we normatively should have a taboo like that.

(And as always in a case like that, we have historical exceptions that people don’t like to talk about because they worked, eg, Knut Haukelid, or the American Revolution. And these examples are distinguished among other factors by a found mood (the opposite of a missing mood) which doesn’t happily jump on the controversial wagon for controversy points, nor gain power and benefit from the atrocity; but quietly and regretfully kills the innocent night watchman who helped you, to prevent the much much larger issue of Nazis getting nuclear weapons.)

This logic applies without any obvious changes to “let’s commit atrocities in pursuit of a brighter tomorrow a million years away” just like it applies to “let’s commit atrocities in pursuit of a brighter tomorrow in 2 years”. Literally any nice thing somebody says you could get would “justify atrocities”, in exactly the same way, if you forgot this rule. If you admit the existence of thousands of American schoolchildren getting suboptimally nutritious lunches, it could, oh no, justify abducting and torturing businessmen into using their ATM cards so you could get more money for the schoolchildren. Obviously then those children must not exist, or maybe they don’t have qualia so their suffering won’t be important, because if they existed and mattered that could justify atrocities, couldn’t it?

There is nothing special about longtermism compared to any other big desideratum in this regard. It is 100% unjustified special attention because people don’t like the desideratum itself. The same way that people ask “How can we spend money on AI safety when children are starving now?” but their mind doesn’t make the same leap about “How can we spend money on fighting global warming when children are starving now?” or say “Hey maybe we should critique total spending on lipstick advertising before we critique spending on rockets.”

As always, transhumanism done correctly is just humanism.

GPTs are Predictors, not Imitators

EliezerYudkowsky8 Apr 2023 19:59 UTC

74 points

12 comments1 min readEA link

EliezerYudkowsky 10 Jan 2023 19:19 UTC
71 points
28 ∶ 18
on: AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years
Suppose you are one of the 0.1% of macro bonds traders familiar with Yudkowskian foom. You reason as follows: “Suppose that in the next 2 years, we get even more alarming news out of GPT-4 and successors. Suppose it’s so incredibly alarming that 10% of macro traders notice, and then 10% of those hear about Yudkowskian foom scenarios. Putting myself into the shoes of one of those normie macro traders, I think I reason… that most actual normal people won’t change their saving behavior any time soon, even if theoretically they should decrease their saving, and that’s not likely to have macro effects. Still as a normie trader who’s heard about Yudkowsky foomdoom, I think I reason that if Yudkowsky’s right, we’re all dead, and if Yudkowsky’s not right, I’ll get embarrassed about a wrong trade and fired. So this normie trader won’t trade on Yudkowsky foomdoom premises. Therefore I don’t think I can profit over the next two years by shorting a TIPS fund… even leaving aside concerning feelings about whether going hugely short LTPZ would have model risk about LTPZ’s actual relation to real interest rates in these scenarios, or whether other traders would expect big AI impacts to hit measured inflation because of AI-driven lower prices or AI-driven unemployment.”
And then one week before the end of the world, the 1% of most clueful macro bonds traders… will take vacation days early, and draw down their rainy day funds to spend time with their family. They still won’t make macro trades about that, because the payoff matrix looks like “If you’re right, you’re dead and not paid, and if you’re wrong, you’re embarrassed and get fired.” Then haha whoops it turns out that the world didn’t end in a week after all, and people go back to work with a nervous laugh and a sick feeling in their stomachs, and everybody actually falls over dead three weeks later.
If Omega tells you today that everyone will be dead in 2030 with probability 1, there’s no direct way to make a market profit on that private information over the next 2 years, except insofar as foresightful traders today expect less foresightful future traders to hear about AI and read an economics textbook and decide that interest rates theoretically ought to rise and go short TIPS index funds. Foresightful traders today don’t expect this.
To put it another way: Yes, savvy market traders don’t believe that in 2025 everybody will realize that the world is ending. The savvy market traders are correct! Even at the point where the world is ending, everybody will not believe this, and so at no point will the savvy trader have made a profit! The death of all of humanity induces a market anomaly wherein savvy traders don’t expect to be able to profit from everyone else’s error because no event occurs where the real thing actually happens and everybody says “Oops” and the savvy trader gets paid off.
There just isn’t any mystery here. You can’t make a short-term profit off correcting these market prices even if Omega whispers the truth in your ear with certainty. That’s it, that’s the mystery explained, you’re done.
What links here?

EliezerYudkowsky 8 Aug 2021 16:26 UTC
71 points
0 ∶ 0
on: Towards a Weaker Longtermism
The final conclusion here strikes me as just the sort of conclusion that you might arrive at as your real bottom line, if in fact you had an arrived at an inner equilibrium between some inner parts of you that enjoy doing something other than longtermism, and your longtermist parts. This inner equilibrium, in my opinion, is fine; and in fact, it is so fine that we ought not to need to search desperately for a utilitarian defense of it. It is wildly unlikely that our utilitarian parts ought to arrive at the conclusion that the present weighs about 50% as much as our long-term future, or 25% or 75%; it is, on the other hand, entirely reasonable that the balance of what our inner parts vote on will end up that way. I am broadly fine with people devoting 50%, 25% or 75% of themselves to longtermism, in that case, as opposed to tearing themselves apart with guilt and ending up doing nothing much, which seems to be the main alternative. But you’re just not going to end up with a utilitarian defense of that bottom line; if the future can matter at all, to the parts of us that care abstractly and according to numbers, it’s going to end up mattering much more than the present; equivalently, any rationalization like exponential discounting that can imply averting this, is going to imply that it is better to eat an ice cream today and destroy a galaxy of happy sapient beings in ten million years. This is crazy, and I think it makes a lot more sense to just admit that part of you cares about galaxies and part of you cares about ice cream and say that neither of these parts are going to be suppressed and beaten down inside you.
I think this is what actually yields an appeal of “regular longtermism”, and since that’s what actually produces the bottom line, I think that what produces this bottom line should just be directly called the justification for it—there’s no point in reaching for a different argument for justification than for conclusion-production.
What links here?
- Simplify EA Pitches to “Holy Shit, X-Risk” by Neel Nanda (11 Feb 2022 1:57 UTC; 184 points)

EliezerYudkowsky 13 Nov 2022 1:42 UTC
66 points
14 ∶ 3
on: IMPCO, don’t injure yourself by returning FTXFF money for services you already provided
Amendment for the sake of thoroughness: Looking back, I think at one point I was passing through the Bahamas anyways, and asked FTX if they wanted me to stay over a week and talk to a bunch of visiting EAs they had in a Nassau hotel. I paid for the hotel (and later an Airbnb when the hotel ran out) out of pocket, and was reimbursed for that by FTX or FTX FF or something. I hardly consider this ‘being funded’, I didn’t go to any beaches or try to turn it into a vacation, more like I did some free work for them and had a place to stay while doing it; but there was nonetheless at one point a reimbursement that went from FTX wallet to Yudkowsky wallet. I myself consider this unimportant, but in matters like these it is important to be thorough.

EliezerYudkowsky 27 Aug 2023 16:47 UTC
65 points
32 ∶ 23
on: Eliezer Yudkowsky Is Frequently, Confidently, Egregiously Wrong
The first object-level issue the author talks about is whether the brain is close to the Landaeur limit. No particular issue is cited, only that somebody else claimed a lot of authority and claimed I was wrong about something, what exactly is not shown.
The brain obviously cannot be operating near the Landaeur limit. Thousands of neurotransmitter molecules and thousands of ions need to be pumped back to their original places after each synaptic flash. Each of these is a thermodynamically irreversible operation and it staggers the imagination that every ion pumped en masse back out of some long axon or dendrite, after ions flooded en masse into it to propagate electrical depolarization, is part of a well-designed informational algorithm that could not be simplified. Any calculation saying that biology is operating close to the Landaeur limit has reached a face-value absurdity.
Of course, this may not seem to address anything, since OP failed to state what I was putatively wrong about and admits to not understanding it themselves; I can’t refute what isn’t shown.
The first substantive criticism OP claims to understand theirself is on Zombies.
I say:
Your “zombie”, in the philosophical usage of the term, is putatively a being that is exactly like you in every respect—identical behavior, identical speech, identical brain; every atom and quark in exactly the same position, moving according to the same causal laws of motion—except that your zombie is not conscious.
The author would have you believe this is a ludicrous straw position.
I invite anyone to simply read the opening paragraphs of the SEP encyclopedia entry on P-zombies:
If zombies are to be counterexamples to physicalism, it is not enough for them to be behaviorally and functionally like normal human beings: plenty of physicalists accept that merely behavioral or functional duplicates of ourselves might lack qualia. Zombies must be like normal human beings in all physical respects, and they must have the physical properties that physicalists suppose we have. This requires them to be subject to the causal closure of the physical, which is why their supposed lack of consciousness is a challenge to physicalism. If instead they were to be conceived of as creatures whose behavior could not be explained physically, physicalists would have no reason to bother with the idea: there is plenty of evidence that, as epiphenomenalists hold, our movements actually are explicable in physical terms (see e.g. Papineau 2002).
This is a debate that has gone on for very long in philosophy. I’d say it’s gone on too long.
But whether or not particular thought experiments, by seeming metaphysically possible, license other conclusions about metaphysics, is exactly the entire substance. The base thought experiment is not or should not be in dispute: it’s a being whose physics duplicate the physics of a human being including the causal closure of what is said to be ‘physics’, i.e., all of the causes of behavior are included into the p-zombie. Some people go on at fantastic length from this to say that it demonstrates the possibility of an extra consciousness that they call “epiphenomenal”, and some say that it demonstrates the possibility of a nonphysical consciousness that they don’t call “epiphenomenal”, but it’s my position that somewhere along the way of a long argument they have dropped the ball on the original thought experiment; whatever they call “consciousness” that isn’t in the supposed p-zombie, it can’t be among the causes of why we talk about consciousness, or why our verbally reportable stream of thought talks about consciousness, etc, because the zombie behaves outwardly like we do and also includes the minimal closure of the causes of that physical behavior.
The author of the above post has misrepresented what my zombies argument was about. It’s not that I think philosophers openly claim that p-zombies demonstrate epiphenomenalism; it’s that I think philosophers are confused about what this thought experiment demonstrates.
The author having been shown to be wrong on the first points addressed, which I chose in order rather than selectively sampling, I hope you accept this as obvious evidence that the rest would be no better if you looked into them in detail or I responded in detail. For a post claiming to show that I’m often grossly wrong, actual quotes from me, with linked context and dates attached, are remarkably thin on the ground.
You will mark that in this comment I first respond to a substantive point and show it to be mistaken before I make any general criticism of the author; which can then be supported by that previously shown, initial, first-thing, object-level point. You will find every post of the Less Wrong sequences written the same way.
As the entire post violates basic rules of epistemic conduct by opening with a series of not-yet-supported personal attacks, I will not be responding to the rest in detail. I’m sad about how anything containing such an egregious violation of basic epistemic conduct got this upvoted, and wonder about sockpuppet accounts or alternatively a downfall of EA. The relevant principle of epistemic good conduct seems to me straightforward: if you’ve got to make personal attacks (and sometimes you do), make them after presenting your object-level points that support those personal attacks. This shouldn’t be a difficult rule to follow, or follow much better than this; and violating it this hugely and explicitly is sufficiently bad news that people should’ve been wary about this post and hesitated to upvote it for that reason alone.
What links here?

EliezerYudkowsky 12 Jan 2023 13:44 UTC
61 points
14 ∶ 2
in reply to: EliezerYudkowsky’s comment on: AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years
Trying again:
OP seems to ambiguate between two ideas, one true idea, and one false idea.
The true idea is that if Omega tells you personally that the world will end in 2030 with probability 1, you personally should not bother saving for retirement. Call this the Personal Idea.
The false idea is that if you believe in foomdoom, you should go long real interest rates and expect a market profit. Call this the Market Idea.
Intuitively, at least if you’re swayed by this essay, the idea in Market probably seems pretty close to the idea in Personal. If everybody started consuming for today and investing less, real interest rates would go up, right? So if you don’t believe that Market is about as strong as Personal, what invalid reasoning step occurs within the gap between the true premise in Personal to the false conclusion in Market?
Is it invalid that if in 2025 everyone started believing that the world would end in 2030 with probability 1, real interest rates would rise in 2025? Honestly, I’m not even sure of that in real life. People are arguing clever-ideas like ‘Shouldn’t everyone take out big loans due later?’ but maybe the lender doesn’t want to lend anymore, if everyone knows that. There’s a supply collapse and a demand collapse and yes I see the theoretical argument but real-world monetary stuff is in fact really strange and complicated; I didn’t see anybody calling the actual interest-rate trajectory surrounding Covid in advance of it actually playing out.
In real life, what zaps you when you think there’s a worldwide pandemic coming and try to trade interest rates, isn’t that you didn’t know about the pandemic ahead of the oblivious market, it’s that you guessed wrong about what the market would really actually do in real life as the pandemic played out and finally ended.
You can sometimes make a profit off an oblivious market, if you guess narrowly enough at reactions that are much more strongly determined. Wei Dai reports making huge profits on the Covid trade against the oblivious market there, after correctly guessing that people soon noticing Covid would at least increase expected volatility and the price of downside put options would go up.
But I don’t think anybody called “there will be a huge market drop followed by an even steeper market ascent, after the Fed’s delayed reaction to a huge drop in TIPS-predicted inflation, and people staying home and saving and trading stocks, followed by skyrocketing inflation later.”
I don’t think the style of reasoning used in the OP is the kind of thing that works reliably. It’s doing the equivalent of trying to predict something harder than ‘once the pandemic really hits, people in the short term will notice and expect more volatility and option prices will go up’; it’s trying to do the equivalent of predicting the market reaction to the Fed reaction a couple of years later.
The entire OP is written as if we lived in an alternate universe where it is way way easier than history has actually shown, to figure out what happens in broad markets after any sort of complicated or nontrivial event occurs in real life that isn’t on the order of “unexpected Fed rate hike” or “company reports much higher-than-expected profits”. And it’s written in such a way as to mislead EAs reading it about the general confidence that the field of economics is able to justly put in predictions about broad market reactions to strange things.
If you haven’t already looked at OP’s recommended investment instrument of (short) LPTZ, which holds inflation-protected Treasuries and is therefore their recommended way of tracking real interest rates, I recommend the following exercise: First try to figure out what you would have believed a priori, without benefit of hindsight, real interest rates would do over the course of a pandemic. Then, decide what investment strategy you’d have followed with LTPZ if you thought you knew about a pandemic ahead of the market. Then, decide what you think happened to real interest rates with benefit of hindsight. Then, go look at the actual price trajectory of their recommended instrument of LTPZ.
I am not sure I can properly convey this thought that I am trying to convey; I have had trouble actually conveying this thought to EAs before. The thought is that people often do long careful serious-sounding writeups which EAs then take Very Seriously, because they are so long and so seriously argued, but in fact fail to bind to reality entirely, in a way that doesn’t have to do with the details of the complicated arguments. Very serious arguments about what ought to happen to the price of an ETF that tracks 15-year TIPS, via the intermediate step of arguing about what logically ought to happen to real interest rates, are the sort of thing that, historically, average economists have not really been able to pull off; it’s a kind of thought that you should expect fails to achieve basic binding to reality. What would LTPZ or its post-facto equivalent have been doing around the time of the Cuban Missile Crisis? My model says ‘no prediction’; they’ll have done whatever. Afterwards somebody will make up a story about it in hindsight, but it is not the sort of thing where history says that long complicated analyses are remotely reliably good at doing it in advance.
But there are even weaker links in the argument, so let’s accept the LTPZ step arguendo and pass on.
An even bigger problem is that, since everybody is going to die before anything really pays out, the marginal foresightful trader does not have a strong incentive to early-on move the market toward where the market would end up in equilibrium after everyone agreed on the actual facts of the matter and had time to trade about them.
Prediction markets, I sometimes explain to people, are tools for transmitting future observables, or more generally propositions that people expect to publicly agree on at some future point even if they don’t agree now, lossily backward in time, manifesting as well-calibrated probability distributions.
To run a prediction market, you first and foremost need a future publicly observable measurement, which is a special case of a place where we expect most people to agree on an extreme probability assignment later, even though they don’t agree now or don’t make extreme probability assignments now. You cannot run a prediction market on whether supplementing lots of potassium can produce weight loss; you can only run a prediction market about what an experiment will report in the way of results, or what a particular judge will say the experimental evidence seems to have indicated in five years. You cannot directly observe “whether potassium causes weight loss” as an underlying fact of biology, so you can’t have a prediction market about that; you can only observe what somebody reports as an experimental result, or what a particular person appointed as judge says out loud about the state of evidence later.
The marginal foresightful trader usually has a motive to run ahead of the market and make trades now, based on where the equilibrium ought to settle later; not because they are nobly undertaking a grand altruistic project of transmitting facts backward in time and making the market behave nicely from the standpoint of theoretical economics, but because they expect to get paid after everyone makes the common observation and the market settles into a new equilibrium reflecting that state of knowledge. And then they expect to have that money, or to get a bonus for earning that money for their proprietary trading firm, and for that money to still hold its value, and for them to be able to spend the money on nice things.
In the unusual case of foomdoom, even if doom proceeds slowly enough that a large-enough group of marginal foresightful traders see the foomdoom coming, even if there is somehow a really definitive fire alarm that says “extremely high probability you are dead within two to four years”, it is incredibly unlikely that everyone in the world will agree that yes we’ll all be dead in two to four years, and that the markets will settle into the equilibrium that an economist would say corresponds to the belief that we’ll all be dead in two to four years; which is what’s required for the foresightful proprietary trader to score a huge profit and get a big bonus that year and have time to spend it on some fancy way of passing the remaining time.
People do not usually agree on what will happen in two to four years. This kind of agreement that reliably reflects a fact, and makes a market pay out in a way that you can trust to correspond to that fact, is usually achieved after that fact is publicly definitively observed, not two to four years ahead of the observation.
In case of foomdoom the world never settles into equilibrium later, the bets never pay out, there is never that moment where everybody says “What a foresightful trader that was!” and agree on the fact that yes we sure are all dead now. So even if a proprietary trader sees doom coming, they do not have much of an incentive to dutifully transmit that information backward in time in order to make the market behave now in the way that an economist thinks ought to correspond to the equilibrium it would settle into after everybody agreed that they were dead.
That incentive would only exist if you expected everybody to agree that they were going to die in a few years, far enough ahead of everybody actually being dead, for the markets to settle into equilibrium and foresightful traders to collect bonuses on having made the trade before that. Which is a much stronger and stranger thing to claim, about a planet like Earth, than the usual much weaker claim that a few sharp traders might see a fact coming, and move the markets a few years ahead of time to where they would go after everybody agreed on that fact later.
Though even then, of course, we have cases like the financial crisis of 2006-2008, where some traders did see it coming and turn huge profits, but couldn’t move enough marginal money around to actually shift the entire broader market.
To suppose that the market is broken around foomdoom is really not a remotely surprising market behavior to suppose! Even in a world where nearly all the prices are efficient relative to you and that’s why you can’t make 10%/day trading Microsoft stock!
What happened in 2006-2008 was much more broken than that! Marginal traders saw it coming, and some of them won huge even though CDSs were not trivial to short; but they didn’t move enough money to shift anything remotely as large as ‘real interest rates’ ahead of the actual materialization of the disaster.
The market’s behavior around Covid also showed much more obliviousness than this; it showed the kind of obliviousness where people I’d previously marked as the strongest EMH challengers reported collecting vast profits over a timespan of a couple of months. (But on chains of logic much less fraught than OP’s, because in real life you can’t call LTPZ movements or real interest rate changes in advance, just things on the order of ‘buy volatility’.)
We should not believe ‘the market’, in the sense of that unusually intelligent entity whose opinions we actually pay attention to, driven by the highly incentivized marginal trader, has any opinion on AGI except that “in the next few years, not everyone will have started believing that they are going to die in a few years after that”. The market is showing no actual opinion on foomdoom, only on what most market participants will believe about foomdoom in a couple of years. The usual incentive mechanism whereby, if a pandemic starts, in a few years most market participants will agree that this pandemic happened and foresightful traders will collect bonuses and spend them—as is responsible for the market sometimes but not always being foresightful, because it is paid to be foresightful—is in this case broken. We are really always seeing, when the market foretells an observable’s value a few years later, that the foresightful marginal trader thinks that a few years later lots of people will hold a certain opinion; it’s just that usually, this common opinion is being mundanely driven by a direct observation.
The market says, “It won’t be the case that in 2030, everyone agrees that they were killed by AGIs.” The market isn’t saying anything about whether that’s because everyone agrees they are alive, or because nobody is left to agree anything.
OP reads like somebody has heard that markets sometimes anticipate things ahead of them happening, that markets sometimes transmit information lossily backward in time, and doesn’t quite seem to have understood the mechanism behind it; that what everyone will agree on later, unusually foresightful marginal traders can sometimes cause the market to reflect now even though not everyone agrees on it yet. Instead they are talking about “What if the market believed...” as if this kind of market belief reflected a numerical majority of the people in the market believing that foomdoom would kill us all before 30-year bonds paid out. But this is not where the market gets its power to say things that we ought to pay special attention to (though even then market isn’t always right about those things, or even righter than us, especially if those things are a little strange, eg Covid etc). The market gets its power from unusually foresightful marginal traders expecting to get a payoff from what everyone believes later after the thing actually happened and therefore most market participants agree about what happened. And this transmission mechanism is broken w/r/t AGI doom in a way that it wouldn’t even be broken for an asteroid strike; with an asteroid strike, you might get weird effects from money losing its value in the future, but at least people could all agree on the asteroid strike coming. With AGI, I think you’d have to be pretty naive to expect everybody to agree that AGI will kill us in two years, two years before AGI kills us. So it is inappropriate to skip over a step we can usually skip over, and compress the true proposition “The market doesn’t believe that everyone in 2030 will believe that in 2030 everyone is dead”, to the false proposition “The market doesn’t believe that in 2030 everyone will be dead.”
Though again—also just to be clear—AGI ruin is a harder call than Covid and I wouldn’t strongly expect the market to get it right, even if transmission weren’t broken; and even if I thought the market would get it right after seeing GPT-4 and that transmission wasn’t broken, I wouldn’t buy “short LTPZ” as a surefire way to profit.
What links here?

EliezerYudkowsky 31 May 2015 5:02 UTC
54 points
0 ∶ 0
in reply to: Dale’s comment on: The history of the term ‘effective altruism’
There’s only so many things you can call it, and accidental namespace collisions / phrase reinventions aren’t surprising. I was surprised when I looked back myself and noticed the phrase was there, so it would be more surprising if Toby Ord remembered than if he didn’t. I’m proud to have used the term “effective altruist” once in 2007, but to say that this means I coined the term, especially when it was re-output by the more careful process described above, might be giving me too much credit—but it’s still nice to have this not-quite-coincidental mention be remembered, so thank you for that!
What links here?
- When Did EA Start? by Jeff Kaufman (25 Jan 2023 14:51 UTC; 117 points)
- When Did EA Start? by jefftk (LessWrong; 25 Jan 2023 14:30 UTC; 37 points)

EliezerYudkowsky 18 Feb 2023 17:38 UTC
53 points
29 ∶ 8
in reply to: lilly’s comment on: People Will Sometimes Just Lie About You
Trying to “steelman” the work of an experienced adversary who relies on, and is exploiting, your tendency to undercompensate and not realize how distorted these things actually are—which is the practical, hard-earned knowledge that Aella is trying to propagate—seems like a mistake.
(Actually, trying to “steelman” is a mistake in general and you should focus on passing Ideological Turing Tests instead, but that’s a much longer conversation.)

Pausing AI Developments Isn’t Enough. We Need to Shut it All Down

EliezerYudkowsky9 Apr 2023 15:53 UTC

50 points

3 comments1 min readEA link

EliezerYudkowsky 12 Nov 2022 7:00 UTC
50 points
29 ∶ 0
in reply to: Arno’s comment on: IMPCO, don’t injure yourself by returning FTXFF money for services you already provided
(Separately from all issues above, if you hold yourself responsible for returning money, you definitely should not be trying to return money now, while the FTX accounts are still being looted, first by people buying NFTs from Bahamian accounts that could still transfer out money, and now by an outright hack or “hack”.)

EliezerYudkowsky 13 Nov 2022 2:19 UTC

48 points

13 ∶ 0

Error

The value
  NIL
is not of type
  SIMPLE-STRING
when binding #:USER-ID163

EliezerYudkowsky 17 Jun 2021 17:23 UTC
47 points
0 ∶ 0
on: Taboo “Outside View”
I worriedly predict that anyone who followed your advice here would just switch to describing whatever they’re doing as “reference class forecasting” since this captures the key dynamic that makes describing what they’re doing as “outside viewing” appealing: namely, they get to pick a choice of “reference class” whose samples yield the answer they want, claim that their point is in the reference class, and then claiming that what they’re doing is what superforecasters do and what Philip Tetlock told them to do and super epistemically virtuous and anyone who argues with them gets all the burden of proof and is probably a bad person but we get to virtuously listen to them and then reject them for having used the “inside view”.
My own take: Rule One of invoking “the outside view” or “reference class forecasting” is that if a point is more dissimilar to examples in your choice of “reference class” than the examples in the “reference class” are dissimilar to each other, what you’re doing is “analogy”, not “outside viewing”.
All those experimental results on people doing well by using the outside view are results on people drawing a new sample from the same bag as previous samples. Not “arguably the same bag” or “well it’s the same bag if you look at this way”, really actually the same bag: how late you’ll be getting Christmas presents this year, based on how late you were in previous years. Superforecasters doing well by extrapolating are extrapolating a time-series over 20 years, which was a straight line over those 20 years, to another 5 years out along the same line with the same error bars, and then using that as the baseline for further adjustments with due epistemic humility about how sometimes straight lines just get interrupted some year. Not by them picking a class of 5 “relevant” historical events that all had the same outcome, and arguing that some 6th historical event goes in the same class and will have that same outcome.
What links here?

EliezerYudkowsky

IMPCO, don’t in­jure your­self by re­turn­ing FTXFF money for ser­vices you already provided