Suppose you are one of the 0.1% of macro bonds traders familiar with Yudkowskian foom. You reason as follows: “Suppose that in the next 2 years, we get even more alarming news out of GPT-4 and successors. Suppose it’s so incredibly alarming that 10% of macro traders notice, and then 10% of those hear about Yudkowskian foom scenarios. Putting myself into the shoes of one of those normie macro traders, I think I reason… that most actual normal people won’t change their saving behavior any time soon, even if theoretically they should decrease their saving, and that’s not likely to have macro effects. Still as a normie trader who’s heard about Yudkowsky foomdoom, I think I reason that if Yudkowsky’s right, we’re all dead, and if Yudkowsky’s not right, I’ll get embarrassed about a wrong trade and fired. So this normie trader won’t trade on Yudkowsky foomdoom premises. Therefore I don’t think I can profit over the next two years by shorting a TIPS fund… even leaving aside concerning feelings about whether going hugely short LTPZ would have model risk about LTPZ’s actual relation to real interest rates in these scenarios, or whether other traders would expect big AI impacts to hit measured inflation because of AI-driven lower prices or AI-driven unemployment.”
And then one week before the end of the world, the 1% of most clueful macro bonds traders… will take vacation days early, and draw down their rainy day funds to spend time with their family. They still won’t make macro trades about that, because the payoff matrix looks like “If you’re right, you’re dead and not paid, and if you’re wrong, you’re embarrassed and get fired.” Then haha whoops it turns out that the world didn’t end in a week after all, and people go back to work with a nervous laugh and a sick feeling in their stomachs, and everybody actually falls over dead three weeks later.
If Omega tells you today that everyone will be dead in 2030 with probability 1, there’s no direct way to make a market profit on that private information over the next 2 years, except insofar as foresightful traders today expect less foresightful future traders to hear about AI and read an economics textbook and decide that interest rates theoretically ought to rise and go short TIPS index funds. Foresightful traders today don’t expect this.
To put it another way: Yes, savvy market traders don’t believe that in 2025 everybody will realize that the world is ending. The savvy market traders are correct! Even at the point where the world is ending, everybody will not believe this, and so at no point will the savvy trader have made a profit! The death of all of humanity induces a market anomaly wherein savvy traders don’t expect to be able to profit from everyone else’s error because no event occurs where the real thing actually happens and everybody says “Oops” and the savvy trader gets paid off.
There just isn’t any mystery here. You can’t make a short-term profit off correcting these market prices even if Omega whispers the truth in your ear with certainty. That’s it, that’s the mystery explained, you’re done.
OP seems to ambiguate between two ideas, one true idea, and one false idea.
The true idea is that if Omega tells you personally that the world will end in 2030 with probability 1, you personally should not bother saving for retirement. Call this the Personal Idea.
The false idea is that if you believe in foomdoom, you should go long real interest rates and expect a market profit. Call this the Market Idea.
Intuitively, at least if you’re swayed by this essay, the idea in Market probably seems pretty close to the idea in Personal. If everybody started consuming for today and investing less, real interest rates would go up, right? So if you don’t believe that Market is about as strong as Personal, what invalid reasoning step occurs within the gap between the true premise in Personal to the false conclusion in Market?
Is it invalid that if in 2025 everyone started believing that the world would end in 2030 with probability 1, real interest rates would rise in 2025? Honestly, I’m not even sure of that in real life. People are arguing clever-ideas like ‘Shouldn’t everyone take out big loans due later?’ but maybe the lender doesn’t want to lend anymore, if everyone knows that. There’s a supply collapse and a demand collapse and yes I see the theoretical argument but real-world monetary stuff is in fact really strange and complicated; I didn’t see anybody calling the actual interest-rate trajectory surrounding Covid in advance of it actually playing out.
In real life, what zaps you when you think there’s a worldwide pandemic coming and try to trade interest rates, isn’t that you didn’t know about the pandemic ahead of the oblivious market, it’s that you guessed wrong about what the market would really actually do in real life as the pandemic played out and finally ended.
You can sometimes make a profit off an oblivious market, if you guess narrowly enough at reactions that are much more strongly determined. Wei Dai reports making huge profits on the Covid trade against the oblivious market there, after correctly guessing that people soon noticing Covid would at least increase expected volatility and the price of downside put options would go up.
But I don’t think anybody called “there will be a huge market drop followed by an even steeper market ascent, after the Fed’s delayed reaction to a huge drop in TIPS-predicted inflation, and people staying home and saving and trading stocks, followed by skyrocketing inflation later.”
I don’t think the style of reasoning used in the OP is the kind of thing that works reliably. It’s doing the equivalent of trying to predict something harder than ‘once the pandemic really hits, people in the short term will notice and expect more volatility and option prices will go up’; it’s trying to do the equivalent of predicting the market reaction to the Fed reaction a couple of years later.
The entire OP is written as if we lived in an alternate universe where it is way way easier than history has actually shown, to figure out what happens in broad markets after any sort of complicated or nontrivial event occurs in real life that isn’t on the order of “unexpected Fed rate hike” or “company reports much higher-than-expected profits”. And it’s written in such a way as to mislead EAs reading it about the general confidence that the field of economics is able to justly put in predictions about broad market reactions to strange things.
If you haven’t already looked at OP’s recommended investment instrument of (short) LPTZ, which holds inflation-protected Treasuries and is therefore their recommended way of tracking real interest rates, I recommend the following exercise: First try to figure out what you would have believed a priori, without benefit of hindsight, real interest rates would do over the course of a pandemic. Then, decide what investment strategy you’d have followed with LTPZ if you thought you knew about a pandemic ahead of the market. Then, decide what you think happened to real interest rates with benefit of hindsight. Then, go look at the actual price trajectory of their recommended instrument of LTPZ.
I am not sure I can properly convey this thought that I am trying to convey; I have had trouble actually conveying this thought to EAs before. The thought is that people often do long careful serious-sounding writeups which EAs then take Very Seriously, because they are so long and so seriously argued, but in fact fail to bind to reality entirely, in a way that doesn’t have to do with the details of the complicated arguments. Very serious arguments about what ought to happen to the price of an ETF that tracks 15-year TIPS, via the intermediate step of arguing about what logically ought to happen to real interest rates, are the sort of thing that, historically, average economists have not really been able to pull off; it’s a kind of thought that you should expect fails to achieve basic binding to reality. What would LTPZ or its post-facto equivalent have been doing around the time of the Cuban Missile Crisis? My model says ‘no prediction’; they’ll have done whatever. Afterwards somebody will make up a story about it in hindsight, but it is not the sort of thing where history says that long complicated analyses are remotely reliably good at doing it in advance.
But there are even weaker links in the argument, so let’s accept the LTPZ step arguendo and pass on.
An even bigger problem is that, since everybody is going to die before anything really pays out, the marginal foresightful trader does not have a strong incentive to early-on move the market toward where the market would end up in equilibrium after everyone agreed on the actual facts of the matter and had time to trade about them.
Prediction markets, I sometimes explain to people, are tools for transmitting future observables, or more generally propositions that people expect to publicly agree on at some future point even if they don’t agree now, lossily backward in time, manifesting as well-calibrated probability distributions.
To run a prediction market, you first and foremost need a future publicly observable measurement, which is a special case of a place where we expect most people to agree on an extreme probability assignment later, even though they don’t agree now or don’t make extreme probability assignments now. You cannot run a prediction market on whether supplementing lots of potassium can produce weight loss; you can only run a prediction market about what an experiment will report in the way of results, or what a particular judge will say the experimental evidence seems to have indicated in five years. You cannot directly observe “whether potassium causes weight loss” as an underlying fact of biology, so you can’t have a prediction market about that; you can only observe what somebody reports as an experimental result, or what a particular person appointed as judge says out loud about the state of evidence later.
The marginal foresightful trader usually has a motive to run ahead of the market and make trades now, based on where the equilibrium ought to settle later; not because they are nobly undertaking a grand altruistic project of transmitting facts backward in time and making the market behave nicely from the standpoint of theoretical economics, but because they expect to get paid after everyone makes the common observation and the market settles into a new equilibrium reflecting that state of knowledge. And then they expect to have that money, or to get a bonus for earning that money for their proprietary trading firm, and for that money to still hold its value, and for them to be able to spend the money on nice things.
In the unusual case of foomdoom, even if doom proceeds slowly enough that a large-enough group of marginal foresightful traders see the foomdoom coming, even if there is somehow a really definitive fire alarm that says “extremely high probability you are dead within two to four years”, it is incredibly unlikely that everyone in the world will agree that yes we’ll all be dead in two to four years, and that the markets will settle into the equilibrium that an economist would say corresponds to the belief that we’ll all be dead in two to four years; which is what’s required for the foresightful proprietary trader to score a huge profit and get a big bonus that year and have time to spend it on some fancy way of passing the remaining time.
People do not usually agree on what will happen in two to four years. This kind of agreement that reliably reflects a fact, and makes a market pay out in a way that you can trust to correspond to that fact, is usually achieved after that fact is publicly definitively observed, not two to four years ahead of the observation.
In case of foomdoom the world never settles into equilibrium later, the bets never pay out, there is never that moment where everybody says “What a foresightful trader that was!” and agree on the fact that yes we sure are all dead now. So even if a proprietary trader sees doom coming, they do not have much of an incentive to dutifully transmit that information backward in time in order to make the market behave now in the way that an economist thinks ought to correspond to the equilibrium it would settle into after everybody agreed that they were dead.
That incentive would only exist if you expected everybody to agree that they were going to die in a few years, far enough ahead of everybody actually being dead, for the markets to settle into equilibrium and foresightful traders to collect bonuses on having made the trade before that. Which is a much stronger and stranger thing to claim, about a planet like Earth, than the usual much weaker claim that a few sharp traders might see a fact coming, and move the markets a few years ahead of time to where they would go after everybody agreed on that fact later.
Though even then, of course, we have cases like the financial crisis of 2006-2008, where some traders did see it coming and turn huge profits, but couldn’t move enough marginal money around to actually shift the entire broader market.
To suppose that the market is broken around foomdoom is really not a remotely surprising market behavior to suppose! Even in a world where nearly all the prices are efficient relative to you and that’s why you can’t make 10%/day trading Microsoft stock!
What happened in 2006-2008 was much more broken than that! Marginal traders saw it coming, and some of them won huge even though CDSs were not trivial to short; but they didn’t move enough money to shift anything remotely as large as ‘real interest rates’ ahead of the actual materialization of the disaster.
The market’s behavior around Covid also showed much more obliviousness than this; it showed the kind of obliviousness where people I’d previously marked as the strongest EMH challengers reported collecting vast profits over a timespan of a couple of months. (But on chains of logic much less fraught than OP’s, because in real life you can’t call LTPZ movements or real interest rate changes in advance, just things on the order of ‘buy volatility’.)
We should not believe ‘the market’, in the sense of that unusually intelligent entity whose opinions we actually pay attention to, driven by the highly incentivized marginal trader, has any opinion on AGI except that “in the next few years, not everyone will have started believing that they are going to die in a few years after that”. The market is showing no actual opinion on foomdoom, only on what most market participants will believe about foomdoom in a couple of years. The usual incentive mechanism whereby, if a pandemic starts, in a few years most market participants will agree that this pandemic happened and foresightful traders will collect bonuses and spend them—as is responsible for the market sometimes but not always being foresightful, because it is paid to be foresightful—is in this case broken. We are really always seeing, when the market foretells an observable’s value a few years later, that the foresightful marginal trader thinks that a few years later lots of people will hold a certain opinion; it’s just that usually, this common opinion is being mundanely driven by a direct observation.
The market says, “It won’t be the case that in 2030, everyone agrees that they were killed by AGIs.” The market isn’t saying anything about whether that’s because everyone agrees they are alive, or because nobody is left to agree anything.
OP reads like somebody has heard that markets sometimes anticipate things ahead of them happening, that markets sometimes transmit information lossily backward in time, and doesn’t quite seem to have understood the mechanism behind it; that what everyone will agree on later, unusually foresightful marginal traders can sometimes cause the market to reflect now even though not everyone agrees on it yet. Instead they are talking about “What if the market believed...” as if this kind of market belief reflected a numerical majority of the people in the market believing that foomdoom would kill us all before 30-year bonds paid out. But this is not where the market gets its power to say things that we ought to pay special attention to (though even then market isn’t always right about those things, or even righter than us, especially if those things are a little strange, eg Covid etc). The market gets its power from unusually foresightful marginal traders expecting to get a payoff from what everyone believes later after the thing actually happened and therefore most market participants agree about what happened. And this transmission mechanism is broken w/r/t AGI doom in a way that it wouldn’t even be broken for an asteroid strike; with an asteroid strike, you might get weird effects from money losing its value in the future, but at least people could all agree on the asteroid strike coming. With AGI, I think you’d have to be pretty naive to expect everybody to agree that AGI will kill us in two years, two years before AGI kills us. So it is inappropriate to skip over a step we can usually skip over, and compress the true proposition “The market doesn’t believe that everyone in 2030 will believe that in 2030 everyone is dead”, to the false proposition “The market doesn’t believe that in 2030 everyone will be dead.”
Though again—also just to be clear—AGI ruin is a harder call than Covid and I wouldn’t strongly expect the market to get it right, even if transmission weren’t broken; and even if I thought the market would get it right after seeing GPT-4 and that transmission wasn’t broken, I wouldn’t buy “short LTPZ” as a surefire way to profit.
What would LTPZ or its post-facto equivalent have been doing around the time of the Cuban Missile Crisis? My model says ‘no prediction’; they’ll have done whatever. Afterwards somebody will make up a story about it in hindsight, but it is not the sort of thing where history says that long complicated analyses are remotely reliably good at doing it in advance.
In appendix 3 the authors cite a paper which looks at more-or-less this precise thing:
Figure 4.6: Observed Treasury and aggregate-equity movements around President Kennedy’s 22 October 1962 19:00 EDT Cuba address. For the cumulative value-weighted CRSP return, a positive value on a date up to and including that of the address indicates a negative return through the date of the address, and a negative value after the address indicates a negative return from the first trading day after the address.
It seems like government interest rates didn’t change much? But I don’t think I understand this graph.
Inflation linked bonds are recent, so virtually all historical analyses are going to use nominal rates. I think this is a tradeoff well worth making to say something about asset pricing under existential risk. For the effects to be wrong because of nominality, the Cuban missile crisis would have had to affect market perceptions primarily because of… inflation expectations. I feel comfortable rejecting that as a story.
I’m a bit confused by this comment. Is the claim that you can’t profit from taking out a loan that you have to pay back in 2040? The fact that you get money now and have to pay it back at a time when hypothetically money no longer matters seems like profit to me. If Omega whispers the truth into my ear that in 2040 there will be foom, then that’s guaranteed profit.
Nobody will give you an unsecured loan to fund consumption or donations with most of the money not due for 15+ years; most people in our society who would borrow on such terms would default. (You can get close with some types of student loan, so if there’s education that you’d experience as intrinsically-valued consumption or be able to rapidly apply to philanthropic ends then this post suggests you should perhaps be more willing to borrow to fund it than you would be otherwise, but your personal upside there is pretty limited.)
It might be challenging to borrow (though I’m not sure), but there seem to be plenty of sophisticated entities that should be selling off their bonds and aren’t. The top-level comment does cut into the gains from shorting (as the OP concedes), but I think it’s right that there are borrowing-esque things to do.
The reason sophisticated entities like e.g. hedge funds hold bonds isn’t so they can collect a cash flow 10 years from now. It’s because they think bond prices will go up tomorrow, or next year.
The big entities that hold bonds for the future cash flows are e.g. pension funds. It would be very surprising and (I think) borderline illegal if the pension funds ever started reasoning, “I guess I don’t need to worry about cash flows after 2045, since the world will probably end before then. So I’ll just hold shorter-term assets.”
I think this adds up to, no big investors can directly profit from the final outcome here. Though as everyone seems to agree, anyone could profit by being short bonds (or underweight bonds) while the market started to price in substantial probability of AGI.
If you’re in charge of investing decisions for a pension fund or sovereign wealth fund or similar, you likely can’t personally derive any benefit from having the fund sell off its bonds and other long-term assets now. You might do this in your personal account but the impact will be small.
For government bonds in particular it also seems relevant that I think most are held by entities that are effectively required to hold them for some reason (e.g. bank capital requirements, pension fund regulations) or otherwise oddly insensitive to their low ROI compared to alternatives. See also the “equity premium puzzle”.
I don’t know whether I defend Yudkowsky’s view (I only skimmed), but:
The “profit” in the scenario you describe doesn’t seem sufficient to move the market (as small investors), because it isn’t “profit” that you can keep growing and reinvesting (snowballing) until your Special Insights (TM) fix the market;
If you end up with a scenario where 2040 arrives and it’s Utopia, then there may not be any serious incentive to care, whereas there would be downside risks if it doesn’t occur: if you’re right, cool, you lived slightly better for 20 years out of a 1,000,000,000-year life of happiness; if you’re wrong you might be in so much debt you can’t pay medical bills/whatever, and die before actual AGI date.
I’m not confident you could get massive loans with no collateral and no business model or whatnot to repay the loans come 2040.
Beyond just taking vacation days, if you’re a bond trader who believes in a very high chance of xrisk in the next five years it probably might make sense to quit your job and fund your consumption out of your retirement savings. At which point you aren’t a bond trader anymore and your beliefs no longer have much impact on bond prices.
Insofar as humans care about x-risk and act on this care, you’d expect bond traders to be people who either think x-risk is very low, think it’s high and think they have no nontrivial way to affect it, or think it’s high and think trading is their best way to affect it (e.g., through earning to give).
My guess for the pushback: 1 week before the end of the world, you think a sizable part of the population will notice and change their economic behavior drastically. I imagine this scenario contains a slow “attack” by AI that everyone sees coming?
If the only scenario for AI were: exist at more or less normal levels of economic growth, then foom, I think Eliezer would be correct. However, I think a likely scenario is that there is accelerated growth, coincident with accelerated risk (via e.g. AI terrorism or wars), before foom. This cluster of outcomes may even be modal. In that case, interest rates would very likely rise and traders would be able to profit, before foom.
Put another way, we often place substantial weight on non-foom good and bad AI scenarios, even if foom is a risk. The market seems to be ruling out non-foom AI risk as well as non-foom AI growth before foom. Eliezer is correct that the market may not be ruling out the (IMO unlikely) scenario of “no-AI induced growth or risk prior to foom, then foom.”
I’m trying to make sure I understand: Is this (a more colorful version) of the same point as the OP makes at the end of “Bet on real rates rising”?
The other risk that could motivate not making this bet is the risk that the market – for some unspecified reason – never has a chance to correct, because (1) transformative AI ends up unaligned and (2) humanity’s conversion into paperclips occurs overnight. This would prevent the market from ever “waking up”.
However, to be clear, expecting this specific scenario requires both:
Buying into specific stories about how takeoff will occur: specifically, Yudkowskian foom-type scenarios with fast takeoff.
Having a lot of skepticism about the optimization forces pushing financial markets towards informational efficiency.
You should be sure that your beliefs are actually congruent with these requirements, if you want to refuse to bet that real rates will rise. Additionally, we will see that the second suggestion in this section (“impatient philanthropy”) is not affected by the possibility of foom scenarios.
I wouldn’t say that I have “a lot of” skepticism about the applicability of the EMH in this case; you only need realism to believe that the bar is above USDT and Covid, for a case where nobody ever says ‘oops’ and the market never pays out.
Not until timelines are even blatantly shorter than present and long-term loans are on offer, and not unless there’s something useful to actually do with the money.
Suppose you are one of the 0.1% of macro bonds traders familiar with Yudkowskian foom. You reason as follows: “Suppose that in the next 2 years, we get even more alarming news out of GPT-4 and successors. Suppose it’s so incredibly alarming that 10% of macro traders notice, and then 10% of those hear about Yudkowskian foom scenarios. Putting myself into the shoes of one of those normie macro traders, I think I reason… that most actual normal people won’t change their saving behavior any time soon, even if theoretically they should decrease their saving, and that’s not likely to have macro effects. Still as a normie trader who’s heard about Yudkowsky foomdoom, I think I reason that if Yudkowsky’s right, we’re all dead, and if Yudkowsky’s not right, I’ll get embarrassed about a wrong trade and fired. So this normie trader won’t trade on Yudkowsky foomdoom premises. Therefore I don’t think I can profit over the next two years by shorting a TIPS fund… even leaving aside concerning feelings about whether going hugely short LTPZ would have model risk about LTPZ’s actual relation to real interest rates in these scenarios, or whether other traders would expect big AI impacts to hit measured inflation because of AI-driven lower prices or AI-driven unemployment.”
And then one week before the end of the world, the 1% of most clueful macro bonds traders… will take vacation days early, and draw down their rainy day funds to spend time with their family. They still won’t make macro trades about that, because the payoff matrix looks like “If you’re right, you’re dead and not paid, and if you’re wrong, you’re embarrassed and get fired.” Then haha whoops it turns out that the world didn’t end in a week after all, and people go back to work with a nervous laugh and a sick feeling in their stomachs, and everybody actually falls over dead three weeks later.
If Omega tells you today that everyone will be dead in 2030 with probability 1, there’s no direct way to make a market profit on that private information over the next 2 years, except insofar as foresightful traders today expect less foresightful future traders to hear about AI and read an economics textbook and decide that interest rates theoretically ought to rise and go short TIPS index funds. Foresightful traders today don’t expect this.
To put it another way: Yes, savvy market traders don’t believe that in 2025 everybody will realize that the world is ending. The savvy market traders are correct! Even at the point where the world is ending, everybody will not believe this, and so at no point will the savvy trader have made a profit! The death of all of humanity induces a market anomaly wherein savvy traders don’t expect to be able to profit from everyone else’s error because no event occurs where the real thing actually happens and everybody says “Oops” and the savvy trader gets paid off.
There just isn’t any mystery here. You can’t make a short-term profit off correcting these market prices even if Omega whispers the truth in your ear with certainty. That’s it, that’s the mystery explained, you’re done.
Trying again:
OP seems to ambiguate between two ideas, one true idea, and one false idea.
The true idea is that if Omega tells you personally that the world will end in 2030 with probability 1, you personally should not bother saving for retirement. Call this the Personal Idea.
The false idea is that if you believe in foomdoom, you should go long real interest rates and expect a market profit. Call this the Market Idea.
Intuitively, at least if you’re swayed by this essay, the idea in Market probably seems pretty close to the idea in Personal. If everybody started consuming for today and investing less, real interest rates would go up, right? So if you don’t believe that Market is about as strong as Personal, what invalid reasoning step occurs within the gap between the true premise in Personal to the false conclusion in Market?
Is it invalid that if in 2025 everyone started believing that the world would end in 2030 with probability 1, real interest rates would rise in 2025? Honestly, I’m not even sure of that in real life. People are arguing clever-ideas like ‘Shouldn’t everyone take out big loans due later?’ but maybe the lender doesn’t want to lend anymore, if everyone knows that. There’s a supply collapse and a demand collapse and yes I see the theoretical argument but real-world monetary stuff is in fact really strange and complicated; I didn’t see anybody calling the actual interest-rate trajectory surrounding Covid in advance of it actually playing out.
In real life, what zaps you when you think there’s a worldwide pandemic coming and try to trade interest rates, isn’t that you didn’t know about the pandemic ahead of the oblivious market, it’s that you guessed wrong about what the market would really actually do in real life as the pandemic played out and finally ended.
You can sometimes make a profit off an oblivious market, if you guess narrowly enough at reactions that are much more strongly determined. Wei Dai reports making huge profits on the Covid trade against the oblivious market there, after correctly guessing that people soon noticing Covid would at least increase expected volatility and the price of downside put options would go up.
But I don’t think anybody called “there will be a huge market drop followed by an even steeper market ascent, after the Fed’s delayed reaction to a huge drop in TIPS-predicted inflation, and people staying home and saving and trading stocks, followed by skyrocketing inflation later.”
I don’t think the style of reasoning used in the OP is the kind of thing that works reliably. It’s doing the equivalent of trying to predict something harder than ‘once the pandemic really hits, people in the short term will notice and expect more volatility and option prices will go up’; it’s trying to do the equivalent of predicting the market reaction to the Fed reaction a couple of years later.
The entire OP is written as if we lived in an alternate universe where it is way way easier than history has actually shown, to figure out what happens in broad markets after any sort of complicated or nontrivial event occurs in real life that isn’t on the order of “unexpected Fed rate hike” or “company reports much higher-than-expected profits”. And it’s written in such a way as to mislead EAs reading it about the general confidence that the field of economics is able to justly put in predictions about broad market reactions to strange things.
If you haven’t already looked at OP’s recommended investment instrument of (short) LPTZ, which holds inflation-protected Treasuries and is therefore their recommended way of tracking real interest rates, I recommend the following exercise: First try to figure out what you would have believed a priori, without benefit of hindsight, real interest rates would do over the course of a pandemic. Then, decide what investment strategy you’d have followed with LTPZ if you thought you knew about a pandemic ahead of the market. Then, decide what you think happened to real interest rates with benefit of hindsight. Then, go look at the actual price trajectory of their recommended instrument of LTPZ.
I am not sure I can properly convey this thought that I am trying to convey; I have had trouble actually conveying this thought to EAs before. The thought is that people often do long careful serious-sounding writeups which EAs then take Very Seriously, because they are so long and so seriously argued, but in fact fail to bind to reality entirely, in a way that doesn’t have to do with the details of the complicated arguments. Very serious arguments about what ought to happen to the price of an ETF that tracks 15-year TIPS, via the intermediate step of arguing about what logically ought to happen to real interest rates, are the sort of thing that, historically, average economists have not really been able to pull off; it’s a kind of thought that you should expect fails to achieve basic binding to reality. What would LTPZ or its post-facto equivalent have been doing around the time of the Cuban Missile Crisis? My model says ‘no prediction’; they’ll have done whatever. Afterwards somebody will make up a story about it in hindsight, but it is not the sort of thing where history says that long complicated analyses are remotely reliably good at doing it in advance.
But there are even weaker links in the argument, so let’s accept the LTPZ step arguendo and pass on.
An even bigger problem is that, since everybody is going to die before anything really pays out, the marginal foresightful trader does not have a strong incentive to early-on move the market toward where the market would end up in equilibrium after everyone agreed on the actual facts of the matter and had time to trade about them.
Prediction markets, I sometimes explain to people, are tools for transmitting future observables, or more generally propositions that people expect to publicly agree on at some future point even if they don’t agree now, lossily backward in time, manifesting as well-calibrated probability distributions.
To run a prediction market, you first and foremost need a future publicly observable measurement, which is a special case of a place where we expect most people to agree on an extreme probability assignment later, even though they don’t agree now or don’t make extreme probability assignments now. You cannot run a prediction market on whether supplementing lots of potassium can produce weight loss; you can only run a prediction market about what an experiment will report in the way of results, or what a particular judge will say the experimental evidence seems to have indicated in five years. You cannot directly observe “whether potassium causes weight loss” as an underlying fact of biology, so you can’t have a prediction market about that; you can only observe what somebody reports as an experimental result, or what a particular person appointed as judge says out loud about the state of evidence later.
The marginal foresightful trader usually has a motive to run ahead of the market and make trades now, based on where the equilibrium ought to settle later; not because they are nobly undertaking a grand altruistic project of transmitting facts backward in time and making the market behave nicely from the standpoint of theoretical economics, but because they expect to get paid after everyone makes the common observation and the market settles into a new equilibrium reflecting that state of knowledge. And then they expect to have that money, or to get a bonus for earning that money for their proprietary trading firm, and for that money to still hold its value, and for them to be able to spend the money on nice things.
In the unusual case of foomdoom, even if doom proceeds slowly enough that a large-enough group of marginal foresightful traders see the foomdoom coming, even if there is somehow a really definitive fire alarm that says “extremely high probability you are dead within two to four years”, it is incredibly unlikely that everyone in the world will agree that yes we’ll all be dead in two to four years, and that the markets will settle into the equilibrium that an economist would say corresponds to the belief that we’ll all be dead in two to four years; which is what’s required for the foresightful proprietary trader to score a huge profit and get a big bonus that year and have time to spend it on some fancy way of passing the remaining time.
People do not usually agree on what will happen in two to four years. This kind of agreement that reliably reflects a fact, and makes a market pay out in a way that you can trust to correspond to that fact, is usually achieved after that fact is publicly definitively observed, not two to four years ahead of the observation.
In case of foomdoom the world never settles into equilibrium later, the bets never pay out, there is never that moment where everybody says “What a foresightful trader that was!” and agree on the fact that yes we sure are all dead now. So even if a proprietary trader sees doom coming, they do not have much of an incentive to dutifully transmit that information backward in time in order to make the market behave now in the way that an economist thinks ought to correspond to the equilibrium it would settle into after everybody agreed that they were dead.
That incentive would only exist if you expected everybody to agree that they were going to die in a few years, far enough ahead of everybody actually being dead, for the markets to settle into equilibrium and foresightful traders to collect bonuses on having made the trade before that. Which is a much stronger and stranger thing to claim, about a planet like Earth, than the usual much weaker claim that a few sharp traders might see a fact coming, and move the markets a few years ahead of time to where they would go after everybody agreed on that fact later.
Though even then, of course, we have cases like the financial crisis of 2006-2008, where some traders did see it coming and turn huge profits, but couldn’t move enough marginal money around to actually shift the entire broader market.
To suppose that the market is broken around foomdoom is really not a remotely surprising market behavior to suppose! Even in a world where nearly all the prices are efficient relative to you and that’s why you can’t make 10%/day trading Microsoft stock!
What happened in 2006-2008 was much more broken than that! Marginal traders saw it coming, and some of them won huge even though CDSs were not trivial to short; but they didn’t move enough money to shift anything remotely as large as ‘real interest rates’ ahead of the actual materialization of the disaster.
The market’s behavior around Covid also showed much more obliviousness than this; it showed the kind of obliviousness where people I’d previously marked as the strongest EMH challengers reported collecting vast profits over a timespan of a couple of months. (But on chains of logic much less fraught than OP’s, because in real life you can’t call LTPZ movements or real interest rate changes in advance, just things on the order of ‘buy volatility’.)
We should not believe ‘the market’, in the sense of that unusually intelligent entity whose opinions we actually pay attention to, driven by the highly incentivized marginal trader, has any opinion on AGI except that “in the next few years, not everyone will have started believing that they are going to die in a few years after that”. The market is showing no actual opinion on foomdoom, only on what most market participants will believe about foomdoom in a couple of years. The usual incentive mechanism whereby, if a pandemic starts, in a few years most market participants will agree that this pandemic happened and foresightful traders will collect bonuses and spend them—as is responsible for the market sometimes but not always being foresightful, because it is paid to be foresightful—is in this case broken. We are really always seeing, when the market foretells an observable’s value a few years later, that the foresightful marginal trader thinks that a few years later lots of people will hold a certain opinion; it’s just that usually, this common opinion is being mundanely driven by a direct observation.
The market says, “It won’t be the case that in 2030, everyone agrees that they were killed by AGIs.” The market isn’t saying anything about whether that’s because everyone agrees they are alive, or because nobody is left to agree anything.
OP reads like somebody has heard that markets sometimes anticipate things ahead of them happening, that markets sometimes transmit information lossily backward in time, and doesn’t quite seem to have understood the mechanism behind it; that what everyone will agree on later, unusually foresightful marginal traders can sometimes cause the market to reflect now even though not everyone agrees on it yet. Instead they are talking about “What if the market believed...” as if this kind of market belief reflected a numerical majority of the people in the market believing that foomdoom would kill us all before 30-year bonds paid out. But this is not where the market gets its power to say things that we ought to pay special attention to (though even then market isn’t always right about those things, or even righter than us, especially if those things are a little strange, eg Covid etc). The market gets its power from unusually foresightful marginal traders expecting to get a payoff from what everyone believes later after the thing actually happened and therefore most market participants agree about what happened. And this transmission mechanism is broken w/r/t AGI doom in a way that it wouldn’t even be broken for an asteroid strike; with an asteroid strike, you might get weird effects from money losing its value in the future, but at least people could all agree on the asteroid strike coming. With AGI, I think you’d have to be pretty naive to expect everybody to agree that AGI will kill us in two years, two years before AGI kills us. So it is inappropriate to skip over a step we can usually skip over, and compress the true proposition “The market doesn’t believe that everyone in 2030 will believe that in 2030 everyone is dead”, to the false proposition “The market doesn’t believe that in 2030 everyone will be dead.”
Though again—also just to be clear—AGI ruin is a harder call than Covid and I wouldn’t strongly expect the market to get it right, even if transmission weren’t broken; and even if I thought the market would get it right after seeing GPT-4 and that transmission wasn’t broken, I wouldn’t buy “short LTPZ” as a surefire way to profit.
In appendix 3 the authors cite a paper which looks at more-or-less this precise thing:
It seems like government interest rates didn’t change much? But I don’t think I understand this graph.
Those look like nominal rates, not real rates.
Inflation linked bonds are recent, so virtually all historical analyses are going to use nominal rates. I think this is a tradeoff well worth making to say something about asset pricing under existential risk. For the effects to be wrong because of nominality, the Cuban missile crisis would have had to affect market perceptions primarily because of… inflation expectations. I feel comfortable rejecting that as a story.
I’m a bit confused by this comment. Is the claim that you can’t profit from taking out a loan that you have to pay back in 2040? The fact that you get money now and have to pay it back at a time when hypothetically money no longer matters seems like profit to me. If Omega whispers the truth into my ear that in 2040 there will be foom, then that’s guaranteed profit.
Nobody will give you an unsecured loan to fund consumption or donations with most of the money not due for 15+ years; most people in our society who would borrow on such terms would default. (You can get close with some types of student loan, so if there’s education that you’d experience as intrinsically-valued consumption or be able to rapidly apply to philanthropic ends then this post suggests you should perhaps be more willing to borrow to fund it than you would be otherwise, but your personal upside there is pretty limited.)
It might be challenging to borrow (though I’m not sure), but there seem to be plenty of sophisticated entities that should be selling off their bonds and aren’t. The top-level comment does cut into the gains from shorting (as the OP concedes), but I think it’s right that there are borrowing-esque things to do.
The reason sophisticated entities like e.g. hedge funds hold bonds isn’t so they can collect a cash flow 10 years from now. It’s because they think bond prices will go up tomorrow, or next year.
The big entities that hold bonds for the future cash flows are e.g. pension funds. It would be very surprising and (I think) borderline illegal if the pension funds ever started reasoning, “I guess I don’t need to worry about cash flows after 2045, since the world will probably end before then. So I’ll just hold shorter-term assets.”
I think this adds up to, no big investors can directly profit from the final outcome here. Though as everyone seems to agree, anyone could profit by being short bonds (or underweight bonds) while the market started to price in substantial probability of AGI.
If you’re in charge of investing decisions for a pension fund or sovereign wealth fund or similar, you likely can’t personally derive any benefit from having the fund sell off its bonds and other long-term assets now. You might do this in your personal account but the impact will be small.
For government bonds in particular it also seems relevant that I think most are held by entities that are effectively required to hold them for some reason (e.g. bank capital requirements, pension fund regulations) or otherwise oddly insensitive to their low ROI compared to alternatives. See also the “equity premium puzzle”.
I don’t know whether I defend Yudkowsky’s view (I only skimmed), but:
The “profit” in the scenario you describe doesn’t seem sufficient to move the market (as small investors), because it isn’t “profit” that you can keep growing and reinvesting (snowballing) until your Special Insights (TM) fix the market;
If you end up with a scenario where 2040 arrives and it’s Utopia, then there may not be any serious incentive to care, whereas there would be downside risks if it doesn’t occur: if you’re right, cool, you lived slightly better for 20 years out of a 1,000,000,000-year life of happiness; if you’re wrong you might be in so much debt you can’t pay medical bills/whatever, and die before actual AGI date.
I’m not confident you could get massive loans with no collateral and no business model or whatnot to repay the loans come 2040.
Beyond just taking vacation days, if you’re a bond trader who believes in a very high chance of xrisk in the next five years it
probablymight make sense to quit your job and fund your consumption out of your retirement savings. At which point you aren’t a bond trader anymore and your beliefs no longer have much impact on bond prices.Or leave your job to go try and reduce x-risk?
Insofar as humans care about x-risk and act on this care, you’d expect bond traders to be people who either think x-risk is very low, think it’s high and think they have no nontrivial way to affect it, or think it’s high and think trading is their best way to affect it (e.g., through earning to give).
People disagreeing, would you say why?
My guess for the pushback: 1 week before the end of the world, you think a sizable part of the population will notice and change their economic behavior drastically. I imagine this scenario contains a slow “attack” by AI that everyone sees coming?
(agree vote = yeah that is the pushback)
If the only scenario for AI were: exist at more or less normal levels of economic growth, then foom, I think Eliezer would be correct. However, I think a likely scenario is that there is accelerated growth, coincident with accelerated risk (via e.g. AI terrorism or wars), before foom. This cluster of outcomes may even be modal. In that case, interest rates would very likely rise and traders would be able to profit, before foom.
Put another way, we often place substantial weight on non-foom good and bad AI scenarios, even if foom is a risk. The market seems to be ruling out non-foom AI risk as well as non-foom AI growth before foom. Eliezer is correct that the market may not be ruling out the (IMO unlikely) scenario of “no-AI induced growth or risk prior to foom, then foom.”
I’m trying to make sure I understand: Is this (a more colorful version) of the same point as the OP makes at the end of “Bet on real rates rising”?
I wouldn’t say that I have “a lot of” skepticism about the applicability of the EMH in this case; you only need realism to believe that the bar is above USDT and Covid, for a case where nobody ever says ‘oops’ and the market never pays out.
What do you make of the ‘impatient philanthropy’ argument? Do you think EAs should be borrowing to spend on AI safety?
Not until timelines are even blatantly shorter than present and long-term loans are on offer, and not unless there’s something useful to actually do with the money.
Am I being dumb or do you mean short TIPS? If real interest rates rise, TIPS go down.
Corrected, thanks.
Yes—you are correct.