Product Director of 8 years, interests in Capital, Prediction markets, Semiconductors. AMF monthly donor for 8 years.
Cameron Holmes
Karma: 28
I think most of my reservations are mostly deontological, plus a few fringe possibilities
Thank you Jonny, admittedly I only made it to one event but it was my first in person interaction with an EA group and I really enjoyed it and found you very welcoming.
I just find it delightful that that HPMOR is the start of so many people’s EA origin story, partly just as a curiosity as I had an opposite path to so many people (AMF > EA > LW > HPMOR)
Presumably there are many people alive today because of a chain of events started with EY writing a fanfic of all things.
Great post
Regarding the second point about how EAs (or anyone else) might exploit an inefficiency in this space, I think it’s tricky just because the amount of other risks that inform the pricing of long-dated bonds. Many of these (climate, demographics, geopolitics, populism etc..) could wipe out any short (or especially leveraged short) position before TAI is realised.As noted in my other comment I expect for someone with high-conviction views on short TAI timelines there are bets that are:
Much higher in expected returns
Less capital intensive
Less susceptible to other risks
Examples of these bets are broadly discussed elsewhere but often are related to long/short equity bets on disrupting/disrupted companies and companies part of the supply chain (semiconductors design/fab/tooling, datacentre, data aggregators, communications etc..)
I think perhaps at best short long-dated bonds could form part of a short-timelines TAI bet in order to hedge against long positions elsewhere/maintain neutrality against other factors rather than the core position. It feels likely there are considerably better options for someone taking such a bet (as you allude to in the opportunities for future work)
Regarding the first point about the extent to which we should update timelines based on the fact the bond market is not pricing in short timelines for TAI; my prior is that in general the fixed income (bonds) markets are fairly efficient and are more sophisticated/efficient than equity markets. This leads me to initially believe we likely should update based on this/consider it more strongly than bullish equity sentiment towards some AI themes.
However on the flipside I think the size of this market does mean it can retain inefficiencies around subtle themes for longer. I think of this as a form of Expecting Short Inferential Distances—there are a lot of inferential reasoning steps around TAI, scaling, take-off etc.. which make it slower for conviction to spread when compared to something like demographic shifts which have a much more straightforward causality. This is relevant because to move government bond markets requires people to take this bet with a huge amount of assets as this is a very capital-intensive trade with a lot of exposure to other uncorrelated risks/conflating variable (climate, demographics, geopolitics, populism etc..). The reason I think it may be unlikely that many people are making this bet is related to this:
An analysis of the most capital-efficient way to bet on short AI timelines and the possible expected returns (“the greatest trade of all time”).
I suspect there are far more highly levered bets that market participants with a high-conviction belief in short TAI timelines could take, potentially diluting the impact on lower beta instruments (like bonds). For example I expect even being long fairly broad equity markets might outperform this bet and much more targeted bets (especially if they could be hedged against other risks, bringing them closer to a ‘pure’ TAI bet) could be expected to return many multiples of the short-US30Y trade.
If the amount of money being managed by those with high conviction TAI views is ‘small’ (<<$100bn) then I expect there are many more favourable inefficiencies/price dislocations for them to exploit and not a sufficient mass of ‘smart TAI’ money to spill over into long-dated bonds.
I’m completely sold on the arguments in general EV terms (the vast suffering, tractability, importance, neglect—even within EA), up to the limits of how confident I can be about anything this complex. That’s basically the fringe possibilities—weird second, third-order impacts from the messiness of life that mean I couldn’t be >98% on something like this.
The deontological point was that maybe there is a good reason I should only care or vastly weight humans over animals through some moral obligation. I don’t currently believe that but I’m hedging for it, because I could be convinced.
I realise now I’m basically saying I 90% agree that rolling a D20 for 3+ is a good idea, when it would be fair to also interpret it that I 100% agree it’s a good idea ex ante.
(Also my first comment was terrible, sorry I just wanted to get on the board on priors before reading the debate)