Ted Sanders

Karma: 437

Ted Sanders Jun 8, 2023, 10:38 PM
2 points
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
In particular, it seems like some of your estimates make more sense to me if I read them as saying “Well there will likely exist some task that AI systems can’t do.” But I think such claims aren’t very relevant for transformative AI, which would in turn lead to AGI.
By the same token, if the AIs were looking at humans they might say “Well there will exist some tasks that humans can’t do” and of course they’d be right, but the relevant thing is the single non-cherry-picked variable of overall economic impact. The AIs would be wrong to conclude that humans have slow economic growth because we can’t do some tasks that AIs are great at, and the humans would be wrong to conclude that AIs will have slow economic growth because they can’t do some tasks we are great at. The exact comparison is only relevant for assessing things like complementarity, which make large impacts happen strictly more quickly than they would otherwise.
(This might be related to me disliking AGI though, and then it’s kind of on OpenPhil for asking about it. They could also have asked about timelines to 100000x electricity production and I’d be making broadly the same arguments, so in some sense it must be me who is missing the point.)
Yep. We’re using the main definition supplied by Open Philanthropy, which I’ll paraphrase as “nearly all human work at human cost or less by 2043.”
If the definition was more liberal, e.g., AGI as smart as humans, or AI causing world GDP to rise by >100%, we would have forecasted higher probabilities. We expect AI to get wildly more powerful over the next decades and wildly change the face of human life and work. The public is absolutely unprepared. We are very bullish on AI progress, and we think AI safety is an important, tractable, and neglected problem. Creating new entities with the potential to be more powerful than humanity is a scary, scary thing.

Ted Sanders Jun 8, 2023, 10:32 PM
1 point
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
I don’t know what “avoid derailment” means. It seems like these are just factors that affect the earlier estimates, so I guess the earlier quantities were supposed to be something like “the probability of developing AGI given that nothing weird happens in the world”? Or something?
Bingo. We didn’t take the time to articulate it fully, but yeah you got it. We think it makes it easier to forecast these things separately rather than invisibly smushing them together into a smaller set of factors.
But weird stuff is guaranteed to be happening in the world. I feel like this is the same deal as above, you should be multiplying out factors.
We are multiplying out factors. Not sure I follow you here.

Ted Sanders Jun 8, 2023, 10:29 PM
1 point
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
I don’t know what massively scaling chips mean—again, it seems like this just depends crucially on how good your algorithms are. It feels more like you should be estimating multiple numbers and then seeing the probability that the product is large enough to be impactful.
Agree 100%. Our essay does exactly this, forecasting over a wide range of potential compute needs, before taking an expected value to arrive a single summary likelihood.
Sounds like you think we should have ascribed more probability to lower ranges, which is a totally fair disagreement.

Ted Sanders Jun 8, 2023, 10:26 PM
2 points
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
I don’t know what level of cheap, quality robots you refer to. The quality of robotics needed to achieve transformative AI depends completely on the quality of your AI. For powerful AI it can be done with existing robot bodies, for weak AI it would need wildly superhuman bodies, at intermediate levels it can be done if humanoid robots cost millions of dollars each.
Interesting—this is perhaps another good crux between us.
My impression is that existing robot bodies are not good enough to do most human jobs, even if we had human-level AGI today. Human bodies self-repair, need infrequent maintenance, last decades, have multi-modal high bandwidth sensors built in, and are incredibly energy efficient.
One piece of evidence for this is how rare tele-operated robots are. There are plenty of generally intelligent humans around the world who would be happy to control robots for $1/hr, and yet they are not being employed to do so.

Ted Sanders Jun 8, 2023, 10:16 PM
10 points
3 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
Are you saying that e.g. a war between China and Taiwan makes it impossible to build AGI? Or that serial time requirements make AGI impossible? Or that scaling chips means AGI is impossible?
C’mon Paul—please extend some principle of charity here. :)

You have repeatedly ascribed silly, impossible beliefs to us and I don’t know why (to be fair, in this particular case you’re just asking, not ascribing). Genuinely, man, I feel bad that our writing has either (a) given the impression that we believe such things or (b) given the impression that we’re the type of people who’d believe such things.

Like, are these sincere questions? Is your mental model of us that there’s a genuine uncertainty over whether we’ll say “Yes, a war precludes AGI” vs “No, a war does preclude AGI.”
To make it clear: No, of course a war between China and Taiwan does not make it impossible to build AGI by 2043. As our essay explicitly says.
Some things can go wrong and you can still get AGI by 2043. If you want to argue you can’t build AGI if something goes wrong, that’s a whole different story. So multiplying probabilities (even conditional probabilities) for none of these things happening doesn’t seem right.
To make it clear: our forecasts are not the odds of wars, pandemics, and depressions not occurring. They are the odds of wars, pandemics, and depressions not delaying AGI beyond 2043. Most wars, most pandemics, and most depressions will not delay AGI beyond 2043, we think. Our methodology is to forecast only the most severe events, and then assume a good fraction won’t delay AGI. As our essay explicitly says.
We probably forecast higher odds of delay than you, because our low likelihoods of TAGI mean that TAGI, if developed, is likeliest to be developed nearer to the end of the period, without many years of slack. If TAGI is easy, and can be developed early or with plenty of slack, then it becomes much harder for these types of events to derail TAGI.

Ted Sanders Jun 8, 2023, 10:03 PM
3 points
2 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
A floating point operation is more like 1e5 bit erasures today and is necessarily at least 16 bit erasures at fp16 (and your estimates don’t allow for large precision reductions e.g. to 1 bit arithmetic). Let’s call it 1.6e21 bit erasures per second, I think quite conservatively?
I don’t follow you here.
Why is a floating point operation 1e5 bit erasures today?
Why does a fp16 operation necessitate 16 bit erasures? As an example, if we have two 16-bit registers (A, B) and we do a multiplication to get (A, A*B), where is the 16 bits of information loss?

(In any case, no real need to reply to this. As someone who has spent a lot of time thinking about the Landauer limit, my main takeaway is that it’s more irrelevant than often supposed, and I suspect getting to the bottom of this rabbit hole is not going to yield much for us in terms of TAGI timelines.)

Ted Sanders Jun 8, 2023, 9:57 PM
1 point
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
It looks to me like you are trying to estimate the difficulty of automating tasks by comparing to the size of brains of animals that perform the task (and in particular human brains). And you are saying that you expect it to take about 1e7 flops for each synapse in a human brain, and then define a probability distribution around there. Am I misunderstanding what’s going on here or is that a fair summary?
Pretty fair summary. 1e6, though, not 1e7. And honestly I could be pretty easily persuaded to go a bit lower by arguments such as:
- Max firing rate of 100 Hz is not the informational content of the channel (that buys maybe 1 OOM)
- Maybe a smaller DNN could be found, but wasn’t
- It might take a lot of computational neurons to simulate the I/O of a single synapse, but it also probably takes a lot of synapses to simulate the I/O of a single computational neuron
Dropping our estimate by 1-2 OOMs would increase step 3 by 10%abs-20%abs. It wouldn’t have much effect on later estimates, as they are already conditional on success in step 3.

Ted Sanders Jun 8, 2023, 9:45 PM
1 point
0 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
In addition to not fully understanding the view, I don’t fully understand the discussion in this section or why it’s justifying this probability. It seems like if you had human-level learning (as we are conditioning on from sections 1+3) then things would probably work in <2 years unless parallelization is surprisingly inefficient.
Maybe, but maybe not, which is why we forecast a number below 100%.
For example, it is very very rare to ever see a CEO hired with <2 years of experience, even if they are very intelligent and have read a lot of books and have watched a lot of interviews. Some reasons might be irrational or irrelevant, but surely some of it is real. A CEO job requires a large constellation of skills practiced and refined over many years. E.g., relationship building with customers, suppliers, shareholders, and employees.
For an AGI to be installed as CEO of a corporation in under two years, human-level learning would not be enough—it would need to be superhuman in its ability to learn. Such superhuman learning could come from simulation (e.g., modeling and simulating how a potential human partner would react to various communication styles), come from parallelization (e.g., being installed as a manager in 1,000 companies and then compiling and sharing learnings across copies), or from something else.
I agree that skills learned from reading or thinking or simulating could happen very fast. Skills requiring real-world feedback that is expensive, rare, or long-delayed would progress more slowly.

Ted Sanders Jun 8, 2023, 9:33 PM
1 point
0 ∶ 2
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
I would guess that more or less anything done by current ML can be done by ML from 2013 but with much more compute and fiddling. So it’s not at all clear to me whether existing algorithms are sufficient for AGI given enough compute, just as it wasn’t clear in 2013. I don’t have any idea what makes this clear to you.
What’s an algorithm from 2013 that you think could yield AGI, if given enough compute? What would its inputs, outputs, and training look like? You’re more informed than me here and I would be happy to learn more.

Ted Sanders Jun 8, 2023, 5:19 AM
3 points
1 ∶ 0
in reply to: Erich_Grunewald 🔸’s comment on: Transformative AGI by 2043 is <1% likely
I think in a lot of those previous years we’ll still have AI doing lots of work to speed up R&D and carry out lots of other economically useful tasks. Like, we know in this world that we’re headed for AGI in 2043 or even earlier, so we should be seeing really capable and useful AI systems already in 2030 and 2035 and so on.
Maybe you think the progression from today’s systems to potentially-transformative AGI will be discontinuous or something like that, with lots of progress (on algorithms, hardware, robotics, etc.) happening near the end?
No, I actually fully agree with you. I don’t think progress will be discontinuous, and I do think we will see increasingly capable and useful systems by 2030 and 2035 that accelerate rates of progress.
I think where we may differ is that:
- I think the acceleration will likely be more “in line” than “out of line” with the exponential acceleration we already see from improving computer tools and specifically LLM computer tools (e.g., GitHub Copilot, GPT-4). Already a software engineer today is many multiples more productive (by some metrics) than a software engineer in the 90s.
- I think that tools that, say, cheaply automate half of work, or expensively automate 100% work probably won’t lead to wild, extra orders of magnitude levels of progress. OpenAI has what, 400 employees?
  - Scenario one: If half their work was automated, ok now those 400 people could do the work of 800 people. That’s great, but honestly I don’t think it’s path-breaking. And sure, that’s only the first order effect. If half the work was automated, we’d of course elastically start spending way much more on the cheap automated half. But on the other hand, there would be diminishing returns, and for every step that becomes free, we just hit bottlenecks in the hard to automate parts. Even in the limit of cheap AGI, those AGIs may be limited by the GPUs they have to experiment on. Labor becoming free just means capital is the constraint.
  - Scenario two: Or, suppose we have human-cost human-level AGIs. I’m not convinced that would, to first order, change much either. There are millions of smart people on earth who aren’t working on AI research now. We could hire them, but we don’t. We’re not limited by brains. We’re limited by willingness to spend. So even if we invent human-cost human-level brains, it actually doesn’t change much, because that wasn’t the constraint. (Of course, this is massively oversimplifying, and obviously human-cost human-level AGIs would be a bigger deal than human workers because of their ability to be rapidly improved and copied. But I hope it nevertheless conveys why I think AGI will need to get close to transformative levels before growth really explodes.)
Overall where it feels like I’m differing from some folks here is that I think higher levels of AI capability will be needed before we get wild self-improvement takeoff. I don’t think it will be early, because even if we get massive automation due to uneven AI we’ll still be bottlenecked by the things it’s bad at. I acknowledge this is a pretty squishy argument and I find it difficult to quantify and articulate, so I think it’s quite reasonable to disagree with me here. In general though, I think we’ve seen a long history of things being harder to automate than we thought (e.g., self-driving, radiology, etc.). It will be exciting to see what happens!

Ted Sanders Jun 7, 2023, 8:06 PM
5 points
1 ∶ 1
in reply to: zchuang’s comment on: Transformative AGI by 2043 is <1% likely
Not harsh at all; I genuinely appreciate the discussion. If there are good criticisms of our approach, I hope that we absorb them into an improved model rather than entrenching ourselves against them.

The issue I see with grouping these factors is then how do we figure what forecast to make for the collective group? The intuitive approach I’d take is to look at the rates of pandemics, world wars, etc. So it feels like we’d still be basing the estimate on mostly independent considerations even if we smush the final product together at the end.

Seems like a tricky forecasting problem in general. You don’t want a model with too many finicky specific scenarios, but you also don’t want amorphous uninterpretable blobs that arise from irreversibly blending many ingredients together.

A model with 1,000 parameters isn’t going to convince anyone and neither will a model with just 1. We tried to keep to a manageable range of 10 overall factors, backed by a few dozen subfactors. But definitely room to move in either direction.

Ted Sanders Jun 7, 2023, 7:52 PM
9 points
2 ∶ 0
in reply to: MichaelDickens’s comment on: Transformative AGI by 2043 is <1% likely
Agreed. Factors like AI progress, inference costs, and manufacturing scale needed are massively correlated. We discuss this in the paper. Our unconditional independent forecast of semiconductor production would be much, much lower than our conditional forecast of 46%, for example.

Ted Sanders Jun 7, 2023, 5:43 PM
2 points
0 ∶ 0
in reply to: Erich_Grunewald 🔸’s comment on: Transformative AGI by 2043 is <1% likely
Thanks!
Your framework excludes some factors that could cause the overall probability to increase.
- For example, I can think of ways that a great power conflict (over Taiwan, say) actually increases the chances of TAI. But your framework doesn’t easily account for this.
Totally reasonable to disagree with us on some of these forecasts—they’re rough educated guesses, after all. We welcome others to contribute their own forecasts. I’m curious: What do you think are the rough odds that invasion of Taiwan increases the likelihood of TAGI by 2043?
But this doesn’t seem like fully conditioning on a world with TAI that is cheap, that can automate ~100% of human tasks, and that can be deployed at scale, and that is relatively unregulated.
it seems like you don’t fully condition on a world where we have TAI that is cheap, that can automate ~100% of human tasks, and that can operate cheap, high-quality robots, and that can probably be deployed to some fairly wide extent even if not (yet) to actually automate ~all human labour.
Agree wholeheartedly. In a world with scaled, cheap TAGI, things are going to look wildly different and it will be hard to predict what happens. Change could be a lot faster than what we’re used to, and historical precedent and intuition might be relatively poor guides relative to first principles thinking.
However, we feel somewhat more comfortable with our predictions prior to scaled, cheap AGI. Like, if it takes 3e30 − 3e35 operations to train an early AGI, then I don’t think we can condition on that AGI accelerating us towards construction of the resources needed to generate 3e30 − 3e35 operations. It would be putting the cart before the horse.
What we can (and try to) condition on are potential predecessors to that AGI; e.g., improved narrow AI or expensive human-level AGI. Both of those we have experience with today, which gives us more confidence that we won’t get an insane productivity explosion in the physical construction of fabs and power plants.
We could be wrong, of course, and we’ll find out in 2043.

Ted Sanders Jun 7, 2023, 3:56 PM
2 points
0 ∶ 1
in reply to: Prometheus’s comment on: Transformative AGI by 2043 is <1% likely
Yep, that’s admittedly a risk of a framework like this. We’ve tried our best to not to make that mistake, and have gone to some length explaining why we think we haven’t. If you disagree, please help us by telling us which disjunctive paths you think we’ve missed or which probabilities you think we’ve underestimated.
As we asked in the post:
If you disagree with our admittedly imperfect guesses, we kindly ask that you supply your own preferred probabilities (or framework modifications). It’s easier to tear down than build up, and we’d love to hear how you think this analysis can be improved.

Ted Sanders Jun 7, 2023, 3:53 PM
4 points
0 ∶ 1
in reply to: Prometheus’s comment on: Transformative AGI by 2043 is <1% likely
What is a margin of error, here, exactly?
The event will either happen (1) or not (0). The 0.4% already reflects our uncertainty. In general, I don’t think it makes mathematical sense to discuss probabilities of probabilities.*

*although of course it can make sense to describe sensitivities of probabilities to new information coming in

Ted Sanders Jun 7, 2023, 3:50 PM
3 points
0 ∶ 2
in reply to: Matthew_Barnett’s comment on: Transformative AGI by 2043 is <1% likely
Thanks for the kind words.
Regarding the multiple stage fallacy, we recognize it’s a risk of a framework like this and go to some lengths explaining why we think our analysis does not suffer from it. (Namely, in the executive summary, the discussion, and the appendix “Why 0.4% might be less confident than it seems.”)
What are the disjunctive alternatives you think our framework misses?
If you disagree with our admittedly imperfect guesses, we kindly ask that you supply your own preferred probabilities (or framework modifications). It’s easier to tear down than build up, and we’d love to hear how you think this analysis can be improved.

Ted Sanders Jun 7, 2023, 6:52 AM
4 points
0 ∶ 2
in reply to: zchuang’s comment on: Transformative AGI by 2043 is <1% likely
To me, the odds of pandemic and wars and regulation feel decently independent, but perhaps I haven’t thought deeply enough about pandemics causing depressions causing wars, or wars leading to engineered pandemics being released, etc.
Looking at the past 100ish years of history, the worst wars (world war I & II), the worst pandemics (various flus, COVID19), and the worst recessions (great depression, great recession) all seem fairly independent.
In any case, we tried our best to come up with probabilities of each, conditional on the others not occurring.
What probabilities would you assign?
As we asked in our post:
If you disagree with our admittedly imperfect guesses, we kindly ask that you supply your own preferred probabilities (or framework modifications). It’s easier to tear down than build up, and we’d love to hear how you think this analysis can be improved.

Ted Sanders Jun 7, 2023, 6:39 AM
7 points
1 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
I think you make a good argument and I’m open to changing my mind. I’m certainly no expert on visual processing in the human brain. Let me flesh out some of my thoughts here.
On whether this framework would have yielded bad forecasts for self-driving:
When we guess that brains use 1e20-1e21 FLOPS, and therefore that early AGIs might need 1e16-1e25, we’re not making a claim about AGIs in general, or the most efficient AGI possible, but AGIs by 2043. We expect early AGIs to be horribly inefficient by later standards, and AGIs to get rapidly more efficient over time. AGI in 2035 will be less efficient than AGI in 2042 which will be less efficient than AGI in 2080.
With that clarification, let’s try to apply our logic to self-driving to see whether it bears weight.
Supposing that self-driving needs 1% of human brainpower, or 1e18-1e19 FLOPS, and then similarly widen our uncertainty to 1e14-1e23 FLOPS, it might say yes, we’d be surprised but not stunned at 1e14 FLOPS being enough to drive (10% → 100%). But, and I know my reasoning is motivated here, that actually seems kind of reasonable? Like, for the first decade and change of trying, 1e14 FLOPS actually was not enough to drive. Even now, it’s beginning to be enough to drive, but still is wildly less sample efficient than human drivers and wildly worse at generalizing than human drivers. So it feels like if in 2010 we predicted self-driving would take 1e14-1e23 FLOPS, and then a time traveler from the future told us that actually it was 1e14 FLOPS, but it would take 13 years to get there, and actually would still be subhuman, then honestly that doesn’t feel too shocking. It was the low end of the range, took many years, and still didn’t quite match human performance.
No doubt with more time and more training 1e14 FLOPS will become more and more capable. Just as we have little doubt that with more time AGIs will require fewer and fewer FLOPS to achieve human performance.
So as I reflect on this framework applied to the test case of self-driving, I come away thinking (a) it actually would have made reasonable predictions and (b) in hindsight I think we should have spent some pages modeling rates of AGI progress, as (obviously) AGI needs are not a fixed target but will decline rapidly over time.
In sum, it’s not obvious to me that this logic would have generated bad predictions for self-driving, and so I’m still unconvinced that we’ve made a big blunder here.
On whether driving takes 1% of the human brain:
I’m going to go way outside my lane here, no pun intended, and I welcome comments from those more informed.
From my very brief Googling, sources say that something like half the brain is involved in vision, though that includes vision+motor, vision+attention, vision+spatial, etc. Going out on a limb, it seems if one knew that fact alone, they might feel reasonable saying that if we can solve vision, then AGI will only need twice as much compute, as vision is half of our brainpower. But it feels like we’ve made massive progress in vision (AlexNet, self-driving, etc.) without being on the cusp of AGI. Somehow even though the brain does lots of vision, these visual identification benchmarks feel far short of AGI.
My feeling is that what makes the human brain superior to today’s AI is its ability to generalize from few examples. To illustrate, suppose you take an AI and a human and you train them over and over to drive a route from A to B. With 1e14 FLOPS and enough training, the AI may be able to eventually outperform the human. But now test the AI and human on route C to D. The human will be wildly better, as their pre-trained world model will have prevented overfitting to the features of route AB. The je ne sais quoi of our intelligence seems to be that we are better able to build models of the world that allow us to more easily reason and generalize to new situations.
To me, getting 1e14 FLOPS to drive a route AB and nothing else is a monumentally different challenge than getting 1e14 FLOPS to drive a route AB, plus any other hypothetical route you throw at it. The first is narrow intelligence. The second is general intelligence. So if we discover that an AI can self-drive route AB with only 1e14 FLOPS, should it be a giant update for how much compute AGI will need? I think it depends: if it’s a narrow brittle overfit AI, then no. If it’s a general (in the sense of general to any road) robust AI, then yes, it should be a big update.
So where does self-driving lay? Well, obviously self-driving cars with 1e14 FLOPS are able to generalize well to all sorts of pre-mapped and pre-simulated routes in a city. But at the same time, they appear to generalize pretty poorly overall—I wouldn’t expect a Waymo vehicle to generalize well to a new city, let alone a new country.
Summarizing, I think the impressiveness of only needing 1e14 FLOPS to drive is really dependent on how well that driving generalizes. If it can generalize as well as human drivers, yes, that’s a big update that AGI may need less 1e20-1e21 FLOPS. But today’s self-driving doesn’t cross that bar for me. It’s quite brittle, and so I am less inclined to update.
Put another way, perhaps brittle self-driving takes 0.1% of human brainpower and robust self-driving takes 10% of human brainpower. Or something like that, who knows. It’s really not clear to me that self-driving in general needs 1% of the human brain, if that self-driving generalizes very poorly to new situations.
Lastly, there is typically going to be a tradeoff between training compute and inference compute. If 1e14 FLOPS is enough to self-drive, but only with millions of miles driven and billions of miles simulated, that should far be less impressive than 1e14 FLOPS being enough to drive after only 100 miles of training. For all I know, it may be possible to get a good driver that only uses 1e12 FLOPS if we train it on even more compute and data; e.g., a billion miles of driving and a trillion miles of simulation. But even if that loosely extrapolates to AGI, it’s only useful if we can afford the “brain equivalent” of a billion miles of training. If that’s too costly, then the existence proof of a 1e12 FLOPS driver may be moot in terms of analogizing AGI inference costs. It’s a pretty interesting question and we certainly could have added more modeling and discussion to our already lengthy essay.

Ted Sanders Jun 7, 2023, 12:40 AM
3 points
1 ∶ 1
in reply to: Paul_Christiano’s comment on: Transformative AGI by 2043 is <1% likely
And also you claim that existing levels of compute would be a way too low to learn to drive even if we had human-level algorithms.
This is not a claim we’ve made.

Ted Sanders Jun 6, 2023, 10:25 PM
6 points
3 ∶ 1
in reply to: Ben_West🔸’s comment on: Transformative AGI by 2043 is <1% likely
Thanks! We agree that a common mistake by forecasters is to equate low probability of derailment with negligible probability of derailment. The future is hard to predict, and we think it’s worth taking tail risks seriously.
it seems weird to assume that TAI is infeasible without [TSMC]?
We do not assume TAI is infeasible without TSMC. That would be a terrible reasoning error, and I apologize for giving you that impression.
What we assume is that losing TSMC would likely delay TAI by a handful of years, as it would take:
- Time for NVIDIA to bid on capacity from Samsung
- Time for Samsung to figure out to what extent it could get out of prior contracted commitments
- Time for NVIDIA and Samsung engineers to retune GPU designs for Samsung’s fab design rules
- Time to manufacture the masks and put the GPUs into production
- Time to iron out early manufacturing and yield issues
- Time to build new fabs to absorb the tsunami of demand from TSMC customers (like Apple) and scale up to NVIDIA’s original TSMC volumes
And on top of this, there would massive geopolitical uncertainty that would slow things like investments into new fabs, as companies wonder whether the conflict will escalate or evaporate (both of which massively change the investment case).

What this might look like in reality will also depend on how close we are to transformative AGI.
Two example scenarios:

Today, for example, NVIDIA is probably not going to outbid Apple (Apple makes ~$10B in PROFIT per month, which would evaporate if they were starved of chips).
Or, imagine it’s 2035 and NVIDIA is worth $10T and the semiconductor industry has been building fabs left and right to fuel the impending AGI boom. In such a world, where NVIDIA is the world’s biggest chip designer, it may already dominate manufacturing on both Samsung and TSMC, meaning that if TSMC goes down, it cannot shift production to Samsung—because it already has production on Samsung.
In any case, we fully agree TAI is feasible without TSMC. But we think losing TSMC delays things by a few years, and if TAI is likely to come in the final 10 or 5 years of this period, then a few years might have a ⁵⁰⁄₅₀ shot of delaying things beyond 2043.

Ted Sanders

On whether this framework would have yielded bad forecasts for self-driving:

On whether driving takes 1% of the human brain: