Hi—great post! I was pointed to this because I’ve been working on a variety of hardware-related projects at FHI and AI Impacts, including generating better hardware forecasts. (I wrote a lot here, but would also be excited to talk to you directly and have even more to say—I contacted you through Facebook.)
At first glance, it seemed to me that the existence of Ajeya’s report demonstrates that the EA community already has enough people with sufficient knowledge and access to expert opinion that, on the margin, adding one expert in hardware to the EA community wouldn’t improve these forecasts much.
I think this isn’t true.
For one, I think while the forecasts in that report are the best publicly available thing we have, there’s significant room to do better, e.g.
The forecasts rely on data for the sale price of hardware along with their reported FLOPS performance. But the sale price is only one component of the costs to run hardware and doesn’t include power, data center costs, storage and networking etc. Arguably, we also care about the price performance for large hardware producers (e.g. Google) more than hardware consumers, and the sale price won’t necessarily be reflective of that since it includes a significant mark-up over the cost of manufacture.
The forecasts don’t consider existing forecasts from e.g. the IRDS that you mention, which are actually very pessimistic about the scaling of energy costs for CMOS chips over the next 15 years. (Of course, this doesn’t preclude better scaling through switching to other technology).
If I recall correctly, the report partially justifies its estimate by guessing that even if chip design improvements bottom out, improvements in manufacturing cost and chip lifetime might still create a relatively steady rate of progress. I think this requires some assumptions about the cost model that may not be true, though I haven’t done enough investigation yet to be sure.
(This isn’t to disparage the report—I think it’s an awesome report and the current estimate is a great starting point, and Ajeya very explicitly disclaims that these are the forecasts most likely to be knowably mistaken.)
As a side note, I think EAs tend to misuse and misunderstand Moore’s Law in general. As you say, Moore’s Law says that the number of transistors on a chip doubles every two years. This has remained true historically, but is only dubiously correlated with ‘price performance Moore’s Law’—a doubling of price performance every two years. As I note above, I think the data publicly collected on price performance is poor, partially because the ‘price’ and ‘performance’ of hardware is trickier to define than it looks. But e.g. this recent paper estimates that the price performance of at least universal processors has slowed considerably in recent years (the paper estimates 8% improvement in performance-per-dollar annually from 2008 − 2013, see section 4.3.2 ‘Current state of performance-per-dollar of universal processors’). Even if price performance Moore’s Law ever held true, it’s really not clear that it holds now.
For two, I think it’s not the case that we have access to enough people with sufficient knowledge and expert opinion. I’ve been really interested in talking to hardware experts, and I think I would selfishly benefit substantially from experts who had thought more about “the big picture” or more speculative hardware possibilities (most people I talk to have domain expertise in something very specific and near-term). I’ve also found it difficult to get a lot of people’s time, and would selfishly benefit from having access more hardware experts that were explicitly longtermist-aligned and excited to give me more of it. :) Basically, I’d be very in favor of having more people in industry available as advisors, as you suggest.
You also touch on this some, but I will say that I do think now is actually a particularly impactful time to influence policy on the company-level (in addition to in government, which seems to be implementing a slew of new semiconductor legislation and seems increasingly interested in regulating hardware companies.) A recent report estimates that ASICs are poised to take over 50% of the hardware market in the coming years, and most ASIC companies now are small start-ups—I think there’s a case that influencing the policy and ethics of these small companies is much more tractable than their larger counterparts, and it would be worth someone thinking carefully about how to do that. Working as an early employee seems like a good potential way.
Lastly, I will say that I think there might be valuable work to be done at the intersection of hardware and economics—for an example, see again this paper. I think things like understanding models of hardware costs, the overall hardware market, cloud computing, etc. are not well-encapsulated by the kind of understanding technical experts tend to have and is valuable for the longtermist community to have access to. (This is also some of what I’ve been working on locally.)
Thank you so much for the detailed comment! I would be very excited to chat offline, but I’ll put a few questions here that are directly related to the comment:
For one, I think while the forecasts in that report are the best publicly available thing we have, there’s significant room to do better
These are all super interesting points! One thing that strikes me as a similarity between many of them is that it’s not straightforward what metric (# transistors per chip, price performance, etc.) is the most useful to forecast AI timelines. Do you think price performance for certain applications could be one of the better ones to use on its own? Or is it perhaps better practice to keep an index of some number of trends?
I think it’s not the case that we have access to enough people with sufficient knowledge and expert opinion. I’ve been really interested in talking to hardware experts, and I think I would selfishly benefit substantially from experts who had thought more about “the big picture” or more speculative hardware possibilities
Are there any specific speculative hardware areas you think may be neglected? I mentioned photonics and quantum computing in the post because these are the only ones I’ve spent more than an hour thinking about. I vaguely plan to read up on the other technologies in the IRDS, but if there are some that might be worth looking more into than others (or some that didn’t make their list at all!) that would help focus this plan significantly.
A recent report estimates that ASICs are poised to take over 50% of the hardware market in the coming years
Thank you for pointing this out! I talked with someone working in hardware that gave me the opposite impression and I haven’t thought to actually look into this myself. (In retrospect this may have been their sales pitch to differentiate themselves from their competitors. FWIW, I think their argument was that AI moves fast enough that an ASIC start-up will be irrelevant before their product hits the market.) I look forward to updating this impression!
I think things like understanding models of hardware costs, the overall hardware market, cloud computing, etc. are not well-encapsulated by the kind of understanding technical experts tend to have.
I would naively think this would be another point in favor of working at start-ups compared to more established companies. My impression is that start-ups have to spend more time thinking carefully about their market is in order to attract funding (and the small size means technical people are more involved with this thinking). Does that seem reasonable?
Do you think price performance for certain applications could be one of the better ones to use on its own? Or is it perhaps better practice to keep an index of some number of trends?
I think price performance, measured in something like “operations / $”, is by far the most important metric, caveating that by itself it doesn’t differentiate between one-time costs of design and purchase and ongoing costs to run hardware, and it doesn’t account for limitations in memory, networking, and software for parallelization that constrain performance as the number of chips are scaled up.
Are there any specific speculative hardware areas you think may be neglected? I mentioned photonics and quantum computing in the post because these are the only ones I’ve spent more than an hour thinking about. I vaguely plan to read up on the other technologies in the IRDS, but if there are some that might be worth looking more into than others (or some that didn’t make their list at all!) that would help focus this plan significantly.
There has been a lot of recent work in optical chips / photonics, so I’ve been following them closely—I have my own notes on publicly available info here. I think quantum computing is likely further from viability but good to pay attention to. I also think it’s worth understanding the likelihood and implications of 3D CMOS chips, just because at least IRDS predictions suggest that might be the way forward in the next decade (I think these are much less speculative than the two above). I haven’t looked into this as much as I’d like, though—I actually also have on my todo list to read through the IRDS list and identify the things that are most likely and have the highest upside. Maybe we can compare notes. :)
I would naively think this would be another point in favor of working at start-ups compared to more established companies. My impression is that start-ups have to spend more time thinking carefully about their market is in order to attract funding (and the small size means technical people are more involved with this thinking). Does that seem reasonable?
I suspect in most roles in either a start-up or a large company you’ll be quite focused on the tech and not very focused on the market or the cost model—I don’t think this strongly favors working for a start-up.
Hi—great post! I was pointed to this because I’ve been working on a variety of hardware-related projects at FHI and AI Impacts, including generating better hardware forecasts. (I wrote a lot here, but would also be excited to talk to you directly and have even more to say—I contacted you through Facebook.)
I think this isn’t true.
For one, I think while the forecasts in that report are the best publicly available thing we have, there’s significant room to do better, e.g.
The forecasts rely on data for the sale price of hardware along with their reported FLOPS performance. But the sale price is only one component of the costs to run hardware and doesn’t include power, data center costs, storage and networking etc. Arguably, we also care about the price performance for large hardware producers (e.g. Google) more than hardware consumers, and the sale price won’t necessarily be reflective of that since it includes a significant mark-up over the cost of manufacture.
The forecasts don’t consider existing forecasts from e.g. the IRDS that you mention, which are actually very pessimistic about the scaling of energy costs for CMOS chips over the next 15 years. (Of course, this doesn’t preclude better scaling through switching to other technology).
If I recall correctly, the report partially justifies its estimate by guessing that even if chip design improvements bottom out, improvements in manufacturing cost and chip lifetime might still create a relatively steady rate of progress. I think this requires some assumptions about the cost model that may not be true, though I haven’t done enough investigation yet to be sure.
(This isn’t to disparage the report—I think it’s an awesome report and the current estimate is a great starting point, and Ajeya very explicitly disclaims that these are the forecasts most likely to be knowably mistaken.)
As a side note, I think EAs tend to misuse and misunderstand Moore’s Law in general. As you say, Moore’s Law says that the number of transistors on a chip doubles every two years. This has remained true historically, but is only dubiously correlated with ‘price performance Moore’s Law’—a doubling of price performance every two years. As I note above, I think the data publicly collected on price performance is poor, partially because the ‘price’ and ‘performance’ of hardware is trickier to define than it looks. But e.g. this recent paper estimates that the price performance of at least universal processors has slowed considerably in recent years (the paper estimates 8% improvement in performance-per-dollar annually from 2008 − 2013, see section 4.3.2 ‘Current state of performance-per-dollar of universal processors’). Even if price performance Moore’s Law ever held true, it’s really not clear that it holds now.
For two, I think it’s not the case that we have access to enough people with sufficient knowledge and expert opinion. I’ve been really interested in talking to hardware experts, and I think I would selfishly benefit substantially from experts who had thought more about “the big picture” or more speculative hardware possibilities (most people I talk to have domain expertise in something very specific and near-term). I’ve also found it difficult to get a lot of people’s time, and would selfishly benefit from having access more hardware experts that were explicitly longtermist-aligned and excited to give me more of it. :) Basically, I’d be very in favor of having more people in industry available as advisors, as you suggest.
You also touch on this some, but I will say that I do think now is actually a particularly impactful time to influence policy on the company-level (in addition to in government, which seems to be implementing a slew of new semiconductor legislation and seems increasingly interested in regulating hardware companies.) A recent report estimates that ASICs are poised to take over 50% of the hardware market in the coming years, and most ASIC companies now are small start-ups—I think there’s a case that influencing the policy and ethics of these small companies is much more tractable than their larger counterparts, and it would be worth someone thinking carefully about how to do that. Working as an early employee seems like a good potential way.
Lastly, I will say that I think there might be valuable work to be done at the intersection of hardware and economics—for an example, see again this paper. I think things like understanding models of hardware costs, the overall hardware market, cloud computing, etc. are not well-encapsulated by the kind of understanding technical experts tend to have and is valuable for the longtermist community to have access to. (This is also some of what I’ve been working on locally.)
Thank you so much for the detailed comment! I would be very excited to chat offline, but I’ll put a few questions here that are directly related to the comment:
These are all super interesting points! One thing that strikes me as a similarity between many of them is that it’s not straightforward what metric (# transistors per chip, price performance, etc.) is the most useful to forecast AI timelines. Do you think price performance for certain applications could be one of the better ones to use on its own? Or is it perhaps better practice to keep an index of some number of trends?
Are there any specific speculative hardware areas you think may be neglected? I mentioned photonics and quantum computing in the post because these are the only ones I’ve spent more than an hour thinking about. I vaguely plan to read up on the other technologies in the IRDS, but if there are some that might be worth looking more into than others (or some that didn’t make their list at all!) that would help focus this plan significantly.
Thank you for pointing this out! I talked with someone working in hardware that gave me the opposite impression and I haven’t thought to actually look into this myself. (In retrospect this may have been their sales pitch to differentiate themselves from their competitors. FWIW, I think their argument was that AI moves fast enough that an ASIC start-up will be irrelevant before their product hits the market.) I look forward to updating this impression!
I would naively think this would be another point in favor of working at start-ups compared to more established companies. My impression is that start-ups have to spend more time thinking carefully about their market is in order to attract funding (and the small size means technical people are more involved with this thinking). Does that seem reasonable?
I think price performance, measured in something like “operations / $”, is by far the most important metric, caveating that by itself it doesn’t differentiate between one-time costs of design and purchase and ongoing costs to run hardware, and it doesn’t account for limitations in memory, networking, and software for parallelization that constrain performance as the number of chips are scaled up.
There has been a lot of recent work in optical chips / photonics, so I’ve been following them closely—I have my own notes on publicly available info here. I think quantum computing is likely further from viability but good to pay attention to. I also think it’s worth understanding the likelihood and implications of 3D CMOS chips, just because at least IRDS predictions suggest that might be the way forward in the next decade (I think these are much less speculative than the two above). I haven’t looked into this as much as I’d like, though—I actually also have on my todo list to read through the IRDS list and identify the things that are most likely and have the highest upside. Maybe we can compare notes. :)
I suspect in most roles in either a start-up or a large company you’ll be quite focused on the tech and not very focused on the market or the cost model—I don’t think this strongly favors working for a start-up.