How many angels can dance on the head on a pin? An infinite number because angels have no spatial extension? Or maybe if we assume angels have a diameter of ~1 nanometre plus ~1 additional nanometre of diameter for clearance for dancing we can come up with a ballpark figure? Or, wait, are angels closer to human-sized? When bugs die do they turn into angels? What about bacteria? Can bacteria dance? Are angels beings who were formerly mortal, or were they âbornâ angels?[1]
AI is just a big thing in the world thatâs growing fast. Anybody capable of reading graphs can see that.
Well, some of the graphs are just made-up, like those in âSituational Awarenessâ, and some of the graphs are woefully misinterpreted to be about AGI when theyâre clearly not, like the famous METR time horizon graph.[2] I imagine that a non-trivial amount of EA misjudgment around AGI results from a failure to correctly read and interpret graphs.
And, of course, when people like titotal examine the math behind some of these graphs, like those in AI 2027, they are sometimes found to be riddled with major mistakes.
What I said elsewhere about AGI discourse in general is true about graphs in particular: the scientifically defensible claims are generally quite narrow, caveated, and conservative. The claims that are broad, unqualified, and bold are generally not scientifically defensible. People at METR themselves caveat the time horizons graph and note its narrow scope (I cited examples of this elsewhere in the comments on this post). Conversely, graphs that attempt to make a broad, unqualified, bold claim about AGI tend to be complete nonsense.
Out of curiosity, roughly what probability would you assign to there being an AI financial bubble that pops sometime within the next five years or so? If there is an AI bubble and if it popped, how would that affect your beliefs around near-term AGI?
How is correctness physically instantiated in space and time and how does it physically cause physical events in the world, such as speaking, writing, brain activity, and so on? Is this an important question to ask in this context? Do we need to get into this?
You can take an epistemic practice in EA such as âthinking that Leopold Aschenbrennerâs graphs are correctâ and ask about the historical origin of that practice without making a judgement about whether the practice is good or bad, right or wrong. You can ask the question in a form like, âHow did people in EA come to accept graphs like those in âSituational Awarenessâ as evidence?â If you want to frame it positively, you could ask the question as something like, âHow did people in EA learn to accept graphs like these as evidence?â If you want to frame it negatively, you could ask, âHow did people in EA not learn not to accept graphs like these as evidence?â And of course you can frame it neutrally.
The historical explanation is a separate question from the evaluation of correctness/âincorrectness and the two donât conflict with each other. By analogy, you can ask, âHow did Laverne come to believe in evolution?â And you could answer, âBecause itâs the correct view,â which would be right, in a sense, if a bit obtuse, or you could answer, âBecause she learned about evolution in her biology classes in high school and collegeâ, which would also be right, and which would more directly answer the question. So, a historical explanation does not necessarily imply that a view is wrong. Maybe in some contexts it insinuates it, but both kinds of answers can be true.
Do you know a source that formally makes the argument that the METR graph is about AGI? I am trying to pin down the series of logical steps that people are using to get from that graph to AGI. I would like to spell out why I think this inference is wrong, but first it would be helpful to see someone spell out the inference theyâre making.
How many angels can dance on the head on a pin? An infinite number because angels have no spatial extension? Or maybe if we assume angels have a diameter of ~1 nanometre plus ~1 additional nanometre of diameter for clearance for dancing we can come up with a ballpark figure? Or, wait, are angels closer to human-sized? When bugs die do they turn into angels? What about bacteria? Can bacteria dance? Are angels beings who were formerly mortal, or were they âbornâ angels?[1]
Well, some of the graphs are just made-up, like those in âSituational Awarenessâ, and some of the graphs are woefully misinterpreted to be about AGI when theyâre clearly not, like the famous METR time horizon graph.[2] I imagine that a non-trivial amount of EA misjudgment around AGI results from a failure to correctly read and interpret graphs.
And, of course, when people like titotal examine the math behind some of these graphs, like those in AI 2027, they are sometimes found to be riddled with major mistakes.
What I said elsewhere about AGI discourse in general is true about graphs in particular: the scientifically defensible claims are generally quite narrow, caveated, and conservative. The claims that are broad, unqualified, and bold are generally not scientifically defensible. People at METR themselves caveat the time horizons graph and note its narrow scope (I cited examples of this elsewhere in the comments on this post). Conversely, graphs that attempt to make a broad, unqualified, bold claim about AGI tend to be complete nonsense.
Out of curiosity, roughly what probability would you assign to there being an AI financial bubble that pops sometime within the next five years or so? If there is an AI bubble and if it popped, how would that affect your beliefs around near-term AGI?
How is correctness physically instantiated in space and time and how does it physically cause physical events in the world, such as speaking, writing, brain activity, and so on? Is this an important question to ask in this context? Do we need to get into this?
You can take an epistemic practice in EA such as âthinking that Leopold Aschenbrennerâs graphs are correctâ and ask about the historical origin of that practice without making a judgement about whether the practice is good or bad, right or wrong. You can ask the question in a form like, âHow did people in EA come to accept graphs like those in âSituational Awarenessâ as evidence?â If you want to frame it positively, you could ask the question as something like, âHow did people in EA learn to accept graphs like these as evidence?â If you want to frame it negatively, you could ask, âHow did people in EA not learn not to accept graphs like these as evidence?â And of course you can frame it neutrally.
The historical explanation is a separate question from the evaluation of correctness/âincorrectness and the two donât conflict with each other. By analogy, you can ask, âHow did Laverne come to believe in evolution?â And you could answer, âBecause itâs the correct view,â which would be right, in a sense, if a bit obtuse, or you could answer, âBecause she learned about evolution in her biology classes in high school and collegeâ, which would also be right, and which would more directly answer the question. So, a historical explanation does not necessarily imply that a view is wrong. Maybe in some contexts it insinuates it, but both kinds of answers can be true.
But this whole diversion has been unnecessary.
Do you know a source that formally makes the argument that the METR graph is about AGI? I am trying to pin down the series of logical steps that people are using to get from that graph to AGI. I would like to spell out why I think this inference is wrong, but first it would be helpful to see someone spell out the inference theyâre making.