Separately, various people seem to think that the appropriate way to make forecasts is to (1) use some outside-view methods, (2) use some inside-view methods, but only if you feel like you are an expert in the subject, and then (3) do a weighted sum of them all using your intuition to pick the weights. This is not Tetlock’s advice, nor is it the lesson from the forecasting tournaments, especially if we use the nebulous modern definition of “outside view” instead of the original definition. (For my understanding of his advice and those lessons, see this post, part 5. For an entire book written by Yudkowsky on why the aforementioned forecasting method is bogus, see Inadequate Equilibria, especially this chapter. Also, I wish to emphasize that I myself was one of these people, at least sometimes, up until recently when I noticed what I was doing!)
This is a bit tangential to the main point of your post, but I thought I’d give some thoughts on this, partly because I basically did exactly this procedure a few months ago in an attempt to come to a personal all-things-considered view about AI timelines (although I did “use some inside-view methods” even though I don’t at all feel like I’m an expert in the subject!).
I liked your AI Impacts post, thanks for linking to it! Maybe a good summary of the recommended procedure is the part at the very end. I do feel like it was useful for me to read it.
Tetlock describes how superforecasters go about making their predictions.56 Here is an attempt at a summary:
Sometimes a question can be answered more rigorously if it is first “Fermi-ized,” i.e. broken down into sub-questions for which more rigorous methods can be applied.
Next, use the outside view on the sub-questions (and/or the main question, if possible). You may then adjust your estimates using other considerations (‘the inside view’), but do this cautiously.
Seek out other perspectives, both on the sub-questions and on how to Fermi-ize the main question. You can also generate other perspectives yourself.
Repeat steps 1 – 3 until you hit diminishing returns.
Your final prediction should be based on an aggregation of various models, reference classes, other experts, etc.
I’m less sure about the direct relevance of Inadequate Equilibria for this, apart from it making the more general point that ~”people should be less scared of relying on their own intuition / arguments / inside view”. Maybe I haven’t scrutinised it closely enough.
To be clear, I don’t think “weighted sum of ‘inside views’ and ‘outside views’” is the gold standard or something. I just think it’s an okay approach sometimes (maybe especially when you want to do something “quick and dirty”).
If you strongly disagree (which I think you do), I’d love for you to change my mind! :)
Re : Inadequate Equilibria: I mean, that was my opinionated interpretation I guess. :) But Yudkowsky was definitely arguing something was bogus. (This is a jab at his polemical style) To say a bit more: Yudkowsky argues that the justifications for heavy reliance on various things called “outside view” don’t hold up to scrutiny, and that what’s really going on is that people are overly focused on matters of who has how much status and which topics are in whose areas of expertise and whether I am being appropriately humble and stuff like that, and that (unconsciously) this is what’s really driving people’s use of “outside view” methods rather than the stated justifications. I am not sure whether I agree with him or not but I do find it somewhat plausible at least. I do think the stated justifications often (usually?) don’t hold up to scrutiny.
To be clear, I don’t think “weighted sum of ‘inside views’ and ‘outside views’” is the gold standard or something. I just think it’s an okay approach sometimes (maybe especially when you want to do something “quick and dirty”).
If you strongly disagree (which I think you do), I’d love for you to change my mind! :)
I mean, depending on what you mean by “an okay approach sometimes… especially when you want to do something quick and dirty” I may agree with you! What I said was:
This is not Tetlock’s advice, nor is it the lesson from the forecasting tournaments, especially if we use the nebulous modern definition of “outside view” instead of the original definition.
And it seems you agree with me on that. What I would say is: Consider the following list of methods: 1. Intuition-weighted sum of “inside view” and “outside view” methods (where those terms refer to the Big Lists summarized in this post) 2. Intuition-weighted sum of “Type X” and “Type Y” methods (where those terms refer to any other partition of the things in the Big Lists summarized in this post) 3. Intuition 4. The method Tetlock recommends (as interpreted by me in the passage of my blog post you quoted)
My opinion is that 1 and 2 are probably typically better than 3 and that 4 is probably typically better than 1 and 2 and that 1 and 2 are probably about the same. I am not confident in this of course, but the reasoning is: Method 4 has some empirical evidence supporting it, plus plausible arguments/models.* So it’s the best. Methods 1 & 2 are like method 3 except that they force you to think more and learn more about the case (incl. relevant arguments about it) before calling on your intuition, which hopefully results in a better-calibrated intuitive judgment. As for comparing 1 & 2, I think we have basically zero evidence that partitioning into “Outside view” and “Inside view” is more effective than any other random partition of the things on the list. It’s still better than pure intuition though, probably, for reasons mentioned.
The view I was arguing against in the OP was the view that method 1 is the best, supported by the evidence from Tetlock, etc. I think I slipped into holding this view myself over the past year or so, despite having done all this research on Tetlock et al earlier! It’s easy to slip into because a lot of people in our community seem to be holding it, and when you squint it’s sorta similar to what Tetlock said. (e.g. it involves aggregating different things, it involves using something called inside view and something called outside view.)
*The margins of this comment are too small to contain, I was going to write a post on this some day…
I mean, depending on what you mean by “an okay approach sometimes… especially when you want to do something quick and dirty” I may agree with you! What I said was:
This is not Tetlock’s advice, nor is it the lesson from the forecasting tournaments, especially if we use the nebulous modern definition of “outside view” instead of the original definition.
I guess I was reacting to the part just after the bit you quoted
For an entire book written by Yudkowsky on why the aforementioned forecasting method is bogus
Which I took to imply “Daniel thinks that the aforementioned forecasting method is bogus”. Maybe my interpretation was incorrect. Anyway, seems very possible we in fact roughly agree here.
Re your 1, 2, 3, 4: It seems cool to try doing 4, and I can believe it’s better (I don’t have a strong view). Fwiw re 1 vs 2, my initial reaction is that partitioning by outside/inside view lets you decide how much weight you give to each, and maybe we think that for non-experts it’s better to mostly give weight to the outside view, so the partitioning performed a useful service. I guess this is kind of what you were trying to argue against and unfortunately you didn’t convince me to repent :).
Yeah, I probably shouldn’t have said “bogus” there, since while I do think it’s overrated, it’s not the worst method. (Though arguably things can be bogus even if they aren’t the worst?)
Partitioning by any X lets you decide how much weight you give to X vs. not-X. My claim is that the bag of things people refer to as “outside view” isn’t importantly different from the other bag of things, at least not more importantly different than various other categorizations one might make.
I do think that people who are experts should behave differently than people who are non-experts. I just don’t think we should summarize that as “Prefer to use outside-view methods” where outside view = the things on the First Big List. I think instead we could say: --Use deference more —Use reference classes more if you have good ones (but if you are a non-expert and your reference classes are more like analogies, they are probably leading you astray) --Trust your models less —Trust your intuition less —Trust your priors less ...etc.
This is a bit tangential to the main point of your post, but I thought I’d give some thoughts on this, partly because I basically did exactly this procedure a few months ago in an attempt to come to a personal all-things-considered view about AI timelines (although I did “use some inside-view methods” even though I don’t at all feel like I’m an expert in the subject!).
I liked your AI Impacts post, thanks for linking to it! Maybe a good summary of the recommended procedure is the part at the very end. I do feel like it was useful for me to read it.
I’m less sure about the direct relevance of Inadequate Equilibria for this, apart from it making the more general point that ~”people should be less scared of relying on their own intuition / arguments / inside view”. Maybe I haven’t scrutinised it closely enough.
To be clear, I don’t think “weighted sum of ‘inside views’ and ‘outside views’” is the gold standard or something. I just think it’s an okay approach sometimes (maybe especially when you want to do something “quick and dirty”).
If you strongly disagree (which I think you do), I’d love for you to change my mind! :)
Thanks!
Re : Inadequate Equilibria: I mean, that was my opinionated interpretation I guess. :) But Yudkowsky was definitely arguing something was bogus. (This is a jab at his polemical style) To say a bit more: Yudkowsky argues that the justifications for heavy reliance on various things called “outside view” don’t hold up to scrutiny, and that what’s really going on is that people are overly focused on matters of who has how much status and which topics are in whose areas of expertise and whether I am being appropriately humble and stuff like that, and that (unconsciously) this is what’s really driving people’s use of “outside view” methods rather than the stated justifications. I am not sure whether I agree with him or not but I do find it somewhat plausible at least. I do think the stated justifications often (usually?) don’t hold up to scrutiny.
I mean, depending on what you mean by “an okay approach sometimes… especially when you want to do something quick and dirty” I may agree with you! What I said was:
And it seems you agree with me on that. What I would say is: Consider the following list of methods:
1. Intuition-weighted sum of “inside view” and “outside view” methods (where those terms refer to the Big Lists summarized in this post)
2. Intuition-weighted sum of “Type X” and “Type Y” methods (where those terms refer to any other partition of the things in the Big Lists summarized in this post)
3. Intuition
4. The method Tetlock recommends (as interpreted by me in the passage of my blog post you quoted)
My opinion is that 1 and 2 are probably typically better than 3 and that 4 is probably typically better than 1 and 2 and that 1 and 2 are probably about the same. I am not confident in this of course, but the reasoning is: Method 4 has some empirical evidence supporting it, plus plausible arguments/models.* So it’s the best. Methods 1 & 2 are like method 3 except that they force you to think more and learn more about the case (incl. relevant arguments about it) before calling on your intuition, which hopefully results in a better-calibrated intuitive judgment. As for comparing 1 & 2, I think we have basically zero evidence that partitioning into “Outside view” and “Inside view” is more effective than any other random partition of the things on the list. It’s still better than pure intuition though, probably, for reasons mentioned.
The view I was arguing against in the OP was the view that method 1 is the best, supported by the evidence from Tetlock, etc. I think I slipped into holding this view myself over the past year or so, despite having done all this research on Tetlock et al earlier! It’s easy to slip into because a lot of people in our community seem to be holding it, and when you squint it’s sorta similar to what Tetlock said. (e.g. it involves aggregating different things, it involves using something called inside view and something called outside view.)
*The margins of this comment are too small to contain, I was going to write a post on this some day…
Nice, thanks for this!
I guess I was reacting to the part just after the bit you quoted
Which I took to imply “Daniel thinks that the aforementioned forecasting method is bogus”. Maybe my interpretation was incorrect. Anyway, seems very possible we in fact roughly agree here.
Re your 1, 2, 3, 4: It seems cool to try doing 4, and I can believe it’s better (I don’t have a strong view). Fwiw re 1 vs 2, my initial reaction is that partitioning by outside/inside view lets you decide how much weight you give to each, and maybe we think that for non-experts it’s better to mostly give weight to the outside view, so the partitioning performed a useful service. I guess this is kind of what you were trying to argue against and unfortunately you didn’t convince me to repent :).
Yeah, I probably shouldn’t have said “bogus” there, since while I do think it’s overrated, it’s not the worst method. (Though arguably things can be bogus even if they aren’t the worst?)
Partitioning by any X lets you decide how much weight you give to X vs. not-X. My claim is that the bag of things people refer to as “outside view” isn’t importantly different from the other bag of things, at least not more importantly different than various other categorizations one might make.
I do think that people who are experts should behave differently than people who are non-experts. I just don’t think we should summarize that as “Prefer to use outside-view methods” where outside view = the things on the First Big List. I think instead we could say:
--Use deference more
—Use reference classes more if you have good ones (but if you are a non-expert and your reference classes are more like analogies, they are probably leading you astray)
--Trust your models less
—Trust your intuition less
—Trust your priors less
...etc.