Thanks! I canât tell if this is cruxy, but for what itâs worth your âpessimal inductionâ vignettes donât resonate with me in a way which makes me less motivated by the unawareness concerns.
For example, Bostrom coined the phrase âattention hazardâ in 2011. I remember someone telling me that MIRI was net-negative for this reason at EAG 2015, and I would be surprised if e.g. Habryka hadnât considered this risk before starting Lightcone. So I disagree with citing him/âthis as a good example of unawareness; itâs more that they mis-estimated a known risk factor.
Similarly, I remember talking about SARP at one of my first EAGs. I think I came across it in Brian Tomasikâs 2007 post, maybe even before I had encountered EA. Perhaps Iâve mis-estimated those concerns, but it doesnât seem like unawareness.
My overall experience is kind of the opposite of yours: when I got involved in EA people talked a lot about âCause Xâ and âCrucial Considerationsâ and now theyâve mostly just⌠stopped? Like people tried to find other considerations, and thereâs some new stuff around s-risks and weird decision theories etc., but if you look at what people talk about at EAGs today it feels mostly like more precise versions of what was discussed in 2016, rather than a large and unpredictable jump from the older understanding. Or, more technically: it feels like weâve had updates in evidence-space, but not as many updates in hypothesis-space, and I understand the latter to be motivating imprecision.
Obviously, this could be because EAs suck at cause prio research, or we just havenât been hit yet with the big update, etc., but the âpessimal inductionâ seems less pessimal to me.
Not a comprehensive reply, but: I think many of the examples youâre talking about are arguably cases of coarse awareness. People were coarsely aware of the potential backfire risks earlier on, but (arguably) the reason they didnât give these risks enough weight was that they didnât have a more fine-grained awareness of the specific causal pathways. I think such cases count as evidence for the pessimistic induction.
Thanks! I canât tell if this is cruxy, but for what itâs worth your âpessimal inductionâ vignettes donât resonate with me in a way which makes me less motivated by the unawareness concerns.
For example, Bostrom coined the phrase âattention hazardâ in 2011. I remember someone telling me that MIRI was net-negative for this reason at EAG 2015, and I would be surprised if e.g. Habryka hadnât considered this risk before starting Lightcone. So I disagree with citing him/âthis as a good example of unawareness; itâs more that they mis-estimated a known risk factor.
Similarly, I remember talking about SARP at one of my first EAGs. I think I came across it in Brian Tomasikâs 2007 post, maybe even before I had encountered EA. Perhaps Iâve mis-estimated those concerns, but it doesnât seem like unawareness.
My overall experience is kind of the opposite of yours: when I got involved in EA people talked a lot about âCause Xâ and âCrucial Considerationsâ and now theyâve mostly just⌠stopped? Like people tried to find other considerations, and thereâs some new stuff around s-risks and weird decision theories etc., but if you look at what people talk about at EAGs today it feels mostly like more precise versions of what was discussed in 2016, rather than a large and unpredictable jump from the older understanding. Or, more technically: it feels like weâve had updates in evidence-space, but not as many updates in hypothesis-space, and I understand the latter to be motivating imprecision.
Obviously, this could be because EAs suck at cause prio research, or we just havenât been hit yet with the big update, etc., but the âpessimal inductionâ seems less pessimal to me.
Interesting, thatâs helpful to know.
Not a comprehensive reply, but: I think many of the examples youâre talking about are arguably cases of coarse awareness. People were coarsely aware of the potential backfire risks earlier on, but (arguably) the reason they didnât give these risks enough weight was that they didnât have a more fine-grained awareness of the specific causal pathways. I think such cases count as evidence for the pessimistic induction.