After thinking about this later, I noticed that one of my claims was wrong. I said:
> Though I’m not particularly excited about refuges, they might be a good test case. I think that if you had this 5N view, refuges would be obviously dumb but if you had the view that I defended in my dissertation then refuges would be interesting from a conceptual perspective.
But then I ran some numbers and this no longer seemed true. If you assumed a population of 10B, an N of 5, a cost of your refuge of $1B, that your risk of doom was 1%, and that your refuge could cut out a thousandth of that 1%, you get a cost per life-equivalent saved of $2000 (with much more favorable figures if you assume higher risk and/or higher refuge effectiveness). So a back-of-the-envelope calculation would suggest that, contrary to what I said, refuges would not be obviously dumb if you had the 5N view. (Link to back-of-envelope calc: https://docs.google.com/spreadsheets/d/1RRlj1sZpPJ8hr-KvMQy5R8NayA3a58EhLODXPu4NgRo/edit#gid=1176340950 .)
My best current guess is that building refuges wouldn’t be this effective at reducing existential risk, but that was after I looked into the issue a bit. I was probably wrong to think that Holden’s 5N heuristic would have ruled out refuges ex ante. (Link to other discussion of refuges: /ea/5r/improving_disaster_shelters_to_increase_the/ .)
Thanks for this clarification. After reading the emails I wanted to make exactly this point!
I do think that comparing how good saving a life today is compared to doing something like building bunkers to reduce risk really comes down to an understanding of the world today rather than an understanding of exactly how big the future might be (after you grant that it could be very big). Though choosing 5 as a multiplier looks rather low to me; I’d be happier with something up in 100-1000 range (and I wouldn’t be surprised if my view of the correct figure to use there changes substantially in the future).
After thinking about this later, I noticed that one of my claims was wrong. I said:
> Though I’m not particularly excited about refuges, they might be a good test case. I think that if you had this 5N view, refuges would be obviously dumb but if you had the view that I defended in my dissertation then refuges would be interesting from a conceptual perspective.
But then I ran some numbers and this no longer seemed true. If you assumed a population of 10B, an N of 5, a cost of your refuge of $1B, that your risk of doom was 1%, and that your refuge could cut out a thousandth of that 1%, you get a cost per life-equivalent saved of $2000 (with much more favorable figures if you assume higher risk and/or higher refuge effectiveness). So a back-of-the-envelope calculation would suggest that, contrary to what I said, refuges would not be obviously dumb if you had the 5N view. (Link to back-of-envelope calc: https://docs.google.com/spreadsheets/d/1RRlj1sZpPJ8hr-KvMQy5R8NayA3a58EhLODXPu4NgRo/edit#gid=1176340950 .)
My best current guess is that building refuges wouldn’t be this effective at reducing existential risk, but that was after I looked into the issue a bit. I was probably wrong to think that Holden’s 5N heuristic would have ruled out refuges ex ante. (Link to other discussion of refuges: /ea/5r/improving_disaster_shelters_to_increase_the/ .)
Thanks for this clarification. After reading the emails I wanted to make exactly this point!
I do think that comparing how good saving a life today is compared to doing something like building bunkers to reduce risk really comes down to an understanding of the world today rather than an understanding of exactly how big the future might be (after you grant that it could be very big). Though choosing 5 as a multiplier looks rather low to me; I’d be happier with something up in 100-1000 range (and I wouldn’t be surprised if my view of the correct figure to use there changes substantially in the future).
I agree with all of that, though maybe I’m a bit more queasy about numbers >100.