I don’t think I said the the US military was good at risk management I think I said that a) the DMDU community (RAND, US military and others) was good at making plans that manage uncertainty and b) that industry was good at risk management
Slight disagreement:
It feels wrong to use reference classes of X to implicitly say that actions the reference class does is good and we ought to emulate them, withoutever an explicit argument that the reference classes’ actions or decision procedures are good!
I do think where reference class X is large and dominant enough it does make sense to assume some trust in their approach or that it is worth some investigation of their approach before dismissing it. For example most (large) businesses have a board and a CEO and a hierarchical management structure so unless I had a good reason to do otherwise that sets a reasonable prior for how I think it is best to run a business.
So even if I had zero evidence I think it would make sense for someone in the EA community to spend time looking into the topic of what tools had worked well in the past to deal with uncertainty and that the US military would be a good place to look for ideas.
Answer:
Answering: is the US military good at making plans that manage uncertainty:
Historical evidence – no. I have zero empirical historical evidence that DMDU tools have worked well for the US military.
Theoretical evidence – yes. I think the theoretical case for these tools is strong, see the case here and here.
Interpersonal evidence – yes. I believe Taleb in Black Swan describes that the people he met in the US military had very good empirical ways of thinking about risk and uncertainty (I don’t have the book here so cannot double check). Similarly to Taleb I have been much impressed by folk in the UK working on counter-terrorism etc, compared to other policy folk who work on risks.
Evidence from trust – mixed. I mostly expect the US military have the right incentives in place to aim to do this well and the ability to test ideas in the field but also would not be surprised if there were a bunch of perverse incentives that corrupted this.
So all in all pretty weak evidence.
Caveat:
My views are probably somewhat moved on from when I wrote this post a year ago. I should revisit it at some point
Great question
Clarification:
I don’t think I said the the US military was good at risk management I think I said that
a) the DMDU community (RAND, US military and others) was good at making plans that manage uncertainty and
b) that industry was good at risk management
Slight disagreement:
I do think where reference class X is large and dominant enough it does make sense to assume some trust in their approach or that it is worth some investigation of their approach before dismissing it. For example most (large) businesses have a board and a CEO and a hierarchical management structure so unless I had a good reason to do otherwise that sets a reasonable prior for how I think it is best to run a business.
For more on this see Common sense as a prior.
So even if I had zero evidence I think it would make sense for someone in the EA community to spend time looking into the topic of what tools had worked well in the past to deal with uncertainty and that the US military would be a good place to look for ideas.
Answer:
Answering: is the US military good at making plans that manage uncertainty:
Historical evidence – no.
I have zero empirical historical evidence that DMDU tools have worked well for the US military.
Theoretical evidence – yes.
I think the theoretical case for these tools is strong, see the case here and here.
Interpersonal evidence – yes.
I believe Taleb in Black Swan describes that the people he met in the US military had very good empirical ways of thinking about risk and uncertainty (I don’t have the book here so cannot double check). Similarly to Taleb I have been much impressed by folk in the UK working on counter-terrorism etc, compared to other policy folk who work on risks.
Evidence from trust – mixed.
I mostly expect the US military have the right incentives in place to aim to do this well and the ability to test ideas in the field but also would not be surprised if there were a bunch of perverse incentives that corrupted this.
So all in all pretty weak evidence.
Caveat:
My views are probably somewhat moved on from when I wrote this post a year ago. I should revisit it at some point