I think that this discussion really comes from the larger discussion about the degree to which we should consider rational choice theory (RCT) to be a normative, as opposed to a positive, theory (for a good overview of the history of this debate, I would highly suggest this article by Wade Hands, especially the example on page 9). As someone with an economics background, I very heavily skew toward seeing it as a positive theory (which is why I pushed back against your statement about economists’ view of risk aversion). In my original reply I wasn’t very specific about what I was saying, so hopefully this will help clarify where I’m coming from!
I just want to say that I agree that rational choice theory (RCT) is dominated by expected utility (EU) theory. However, I disagree with your portrayal of risk aversion. In particular, I agree that risk aversion over expected utility is irrational—but my reasoning for saying this is very different. From an economic standpoint, risk aversion over utils is, by its very definition, irrational. When you define ‘rational’ to mean ‘that which maximizes expected utility’ (as it is defined in EU and RCT models), then of course being risk averse over utils is irrational—under this framework, risk neutrality over utils is a necessary pre-requisite for the model to work at all. This is why, in cases where risk aversion is important (such as the yield example), expected utility calculations take risk aversion into account when calculating the utils associated with each situation—thus making risk aversion over the utils themselves redundant.
Put in a slightly different way, we need to remember that utils do not exist—they are an artifact of our modeling efforts. Risk neutrality over utils is a necessary assumption of RCT in order to develop models that accurately describe decision-making (since RCT was developed as a positive theory). Because of this, the phrase ‘risk aversion over utility’ has not real-world interpretation.
With that in mind, people don’t fail the Allais paradox because of risk aversion over utils, since there is no such thing as being risk averse over utils. Instead, the Allais paradox is a case showing that older RCT models are insufficient for describing the actions of humans—since the empirical results appear to show, in a way, something akin to risk aversion over utils, which in turn breaks the model. This is an important point—put differently, risk neutrality over utils is a necessary assumption of the model, and empirical results that disprove this assumption do not mean that humans are wrong (even though that may be true), it means that the model fails to capture reality. It was because the model broke (in this case, and in others), that economics developed newer positive theories of choice, such as behavioral economics and bounded rationality models, that better describe decision-making.
At most, you can say that the Allais paradox is a case showing that people’s heuristics associated with risk aversion are systematically biased toward decisions that they would not choose if they thought the problem through a bit more. This is definitely a case showing that people are irrational sometimes, and that maybe they should think through these decisions a little more thoroughly, but it does not have anything to do with risk aversion over utility.
Anyways, to bring this back to the main discussion—from this perspective, risk aversion is a completely fine thing to put into models, and it would not be irrational to Alex to factor in risk aversion. This would especially be fine if Alex is worried about the validity of their model itself (which, Alex not being an expert on modeling nor AI risk, should consider to be a real concern). As a last point, I do personally think that we should be more averse to the risks associated with supporting work on far-future stuff and X-risks (which I’ve discussed partially here), but that’s a whole other issue entirely.
I think that this discussion really comes from the larger discussion about the degree to which we should consider rational choice theory (RCT) to be a normative, as opposed to a positive, theory (for a good overview of the history of this debate, I would highly suggest this article by Wade Hands, especially the example on page 9). As someone with an economics background, I very heavily skew toward seeing it as a positive theory (which is why I pushed back against your statement about economists’ view of risk aversion). In my original reply I wasn’t very specific about what I was saying, so hopefully this will help clarify where I’m coming from!
I just want to say that I agree that rational choice theory (RCT) is dominated by expected utility (EU) theory. However, I disagree with your portrayal of risk aversion. In particular, I agree that risk aversion over expected utility is irrational—but my reasoning for saying this is very different. From an economic standpoint, risk aversion over utils is, by its very definition, irrational. When you define ‘rational’ to mean ‘that which maximizes expected utility’ (as it is defined in EU and RCT models), then of course being risk averse over utils is irrational—under this framework, risk neutrality over utils is a necessary pre-requisite for the model to work at all. This is why, in cases where risk aversion is important (such as the yield example), expected utility calculations take risk aversion into account when calculating the utils associated with each situation—thus making risk aversion over the utils themselves redundant.
Put in a slightly different way, we need to remember that utils do not exist—they are an artifact of our modeling efforts. Risk neutrality over utils is a necessary assumption of RCT in order to develop models that accurately describe decision-making (since RCT was developed as a positive theory). Because of this, the phrase ‘risk aversion over utility’ has not real-world interpretation.
With that in mind, people don’t fail the Allais paradox because of risk aversion over utils, since there is no such thing as being risk averse over utils. Instead, the Allais paradox is a case showing that older RCT models are insufficient for describing the actions of humans—since the empirical results appear to show, in a way, something akin to risk aversion over utils, which in turn breaks the model. This is an important point—put differently, risk neutrality over utils is a necessary assumption of the model, and empirical results that disprove this assumption do not mean that humans are wrong (even though that may be true), it means that the model fails to capture reality. It was because the model broke (in this case, and in others), that economics developed newer positive theories of choice, such as behavioral economics and bounded rationality models, that better describe decision-making.
At most, you can say that the Allais paradox is a case showing that people’s heuristics associated with risk aversion are systematically biased toward decisions that they would not choose if they thought the problem through a bit more. This is definitely a case showing that people are irrational sometimes, and that maybe they should think through these decisions a little more thoroughly, but it does not have anything to do with risk aversion over utility.
Anyways, to bring this back to the main discussion—from this perspective, risk aversion is a completely fine thing to put into models, and it would not be irrational to Alex to factor in risk aversion. This would especially be fine if Alex is worried about the validity of their model itself (which, Alex not being an expert on modeling nor AI risk, should consider to be a real concern). As a last point, I do personally think that we should be more averse to the risks associated with supporting work on far-future stuff and X-risks (which I’ve discussed partially here), but that’s a whole other issue entirely.
Hope that helps clarify my position!