I don’t think EV calculations directly guard against motivated reasoning.
I think the main benefit of EV calculations is that they allow more precise comparison between interventions (compared to say, just calling many interventions ‘good’).
However, many EV calculations involve probabilities and estimates derived from belief rather than from empirical evidence. These probabilities and estimates are highly prone to motivated reasoning and cognitive biases.
For example, if I was to calculate the EV of an EA funding org investing in more transparency, I might need to estimate a percentage of grants which were approved but ideally should not have been. As someone who has a strong prior in favour of transparency, I might estimate this to be much higher than someone who has a strong prior against transparency. This could have a large effect on my calculated EV.
That being said, there are certainly EV calculations where all the inputs can be pegged to empirical evidence, especially in the cause area of international development and global health. These EV calculations are less prone to motivated reasoning, but motivated reasoning remains nonetheless, because where there is empirical evidence available from different sources, motivated reasoning may affect the source used. (Guy Raveh points out some other ways that motivated reasoning can affect these calculations too)
With sufficient transparency, I think EV calculations can help reduce motivated reasoning since people can debate the inputs into the EV calculation, allowing the probabilities and estimates derived from belief to be refined, which may make them more accurate than before.
I agree that EV calculations are less susceptible to motivated reasoning than alternative approaches, but I think they are very susceptible nonetheless, which is why I think we should make certain changes to how they are used and implement stronger safeguards against motivated reasoning.
Thanks for your comment.
I don’t think EV calculations directly guard against motivated reasoning.
I think the main benefit of EV calculations is that they allow more precise comparison between interventions (compared to say, just calling many interventions ‘good’).
However, many EV calculations involve probabilities and estimates derived from belief rather than from empirical evidence. These probabilities and estimates are highly prone to motivated reasoning and cognitive biases.
For example, if I was to calculate the EV of an EA funding org investing in more transparency, I might need to estimate a percentage of grants which were approved but ideally should not have been. As someone who has a strong prior in favour of transparency, I might estimate this to be much higher than someone who has a strong prior against transparency. This could have a large effect on my calculated EV.
That being said, there are certainly EV calculations where all the inputs can be pegged to empirical evidence, especially in the cause area of international development and global health. These EV calculations are less prone to motivated reasoning, but motivated reasoning remains nonetheless, because where there is empirical evidence available from different sources, motivated reasoning may affect the source used. (Guy Raveh points out some other ways that motivated reasoning can affect these calculations too)
With sufficient transparency, I think EV calculations can help reduce motivated reasoning since people can debate the inputs into the EV calculation, allowing the probabilities and estimates derived from belief to be refined, which may make them more accurate than before.
I agree that EV calculations are less susceptible to motivated reasoning than alternative approaches, but I think they are very susceptible nonetheless, which is why I think we should make certain changes to how they are used and implement stronger safeguards against motivated reasoning.