Right. I think using unsharp probabilities, and expected values is fine to highlight it is unclear which of the interventions being compared has the highest expected cost-effectiveness. However, I do not see what is the advantage of this relative to just getting wide distributions for the cost-effectiveness, and showing these overlap a lot, which would be a sign that decreasing their uncertaity may have a higher expected cost-effectiveness than picking the intervention with the highest expected cost-effectiveness. One can analyse value of information (VOI) using perfectly sharp credences.
just getting wide distributions for the cost-effectiveness
A normal Gaussian distribution? If so, then you still think the value in the middle of the curve is uniquely appropriate (even if barely so). To me, that’s the key difference between A) imprecision and B) precision with severe credal fragility. The former assumes you can’t non-arbitrarily pin down a precise credence at all, while the latter assumes you still can.
If VOI is overwhelmingly high, both A and B might recommend research, such that the difference doesn’t matter. But it matters a lot at least in situations where actors want to fund non-research things (because they think VOI is not that high or whatever). Then, A and B deeply disagree on what should be done.
I think cost-effectiveness accounting for effects on all organisms spans many orders of magnitude (OOMs) due to large uncertainty about how to compare welfare across species. So I expect something like a loguniform or lognormal distributions would be more appropriate. Ideally, one would model the inputs as distributions instead of assuming a distribution for the cost-effectiveness.
In the context of assessing interventions with very uncertain cost-effectiveness (in my view, practically any context), in which sense would it matter a lot whether one uses sharp or unsharp probabilities? With sharp probabilities, it would be close to arbitrary which interventions should be supported. With unsharp probabilities, it would be indeterminate which interventions should be supported, but one would still end up supporting something based on some criteria. From my perspective, it is unclear which one would lead to greater impact. Given the large uncertainty, it is not even clear to me whether any of the approaches would outperform picking interventions randomly.
So I believe the priority would be decreasing uncertainty. I expect this can be most cost-effectively achieved via research (on comparing welfare across species). However, supporting the interventions under comparison also indirectly decreases uncertainty to some extent. Funders who do not want to fund research directly decreasing the uncertainty might be open to funding research aiming to figure out how to decrease uncertainty via supporting existing interventions. They could then update to some extent towards funding interventions which look better in terms of decreasing uncertainty. I guess ones contributing to moral circle expansion help attracts resources to target more neglected animals, including to study how their welfare compares with that of other less neglected animals.
In the context of assessing interventions with very uncertain cost-effectiveness (in my view, practically any context), in which sense would it matter a lot whether one uses sharp or unsharp probabilities? With sharp probabilities, it would be close to arbitrary which interventions should be supported. With unsharp probabilities, it would be indeterminate which interventions should be supported, but one would still end up supporting something based on some criteria.
One thing is whoever does not reject UNSHARP might not have severely imprecise credences about everything. I might believe that
intervention 1 has severely indeterminate but astronomically high (positive or negative) EV.
intervention 2 seems overall good, although it has lower EV.
Then, I’d probably prioritize intervention 2. If I instead endorsed SHARP, I might favor intervention 1 (because of a sufficient 51% credence 1 is good). (I’m actually not sure about this, though. One could argue that 1 and 2 remain incomparable and that I have no reason to favor 2 over 1.)
Another thing, assuming there is no 2-like intervention, is that the criterion to pick could be something other than “act straightforwardly as if you were endorsing SHARP”. It could instead be, e.g., some (other) form of bracketing.
One could argue that 1 and 2 remain incomparable and that I have no reason to favor 2 over 1.
If the absolute value of the expected cost-effectiveness of 1 was astronomically larger than that of intervention 2, I think comparing the interventions would be similar to comparing intervention 1 with one with cost-effectiveness of 0 (burning money). It is very unclear whether the expected cost-effectiveness of 1 is positive or negative. So it would be close to arbitrary which intervention has the highest expected cost-effectiveness.
Another thing, assuming there is no 2-like intervention, is that the criterion to pick could be something other than “act straightforwardly as if you were endorsing SHARP”. It could instead be some (other) form of bracketing.
Right. I think using unsharp probabilities, and expected values is fine to highlight it is unclear which of the interventions being compared has the highest expected cost-effectiveness. However, I do not see what is the advantage of this relative to just getting wide distributions for the cost-effectiveness, and showing these overlap a lot, which would be a sign that decreasing their uncertaity may have a higher expected cost-effectiveness than picking the intervention with the highest expected cost-effectiveness. One can analyse value of information (VOI) using perfectly sharp credences.
A normal Gaussian distribution? If so, then you still think the value in the middle of the curve is uniquely appropriate (even if barely so). To me, that’s the key difference between A) imprecision and B) precision with severe credal fragility. The former assumes you can’t non-arbitrarily pin down a precise credence at all, while the latter assumes you still can.
If VOI is overwhelmingly high, both A and B might recommend research, such that the difference doesn’t matter. But it matters a lot at least in situations where actors want to fund non-research things (because they think VOI is not that high or whatever). Then, A and B deeply disagree on what should be done.
I think cost-effectiveness accounting for effects on all organisms spans many orders of magnitude (OOMs) due to large uncertainty about how to compare welfare across species. So I expect something like a loguniform or lognormal distributions would be more appropriate. Ideally, one would model the inputs as distributions instead of assuming a distribution for the cost-effectiveness.
In the context of assessing interventions with very uncertain cost-effectiveness (in my view, practically any context), in which sense would it matter a lot whether one uses sharp or unsharp probabilities? With sharp probabilities, it would be close to arbitrary which interventions should be supported. With unsharp probabilities, it would be indeterminate which interventions should be supported, but one would still end up supporting something based on some criteria. From my perspective, it is unclear which one would lead to greater impact. Given the large uncertainty, it is not even clear to me whether any of the approaches would outperform picking interventions randomly.
So I believe the priority would be decreasing uncertainty. I expect this can be most cost-effectively achieved via research (on comparing welfare across species). However, supporting the interventions under comparison also indirectly decreases uncertainty to some extent. Funders who do not want to fund research directly decreasing the uncertainty might be open to funding research aiming to figure out how to decrease uncertainty via supporting existing interventions. They could then update to some extent towards funding interventions which look better in terms of decreasing uncertainty. I guess ones contributing to moral circle expansion help attracts resources to target more neglected animals, including to study how their welfare compares with that of other less neglected animals.
One thing is whoever does not reject UNSHARP might not have severely imprecise credences about everything. I might believe that
intervention 1 has severely indeterminate but astronomically high (positive or negative) EV.
intervention 2 seems overall good, although it has lower EV.
Then, I’d probably prioritize intervention 2. If I instead endorsed SHARP, I might favor intervention 1 (because of a sufficient 51% credence 1 is good). (I’m actually not sure about this, though. One could argue that 1 and 2 remain incomparable and that I have no reason to favor 2 over 1.)
Another thing, assuming there is no 2-like intervention, is that the criterion to pick could be something other than “act straightforwardly as if you were endorsing SHARP”. It could instead be, e.g., some (other) form of bracketing.
If the absolute value of the expected cost-effectiveness of 1 was astronomically larger than that of intervention 2, I think comparing the interventions would be similar to comparing intervention 1 with one with cost-effectiveness of 0 (burning money). It is very unclear whether the expected cost-effectiveness of 1 is positive or negative. So it would be close to arbitrary which intervention has the highest expected cost-effectiveness.
Bracketing departs from impartiality, and I find this very unappealing.