Lorenzo Buonanno🔸 comments on Probability distributions of Cost-Effectiveness can be misleading

[Lorenzo Buonanno🔸]

I think that the problem is with taking the expectation over $\frac{1}{effect}$, instead of multiplying the expected values $mean\left(\frac{1}{effect}\right)\cdot mean\left(cost\right)$.

I agree! It’s the division that’s not linear, not the multiplication.
How do you think we could make it clearer in the post?

Edit: on more thought, I’m not sure I understand your point.$mean\left(1/effect\right)\cdot mean\left(cost\right)$ gives the “wrong” result (e.g. if $effect$ can be 0 with non 0 probability, $mean\left(1/effect\right)$ will be $+\infty$)

[SamNolan]

Thank you so much for the post! I might communicate it as:

People are asking the question “How much money do you have to donate to get an expected value of 1 unit of good” Which could be formulated as:

$E\left(good\right(x\left)\right)=1$

where $x$ is the amount you donate and $good\left(x\right)$ is the amount of utility you get out of it.

In most cases, this is linear, so: $good\left(x\right)=\frac{good}{cost}\ast x$. And $E\left(\frac{good}{cost}x\right)=1$.

Solving for x in this case gets $x=E(\frac{good}{cost}{)}^{-1}$, but the mistake is to solve it and get $x=E\left(\frac{cost}{good}\right)$.

Please correct me if this is a bad way to formulate the problem! Can’t wait to see your future work as well

[EdoArad]

nice explanation :)

[Vasco Grilo🔸]

People are asking the question “How much money do you have to donate to get an expected value of 1 unit of good”.

I think the question is:

• How can I do as much good as possible with C units of cost?

This corresponds to the problem of maximising E(U(C)), where U(c) is the utility achieved (via a certain intervention) for the cost c (which must not exceed C). If the budget C is small enough (thinking at the margin):

• U(C) = U’(0)*C, where U’(c) is the derivate of U with respect to cost.

Assuming U’(0) and C are independent, mean(“effect”/​”cost”) equals mean(“effect”)/​mean(“cost”):

• mean(“effect”/​”cost”) = E(U(C)/​C) = E(U’(0)*C/​C) = E(U’(0)).

• mean(“effect”)/​mean(“cost”) = E(U(C))/​E(C) = E(U’(0))*E(C)/​E(C) (assuming independence between U’(0) and C) = E(U’(0)).

So, it seems that, regardless of the metric we choose, we should maximise E(U’(0)), i.e. the expected marginal cost-effectiveness. However, U’(0) and C will not be independent for large C, so I think it is better to maximise mean(“effect”/​”cost”).

[Stan Pinsent]

Is that a typo on your final bullet point? Should say mean(“effect”)/​mean(“cost”).

[Vasco Grilo🔸]

Hi Stan,

Yes, it was a typo, which I have now corrected. Thanks for catching it! I have also added one extra sentence at the end:

However, U’(0) and C will not be independent for large C, so I think it is better to maximise mean(“effect”/​”cost”).

[Lorenzo Buonanno🔸]

Thanks for commenting here, and thanks again for your initial feedback!

Can’t wait to see your future work as well

I don’t really have anything planned in this area, what would you be excited to see?

[EdoArad]

I think Michael Dickens explains this better than me in a more recent comment. The point is that we usually care about effect/​cost rather than the other way around (although I’d love to understand more clearly why and when exactly that’s true). In your example, you have implicitly computed this and then compare it to Guesstimate’s model computing cost/​effect.

[Lorenzo Buonanno🔸]

Replied to Michael Dickens, curious about your thoughts! We should definitely add a section on this

[Vasco Grilo🔸]

How do you think we could make it clearer in the post?

I think your example should focus on the mean of the ratio between the effect and cost, not on the mean of the ratio between the cost and effect. The latter is a bad metric because:

• A very small “cost”/​”effect” could correspond to interventions that are either quite bad or good (since “cost”/​$\infty$ = 0). This means small numerical errors could lead to large differences in mean(“cost”/​”effect”), which is bad.

• When changing from negative values of “cost”/​”effect” to positive ones, the goodness of the intervention increases (changing from harmful to beneficial). However, for negative and positive values, a higher “cost”/​”effect” corresponds to a worse intervention.

The metric “effect”/​”cost” has good properties:

• A higher value always implies a better intervention (at least in theory).

• A null value correspond to neutrality.