How do you think we could make it clearer in the post?
I think your example should focus on the mean of the ratio between the effect and cost, not on the mean of the ratio between the cost and effect. The latter is a bad metric because:
A very small âcostâ/ââeffectâ could correspond to interventions that are either quite bad or good (since âcostâ/ââ = 0). This means small numerical errors could lead to large differences in mean(âcostâ/ââeffectâ), which is bad.
When changing from negative values of âcostâ/ââeffectâ to positive ones, the goodness of the intervention increases (changing from harmful to beneficial). However, for negative and positive values, a higher âcostâ/ââeffectâ corresponds to a worse intervention.
The metric âeffectâ/ââcostâ has good properties:
A higher value always implies a better intervention (at least in theory).
I think your example should focus on the mean of the ratio between the effect and cost, not on the mean of the ratio between the cost and effect. The latter is a bad metric because:
A very small âcostâ/ââeffectâ could correspond to interventions that are either quite bad or good (since âcostâ/ââ = 0). This means small numerical errors could lead to large differences in mean(âcostâ/ââeffectâ), which is bad.
When changing from negative values of âcostâ/ââeffectâ to positive ones, the goodness of the intervention increases (changing from harmful to beneficial). However, for negative and positive values, a higher âcostâ/ââeffectâ corresponds to a worse intervention.
The metric âeffectâ/ââcostâ has good properties:
A higher value always implies a better intervention (at least in theory).
A null value correspond to neutrality.