I’ve now edited the post to reference mean(QALYs)/mean($). You can find this by ctrl+f for “EDIT 22/06/2022” and under the graph charts.
Note that I’ve used mean($)/mean(QALYS) ($8k) rather than 1/mean(QALYs/$) ($5k), because it seems to me that is more the quantity of interest, but I’m not hugely certain of this.
Another modeling issue is that each individual variable is log-normal rather than normal/uniform. This means that while probability of success is “0.01 to 0.1”, suggesting 5.5% as the “average”, the actual computed average is 4%. This doesn’t make a big difference on its own but it’s important when multiplying together lots of numbers. I’m not sure that converting log-normal to uniform would in general lead to better estimates but it’s important to flag.
Quick point that I’m fairly suspicious of uniform distributions for such uncertainties.
I’d agree that our format of a 90% CI can be deceptive, especially when people aren’t used to it. I imagine it would eventually be really neat to have probability distribution support right in the EA Forum. Until then, I’m curious if there are better ways to write the statistical summaries of many variables.
To me, “0.01 to 0.1” doesn’t suggest that 5.5% is the “mean”, but I could definitely appreciate that others would think that.
Why did you take the mean $/QALY instead of mean QALY/$ (which expected value analysis would suggest)? When I do that I get $5000/QALY as the mean.
Because I didn’t think about it and I liked having numbers which were more interpretable, e.g. 3.4*10^-6 QALYs/$ is to me less interpretable that $290k/QALY, and same with 7.7 * 10^-4 vs $1300/QALY.
Another poster reached out and mentioned he was writing a post about this particular mistake, so I thought I’d leave the example up.
Another poster reached out and mentioned he was writing a post about this particular mistake, so I thought I’d leave the example up.
Please feel free to edit it, it will take me a while to actually post anything, I’m still thinking about how to handle this issue in the general case. I think you can keep the interpretability by doing mean(cost)/mean(QALY), but I still don’t know how to handle the probability distribution.
I think by not editing it we risk other people copying the mistake
Why did you take the mean $/QALY instead of mean QALY/$ (which expected value analysis would suggest)? When I do that I get $5000/QALY as the mean.
I’ve now edited the post to reference mean(QALYs)/mean($). You can find this by ctrl+f for “EDIT 22/06/2022” and under the graph charts.
Note that I’ve used mean($)/mean(QALYS) ($8k) rather than 1/mean(QALYs/$) ($5k), because it seems to me that is more the quantity of interest, but I’m not hugely certain of this.
Another modeling issue is that each individual variable is log-normal rather than normal/uniform. This means that while probability of success is “0.01 to 0.1”, suggesting 5.5% as the “average”, the actual computed average is 4%. This doesn’t make a big difference on its own but it’s important when multiplying together lots of numbers. I’m not sure that converting log-normal to uniform would in general lead to better estimates but it’s important to flag.
Quick point that I’m fairly suspicious of uniform distributions for such uncertainties.
I’d agree that our format of a 90% CI can be deceptive, especially when people aren’t used to it. I imagine it would eventually be really neat to have probability distribution support right in the EA Forum. Until then, I’m curious if there are better ways to write the statistical summaries of many variables.
To me, “0.01 to 0.1” doesn’t suggest that 5.5% is the “mean”, but I could definitely appreciate that others would think that.
Because I didn’t think about it and I liked having numbers which were more interpretable, e.g. 3.4*10^-6 QALYs/$ is to me less interpretable that $290k/QALY, and same with 7.7 * 10^-4 vs $1300/QALY.
Another poster reached out and mentioned he was writing a post about this particular mistake, so I thought I’d leave the example up.
Please feel free to edit it, it will take me a while to actually post anything, I’m still thinking about how to handle this issue in the general case. I think you can keep the interpretability by doing mean(cost)/mean(QALY), but I still don’t know how to handle the probability distribution.
I think by not editing it we risk other people copying the mistake
Done.
Post I mentioned in the comments below now here: Probability distributions of Cost-Effectiveness can be misleading