the large set of comparison data was used to construct a cardinal ordering
I really should know this, but is a cardinal ordering one where you can say âA is twice as bad as Bâ or not? My understanding of their process is they used their stats to put everything in order from least-harmful to most-harmful, but this didnât get them more than just an ordering. The equivalence questions added not only the information necessary to put this on the 0-1 scale, but also what was needed to say âA costs 3x less than B to treat, but we should still treat B because itâs 5x worseâ. An ordering by itself doesnât get us very far.
A cardinal ordering, strictly speaking, is one where you can say âthe difference between A and B is twice as large as the difference between A and Câ. Iâd assumed that if you had âperfect healthâ on your scale, you could use distance from this to express âtwice as bad asâ.
From section 2A of the appendix you linked to:
The implication of this is that the probit regression yields estimates of values for each health state that capture the relative differences in health levels between states, consistent with the paired comparison responses, but that these health-state values are on an arbitrary scale rather than on a unique disability weight scale that ranges between 0 and 1.
However from looking at the rest of section 2 it seems that they didnât (as I had thought) just take this cardinal scale at face value, using the population health equivalence questions to anchor the endpoints. Rather they made some assumptions about the shape of transformation required, and then used the extra data to fill in some of the parameter choices. So âtwice as far from perfect healthâ doesnât necessarily translate to âtwice as badââbut it does translate to a precise statement about how much worse.
If you can turn a bunch of âA is worse than Bâ statements into a cardinal ordering, then why do you need the population equivalence questions at all? Why not just include âperfect healthâ and âdeathâ among your disabilities? Then we can eventually say âthe difference between perfect health and A is X% of the difference between perfect health and death.â
I guess part of my confusion is I donât really see how you can get this cardinal ordering from the data. So letâs say we find that condition A is universally considered worse than all other conditions. Perhaps itâs âdeathâ, perhaps itâs just clearly the worst of the conditions weâre looking at. How can statistics give us a ratio by which itâs worse? If somehow it were twice as bad we would still see it be considered as âworstâ in all itâs comparisons.
You are correct. You canât really turn the ordinal stuff into a cardinal ordering, just into a kind of proxy ordering that has some cardinal structure, but it might not correspond to the cardinal structure we care about. For example if âperfect healthâ was added and 100% of people ranked this above the other choice, then it would end up very far (possibly infinitely far) from the nearest option on the cardinal scale. What it is really measuring is the amount of disagreement about things at this part of the ordering, which is a proxy for closeness of the health levels, but there are cases like âperfect healthâ vs slightly worse than that where they are close but there is no disagreement.
I really should know this, but is a cardinal ordering one where you can say âA is twice as bad as Bâ or not? My understanding of their process is they used their stats to put everything in order from least-harmful to most-harmful, but this didnât get them more than just an ordering. The equivalence questions added not only the information necessary to put this on the 0-1 scale, but also what was needed to say âA costs 3x less than B to treat, but we should still treat B because itâs 5x worseâ. An ordering by itself doesnât get us very far.
A cardinal ordering, strictly speaking, is one where you can say âthe difference between A and B is twice as large as the difference between A and Câ. Iâd assumed that if you had âperfect healthâ on your scale, you could use distance from this to express âtwice as bad asâ.
From section 2A of the appendix you linked to:
However from looking at the rest of section 2 it seems that they didnât (as I had thought) just take this cardinal scale at face value, using the population health equivalence questions to anchor the endpoints. Rather they made some assumptions about the shape of transformation required, and then used the extra data to fill in some of the parameter choices. So âtwice as far from perfect healthâ doesnât necessarily translate to âtwice as badââbut it does translate to a precise statement about how much worse.
If you can turn a bunch of âA is worse than Bâ statements into a cardinal ordering, then why do you need the population equivalence questions at all? Why not just include âperfect healthâ and âdeathâ among your disabilities? Then we can eventually say âthe difference between perfect health and A is X% of the difference between perfect health and death.â
I guess part of my confusion is I donât really see how you can get this cardinal ordering from the data. So letâs say we find that condition A is universally considered worse than all other conditions. Perhaps itâs âdeathâ, perhaps itâs just clearly the worst of the conditions weâre looking at. How can statistics give us a ratio by which itâs worse? If somehow it were twice as bad we would still see it be considered as âworstâ in all itâs comparisons.
You are correct. You canât really turn the ordinal stuff into a cardinal ordering, just into a kind of proxy ordering that has some cardinal structure, but it might not correspond to the cardinal structure we care about. For example if âperfect healthâ was added and 100% of people ranked this above the other choice, then it would end up very far (possibly infinitely far) from the nearest option on the cardinal scale. What it is really measuring is the amount of disagreement about things at this part of the ordering, which is a proxy for closeness of the health levels, but there are cases like âperfect healthâ vs slightly worse than that where they are close but there is no disagreement.