the large set of comparison data was used to construct a cardinal ordering
I really should know this, but is a cardinal ordering one where you can say “A is twice as bad as B” or not? My understanding of their process is they used their stats to put everything in order from least-harmful to most-harmful, but this didn’t get them more than just an ordering. The equivalence questions added not only the information necessary to put this on the 0-1 scale, but also what was needed to say “A costs 3x less than B to treat, but we should still treat B because it’s 5x worse”. An ordering by itself doesn’t get us very far.
A cardinal ordering, strictly speaking, is one where you can say “the difference between A and B is twice as large as the difference between A and C”. I’d assumed that if you had “perfect health” on your scale, you could use distance from this to express “twice as bad as”.
From section 2A of the appendix you linked to:
The implication of this is that the probit regression yields estimates of values for each health state that capture the relative differences in health levels between states, consistent with the paired comparison responses, but that these health-state values are on an arbitrary scale rather than on a unique disability weight scale that ranges between 0 and 1.
However from looking at the rest of section 2 it seems that they didn’t (as I had thought) just take this cardinal scale at face value, using the population health equivalence questions to anchor the endpoints. Rather they made some assumptions about the shape of transformation required, and then used the extra data to fill in some of the parameter choices. So “twice as far from perfect health” doesn’t necessarily translate to “twice as bad”—but it does translate to a precise statement about how much worse.
If you can turn a bunch of “A is worse than B” statements into a cardinal ordering, then why do you need the population equivalence questions at all? Why not just include “perfect health” and “death” among your disabilities? Then we can eventually say “the difference between perfect health and A is X% of the difference between perfect health and death.”
I guess part of my confusion is I don’t really see how you can get this cardinal ordering from the data. So let’s say we find that condition A is universally considered worse than all other conditions. Perhaps it’s “death”, perhaps it’s just clearly the worst of the conditions we’re looking at. How can statistics give us a ratio by which it’s worse? If somehow it were twice as bad we would still see it be considered as “worst” in all it’s comparisons.
You are correct. You can’t really turn the ordinal stuff into a cardinal ordering, just into a kind of proxy ordering that has some cardinal structure, but it might not correspond to the cardinal structure we care about. For example if ‘perfect health’ was added and 100% of people ranked this above the other choice, then it would end up very far (possibly infinitely far) from the nearest option on the cardinal scale. What it is really measuring is the amount of disagreement about things at this part of the ordering, which is a proxy for closeness of the health levels, but there are cases like ‘perfect health’ vs slightly worse than that where they are close but there is no disagreement.
I really should know this, but is a cardinal ordering one where you can say “A is twice as bad as B” or not? My understanding of their process is they used their stats to put everything in order from least-harmful to most-harmful, but this didn’t get them more than just an ordering. The equivalence questions added not only the information necessary to put this on the 0-1 scale, but also what was needed to say “A costs 3x less than B to treat, but we should still treat B because it’s 5x worse”. An ordering by itself doesn’t get us very far.
A cardinal ordering, strictly speaking, is one where you can say “the difference between A and B is twice as large as the difference between A and C”. I’d assumed that if you had “perfect health” on your scale, you could use distance from this to express “twice as bad as”.
From section 2A of the appendix you linked to:
However from looking at the rest of section 2 it seems that they didn’t (as I had thought) just take this cardinal scale at face value, using the population health equivalence questions to anchor the endpoints. Rather they made some assumptions about the shape of transformation required, and then used the extra data to fill in some of the parameter choices. So “twice as far from perfect health” doesn’t necessarily translate to “twice as bad”—but it does translate to a precise statement about how much worse.
If you can turn a bunch of “A is worse than B” statements into a cardinal ordering, then why do you need the population equivalence questions at all? Why not just include “perfect health” and “death” among your disabilities? Then we can eventually say “the difference between perfect health and A is X% of the difference between perfect health and death.”
I guess part of my confusion is I don’t really see how you can get this cardinal ordering from the data. So let’s say we find that condition A is universally considered worse than all other conditions. Perhaps it’s “death”, perhaps it’s just clearly the worst of the conditions we’re looking at. How can statistics give us a ratio by which it’s worse? If somehow it were twice as bad we would still see it be considered as “worst” in all it’s comparisons.
You are correct. You can’t really turn the ordinal stuff into a cardinal ordering, just into a kind of proxy ordering that has some cardinal structure, but it might not correspond to the cardinal structure we care about. For example if ‘perfect health’ was added and 100% of people ranked this above the other choice, then it would end up very far (possibly infinitely far) from the nearest option on the cardinal scale. What it is really measuring is the amount of disagreement about things at this part of the ordering, which is a proxy for closeness of the health levels, but there are cases like ‘perfect health’ vs slightly worse than that where they are close but there is no disagreement.