I think this is better parsed as diminishing marginal returns to information.
How does this account for the leftmost digit giving the most information, rather than the rightmost digit (or indeed any digit between them)?
per-thousandths does not have double the information of per-cents, but 50% more
Let’s say I give you $1 + $Y where Y is either 0, $0.1, $0.2 … or $0.9. (Note $1 is analogous to 1%, and Y is equivalent adding a decimal place. I.e. per-thousandths vs per-cents.) The average value of Y, given a uniform distribution, is $0.45. Thus, against $1, Y adds almost half the original value, i.e. $0.45/$1 (45%). But what if I instead gave you $99 + $Y? $0.45 is less than 1% of the value of $99.
The leftmost digit is more valuable because it corresponds to a greater place value (so the magnitude of the value difference between places is going to be dependent on the numeric base you use). I don’t know information theory, so I’m not sure how to calculate the value of the first two digits compared to the third, but I don’t think per-thousandths has 50% more information than per-cents.
How does this account for the leftmost digit giving the most information, rather than the rightmost digit (or indeed any digit between them)?
Let’s say I give you $1 + $Y where Y is either 0, $0.1, $0.2 … or $0.9. (Note $1 is analogous to 1%, and Y is equivalent adding a decimal place. I.e. per-thousandths vs per-cents.) The average value of Y, given a uniform distribution, is $0.45. Thus, against $1, Y adds almost half the original value, i.e. $0.45/$1 (45%). But what if I instead gave you $99 + $Y? $0.45 is less than 1% of the value of $99.
The leftmost digit is more valuable because it corresponds to a greater place value (so the magnitude of the value difference between places is going to be dependent on the numeric base you use). I don’t know information theory, so I’m not sure how to calculate the value of the first two digits compared to the third, but I don’t think per-thousandths has 50% more information than per-cents.