the OOM of variation in “ground truth” come from alpha and n, not xmin
alpha, we could talk all day, but the model is not extremely sensitive to it
on the other hand, if you say let’s have more OOMs in the possible values of ground truth, following the power law, that means jacking n up
and when you jack n up you have even more opportunities for errors to be crazy big, and this effect dominates (at least that’s what I read from the OP) and the curse becomes worse
now if we change alpha and n at the same time, idk
my honest opinion is that numbers are just one way to process information, and using them for this is so out of distribution that it’s essentially meaningless (as it is when discussing p(doom) and stuff like that)
Sure, I understand where the values come from. I’m saying the distribution created leads to (IMO) clearly wacky results. The difference between the most vs least spooky X-risks is way more than a 100X difference.
I personally get some value out of numbers & sanity checking them like this, but your mileage may vary.
Here is an example run I did where I tuned down the alpha to 1.5 and tuned the lognormal standard deviation down to 1.5:
Here, the top actual threat really is 7 orders of magnitude more dangerous than the bottom evaluated threat. However, the top apparent threat is overestimated by a factor of 10,000 or so.
If I do a bunch of runs with these settings, the median overestimate is over 100x:
So even if I trust your vibes here (which do not seem to be based on anything), the curse can still hit quite badly. I personally believe that the spread of numbers that people make up is going to be higher than the spread of actual threats: when we look at actual surveys you get a highest-lowest estimate spread of 11 orders of magnitude for some questions.
One thing that might be confusing you is that the power law model assumes that only threats above a certain threshold of actual danger are considered (this is the xmin factor). Obviously nuclear risk is a much greater risk than stubbing your toe, but it’s not going to show up the model.
The difference between the most vs least spooky X-risks is way more than a 100X difference.
I think I would agree with this, if I had to put a number.
What I mean in my comment is, with this model, if you say okay let’s pick a bigger n so that we see bigger differences in OOMs, then you are also introducing more points of failure in the estimation, and that effect dominates.
Do you have an a priori reason to discard this? Besides the conclusion being wacky, which is a good reason to discard a model anyways.
the OOM of variation in “ground truth” come from alpha and n, not xmin
alpha, we could talk all day, but the model is not extremely sensitive to it
on the other hand, if you say let’s have more OOMs in the possible values of ground truth, following the power law, that means jacking n up
and when you jack n up you have even more opportunities for errors to be crazy big, and this effect dominates (at least that’s what I read from the OP) and the curse becomes worse
now if we change alpha and n at the same time, idk
my honest opinion is that numbers are just one way to process information, and using them for this is so out of distribution that it’s essentially meaningless (as it is when discussing p(doom) and stuff like that)
Sure, I understand where the values come from. I’m saying the distribution created leads to (IMO) clearly wacky results. The difference between the most vs least spooky X-risks is way more than a 100X difference.
I personally get some value out of numbers & sanity checking them like this, but your mileage may vary.
Here is an example run I did where I tuned down the alpha to 1.5 and tuned the lognormal standard deviation down to 1.5:
Here, the top actual threat really is 7 orders of magnitude more dangerous than the bottom evaluated threat. However, the top apparent threat is overestimated by a factor of 10,000 or so.
If I do a bunch of runs with these settings, the median overestimate is over 100x:
So even if I trust your vibes here (which do not seem to be based on anything), the curse can still hit quite badly. I personally believe that the spread of numbers that people make up is going to be higher than the spread of actual threats: when we look at actual surveys you get a highest-lowest estimate spread of 11 orders of magnitude for some questions.
One thing that might be confusing you is that the power law model assumes that only threats above a certain threshold of actual danger are considered (this is the xmin factor). Obviously nuclear risk is a much greater risk than stubbing your toe, but it’s not going to show up the model.
The difference between the most vs least spooky X-risks is way more than a 100X difference.
I think I would agree with this, if I had to put a number.
What I mean in my comment is, with this model, if you say okay let’s pick a bigger n so that we see bigger differences in OOMs, then you are also introducing more points of failure in the estimation, and that effect dominates.
Do you have an a priori reason to discard this? Besides the conclusion being wacky, which is a good reason to discard a model anyways.
I agree that merely increasing n would not change the OP’s conclusion that errors dominate.
My point is more that they picked too big of parameters for error variance and too small of parameters for risk-size variance.