One quick thought; often, when things are very grim, you’re pretty okay taking chances.
Imagine we need 500 units of AI progress in order to save the world. In expectation, we’d expect 100. Increasing our amount to 200 doesn’t help us, all that matters is if we can get over 500. In this case, we might want a lot of bifurcation. We’d much prefer a 1% chance of 501 units, than a 100% chance of 409 units, for example.
In this case, lots of randomness/bifurcation will increase total expected value (which is correlated with our chances of getting over 500 units, moreso than it is correlated with the expected units of progress).
I imagine this mainly works with discontinuities, like the function described above (Utility = 0 for units 0 to 499, and Utility = 1 for units of 500+)
One quick thought; often, when things are very grim, you’re pretty okay taking chances.
Imagine we need 500 units of AI progress in order to save the world. In expectation, we’d expect 100. Increasing our amount to 200 doesn’t help us, all that matters is if we can get over 500. In this case, we might want a lot of bifurcation. We’d much prefer a 1% chance of 501 units, than a 100% chance of 409 units, for example.
In this case, lots of randomness/bifurcation will increase total expected value (which is correlated with our chances of getting over 500 units, moreso than it is correlated with the expected units of progress).
I imagine this mainly works with discontinuities, like the function described above (Utility = 0 for units 0 to 499, and Utility = 1 for units of 500+)