You could also consider providing your probabilities in the form of distributions, for example by answering a question like “What probability would you assign to tribbles [1] being sentient after 1000 more hours of research?” This would perhaps solve part of the problem of communicating the uncertainty which you want to communicate.
Some example answers might be:
In the first one your uncertainty is significant. You have a probability of ~45%, but you consider it likely that you will update a lot. You just don’t know in which direction: You consider it equally likely that you will end up at 30% or at 70%.
In the second one, research has mostly been done and you’ve already mostly made up your mind. You have a probability of ~45%, and you believe that further research is most likely to move you to 42%, or to 47%, but not that much further. You’d be shocked if you ended up with a probability of more than 60%.
The third one is the distributional equivalent of you making a shrug. Your probability is 45%, but really, tribbles being so underexplored means that your distribution looks pretty much uniform.
Note that it’s even possible that all three distributions have the same mean (~45%). This would mean that in all three cases you’d think that a bet of 45:55 would be a fair bet (that is, that it has an expected value of 0).
You could also predict the standard deviation of your distribution (how broad it is) after 1000h of research, 2000h, 5000h, etc., aggregate the distributions of all your researchers, and do other nice things.
You could also consider providing your probabilities in the form of distributions, for example by answering a question like “What probability would you assign to tribbles [1] being sentient after 1000 more hours of research?” This would perhaps solve part of the problem of communicating the uncertainty which you want to communicate.
Some example answers might be:
In the first one your uncertainty is significant. You have a probability of ~45%, but you consider it likely that you will update a lot. You just don’t know in which direction: You consider it equally likely that you will end up at 30% or at 70%.
In the second one, research has mostly been done and you’ve already mostly made up your mind. You have a probability of ~45%, and you believe that further research is most likely to move you to 42%, or to 47%, but not that much further. You’d be shocked if you ended up with a probability of more than 60%.
The third one is the distributional equivalent of you making a shrug. Your probability is 45%, but really, tribbles being so underexplored means that your distribution looks pretty much uniform.
Note that it’s even possible that all three distributions have the same mean (~45%). This would mean that in all three cases you’d think that a bet of 45:55 would be a fair bet (that is, that it has an expected value of 0).
You could also predict the standard deviation of your distribution (how broad it is) after 1000h of research, 2000h, 5000h, etc., aggregate the distributions of all your researchers, and do other nice things.
[1] Fictional species.
Thanks! That’s helpful way of thinking about the topic and a useful strategy for addressing the problem.