Hey it’s cool that this got written up. I’ve only read your summary, not the paper, and I’m not an expert. However, I think I’d argue against using this approach, and instead in favour of another. It seems equivalent to using ratios to me, and I think you should use the geometric mean of odds instead when pooling ideas together (as you mentioned/asked about in your question 7.). The reason is that Upco uses each opinion as a new piece of independent evidence. Geometric mean of odds merely takes the (correct) average of opinions (in a Bayesian sense).
Here’s one huge problem with the Upco method as you present it: two people think that it’s a 1⁄6 chance of rolling a six on a (fair) die. This opinion shouldn’t change when you update on others’ opinions. If you used Upco, that’s a 1:5 ratio, giving final odds of 1:25 - clearly incorrect. On the other hand, a geometric mean approach gives sqrt((1*1)/(5*5))=1:5, as it should be.
You can also pool more than two credences together, or weight others’ opinions differently using a geometric mean. For example, if I thought there was a 1:1 chance of an event, but someone whose opinion I trusted twice as much as myself put it at 8:1 odds, then I would do the following calculation, in which we effectively proceed as if we have three opinions (mine and another person’s twice), hence we use a cube root (effectively one root per person):
cube root((1*8*8)/(1*1*1)=4
So we update our opinion to 4:1 odds.
This also solves the problem of Upco not being able to update on an opinion of 50⁄50 odds (which I think is a problem—sometimes 1:1 is definitely the right credence to have (e.g. a fair coin flip)). If we wanted to combine 1:1 odds and 1:100 000 odds, it should land in the order of somewhere in between. Upco stays at ((1*1)/(1*100 000))=100 000 (i.e. 1:100 000), which is not updating at all using the 1:1. Whereas the geometric mean gives sqrt((1*1/1*100 000)=1:100. 1:100 is a far more reasonable update from the two of them.
In terms of your question 3, where people have credences of 0 or 1, I think some things we could do are to push them to use a ratio instead (like asking if rather than certain, they’d say 1 in a billion), weight their opinions down a lot of they seem far too close to 0 or 1, or just discount them if they insist on 0 or 1 (which is basically epistemically inconceivable/incorrect from a Bayesian approach).
You might be right that the geometric mean of odds performs better than Ucpo as an updating rule although I’m still unsure exactly how you would implement it. If you used the geometric mean of odds as an updating rule for a first person and you learn the credence of another person, would you then change the weight (in the exponent) you gave the first peer to 1⁄2 and sort of update as though you had just learnt the first and second persons’ credence? That seems pretty cumbersome as you’d have to keep track of the number of credences you already updated on for each proposition in order to assign the correct weight to a new credence. Even if using the geometric mean of odds(GMO) was a better approximation of the ideal bayesian response than Upco (which to me is an open question), it thus seems like Upco is practically feasible and GMO is not.
Here’s one huge problem with the Upco method as you present it: two people think that it’s a 1⁄6 chance of rolling a six on a (fair) die. This opinion shouldn’t change when you update on others’ opinions. If you used Upco, that’s a 1:5 ratio, giving final odds of 1:25 - clearly incorrect. On the other hand, a geometric mean approach gives sqrt((1*1)/(5*5))=1:5, as it should be.
If one person reports credence 1⁄6 in rolling a six on a fair die and this is part of a partition containing one proposition for each possible outcome {”Die lands on 1“, Die lands on 2”, … “Die lands on 6”}, then the version of upco that can deal with complex partitions like this will tell you not to update on this credence (see section on “arbitrary partitions”). I think the problem you mention just occurs because you are using a partition that collapses all five other possible outcomes into one proposition—“the die will not land on 6”.
This case does highlight that the output of Upco depends on the partition you assume someone to report their credence over. But since GMO as an updating rule just differs from Upco in assigning a weight of 1/n to each persons credence (where n is the number of credence already learned), I’m pretty sure you can find the same partition dependence with GMO.
Hey it’s cool that this got written up. I’ve only read your summary, not the paper, and I’m not an expert. However, I think I’d argue against using this approach, and instead in favour of another. It seems equivalent to using ratios to me, and I think you should use the geometric mean of odds instead when pooling ideas together (as you mentioned/asked about in your question 7.). The reason is that Upco uses each opinion as a new piece of independent evidence. Geometric mean of odds merely takes the (correct) average of opinions (in a Bayesian sense).
Here’s one huge problem with the Upco method as you present it: two people think that it’s a 1⁄6 chance of rolling a six on a (fair) die. This opinion shouldn’t change when you update on others’ opinions. If you used Upco, that’s a 1:5 ratio, giving final odds of 1:25 - clearly incorrect. On the other hand, a geometric mean approach gives sqrt((1*1)/(5*5))=1:5, as it should be.
You can also pool more than two credences together, or weight others’ opinions differently using a geometric mean. For example, if I thought there was a 1:1 chance of an event, but someone whose opinion I trusted twice as much as myself put it at 8:1 odds, then I would do the following calculation, in which we effectively proceed as if we have three opinions (mine and another person’s twice), hence we use a cube root (effectively one root per person):
cube root((1*8*8)/(1*1*1)=4
So we update our opinion to 4:1 odds.
This also solves the problem of Upco not being able to update on an opinion of 50⁄50 odds (which I think is a problem—sometimes 1:1 is definitely the right credence to have (e.g. a fair coin flip)). If we wanted to combine 1:1 odds and 1:100 000 odds, it should land in the order of somewhere in between. Upco stays at ((1*1)/(1*100 000))=100 000 (i.e. 1:100 000), which is not updating at all using the 1:1. Whereas the geometric mean gives sqrt((1*1/1*100 000)=1:100. 1:100 is a far more reasonable update from the two of them.
In terms of your question 3, where people have credences of 0 or 1, I think some things we could do are to push them to use a ratio instead (like asking if rather than certain, they’d say 1 in a billion), weight their opinions down a lot of they seem far too close to 0 or 1, or just discount them if they insist on 0 or 1 (which is basically epistemically inconceivable/incorrect from a Bayesian approach).
Hey Daniel,
thanks for engaging with this! :)
You might be right that the geometric mean of odds performs better than Ucpo as an updating rule although I’m still unsure exactly how you would implement it. If you used the geometric mean of odds as an updating rule for a first person and you learn the credence of another person, would you then change the weight (in the exponent) you gave the first peer to 1⁄2 and sort of update as though you had just learnt the first and second persons’ credence? That seems pretty cumbersome as you’d have to keep track of the number of credences you already updated on for each proposition in order to assign the correct weight to a new credence. Even if using the geometric mean of odds(GMO) was a better approximation of the ideal bayesian response than Upco (which to me is an open question), it thus seems like Upco is practically feasible and GMO is not.
If one person reports credence 1⁄6 in rolling a six on a fair die and this is part of a partition containing one proposition for each possible outcome {”Die lands on 1“, Die lands on 2”, … “Die lands on 6”}, then the version of upco that can deal with complex partitions like this will tell you not to update on this credence (see section on “arbitrary partitions”). I think the problem you mention just occurs because you are using a partition that collapses all five other possible outcomes into one proposition—“the die will not land on 6”.
This case does highlight that the output of Upco depends on the partition you assume someone to report their credence over. But since GMO as an updating rule just differs from Upco in assigning a weight of 1/n to each persons credence (where n is the number of credence already learned), I’m pretty sure you can find the same partition dependence with GMO.