It does, thanks—at least, we’re clarifying where the disagreements are.

If you think that choosing a set of probability functions was arbitrary, then having a meta-probability distribution over your probability distributions seems even more arbitrary, unless I’m missing something. It doesn’t seem to me like the kind of situations where going meta helps: intuitively, if someone is very unsure about what prior to use in the first place, they should also probably be unsure about coming up with a second-order probability distribution over their set of priors .

All you need to do to come up with that meta-probability distribution is to have some information about the relative value of each item in your set of probability functions. If our conclusion for a particular dilemma turns on a disagreement between virtue ethics, utilitarian ethics, and deontological ethics, this is a difficult problem that people will disagree strongly on. But can you even agree that these each bound, say, to be between 1% and 99% likely to be the correct moral theory? If so, you have a slightly informative prior and there is a possibility you can make progress. If we really have completely no idea, then I agree, the situation really is entirely clueless. But I think with extended consideration, many reasonable people might be able to come to an agreement.

Upon immediately encountering the above problem, my brain is like the mug: just another object that does not have an expected value for the act of giving to Malaria Consortium. Nor is there any reason to think that an expected value must “really be there”, deep down, lurking in my subconscious.

I agree with this. If the question is, “can anyone, at any moment in time, give a sensible probability distribution for any question”, then I agree the answer is “no”.

But with some time, I think you can assign a sensible probability distribution to many difficult-to-estimate things that are not completely arbitrary nor completely uninformative. So, specifically, while I can’t tell you right now about the expected long-run value for giving to Malaria Consortium, I think I might be able to spend a year or so understanding the relationship between giving to Malaria Consortium and long-run aggregate sentient happiness, and that might help me to come up with a reasonable estimate of the distribution of values.

We’d still be left with a case where, very counterintuitively, the actual act of saving lives is mostly only incidental to the real value of giving to Malaria Consortium, but it seems to me we can probably find a value estimate.

About this, Greaves (2016) says,

averting child deaths has longer-run effects on population size: both because the children in question will (statistically) themselves go on to have children, and because a reduction in the child mortality rate has systematic, although difficult to estimate, effects on the near-future fertility rate. Assuming for the sake of argument that the net effect of averting child deaths is to increase population size, the arguments concerning whether this is a positive, neutral or a negative thing are complex.

And I wholeheartedly agree, but it doesn’t follow from the fact you can’t immediately form an opinion about it that you can’t, with much research, make an informed estimate that has better than an entirely indeterminate or undefined value.

EDIT: I haven’t heard Greaves’ most recent podcast on the topic, so I’ll check that out and see if I can make any progress there.

EDIT 2: I read the transcript to the podcast that you suggested, and I don’t think it really changes my confidence that estimating a Bayesian joint probability distribution could get you past cluelessness.

So you can easily imagine that getting just a little bit of extra information would massively change your credences. And there, it might be that here’s why we feel so uncomfortable with making what feels like a high-stakes decision on the basis of really non-robust credences, is because what we really want to do is some third thing that wasn’t given to us on the menu of options. We want to do more thinking or more research first, and then decide the first-order question afterwards.

Hilary Greaves: So that’s a line of thought that was investigated by Amanda Askell in a piece that she wrote on cluelessness. I think that’s a pretty plausible hypothesis too. I do feel like it doesn’t really… It’s not really going to make the problem go away because it feels like for some of the subject matters we’re talking about, even given all the evidence gathering I could do in my lifetime, it’s patently obvious that the situation is not going to be resolved.

My reaction to that (beyond I should read Askell’s piece) is that I disagree with Greaves that a lifetime of research could resolve the subject matter for something like giving to Malaria Consortium. I think it’s quite possible one could make enough progress to arrive at an informative probability distribution. And perhaps it only says “across the probability distribution, there’s a 52% likelihood that giving to x charity is good and a 48% probability that it’s bad”, but actually, if the expected value is pretty high, it’s still strong impetus to give to x charity.

I still reach the problem where we’ve arrived at a framework where our choices for short-term interventions are probably going to be dominated by their long-run effects, and that’s extremely counterintuitive, but at least I have some indication.

> Hope this helps.

It does, thanks—at least, we’re clarifying where the disagreements are.

All you need to do to come up with that meta-probability distribution is to have

someinformation about the relative value of each item in your set of probability functions. If our conclusion for a particular dilemma turns on a disagreement between virtue ethics, utilitarian ethics, and deontological ethics, this is a difficult problem that people will disagree strongly on. But can you even agree that these each bound, say, to be between 1% and 99% likely to be the correct moral theory? If so, you have a slightly informative prior and there is a possibility you can make progress. If we really have completely no idea, then I agree, the situation really is entirely clueless. But I think with extended consideration, many reasonable people might be able to come to an agreement.I agree with this. If the question is, “can anyone, at any moment in time, give a sensible probability distribution for any question”, then I agree the answer is “no”.

But with some time, I think you can assign a sensible probability distribution to many difficult-to-estimate things that are not completely arbitrary nor completely uninformative. So, specifically, while I can’t tell you right now about the expected long-run value for giving to Malaria Consortium, I think I might be able to spend a year or so understanding the relationship between giving to Malaria Consortium and long-run aggregate sentient happiness, and that might help me to come up with a reasonable estimate of the distribution of values.

We’d still be left with a case where, very counterintuitively, the actual act of saving lives is mostly only incidental to the real value of giving to Malaria Consortium, but it seems to me we can probably find a value estimate.

About this, Greaves (2016) says,

And I wholeheartedly agree, but it doesn’t follow from the fact you can’t immediately form an opinion about it that you can’t, with much research, make an informed estimate that has better than an entirely indeterminate or undefined value.

EDIT: I haven’t heard Greaves’ most recent podcast on the topic, so I’ll check that out and see if I can make any progress there.

EDIT 2: I read the transcript to the podcast that you suggested, and I don’t think it really changes my confidence that estimating a Bayesian joint probability distribution could get you past cluelessness.

My reaction to that (beyond

I should read Askell’s piece) is that I disagree with Greaves that a lifetime of research could resolve the subject matter for something like giving to Malaria Consortium. I think it’s quite possible one could make enough progress to arrive at an informative probability distribution. And perhaps it only says “across the probability distribution, there’s a 52% likelihood that giving to x charity is good and a 48% probability that it’s bad”, but actually, if the expected value is pretty high, it’s still strong impetus to give to x charity.I

stillreach the problem where we’ve arrived at a framework where our choices for short-term interventions are probably going to be dominated by their long-run effects, and that’s extremely counterintuitive, but at least I have some indication.