Data scientist working on AI governance at MIRI, previously forecasting at Epoch and the Stanford AI Index. GWWC pledge member since 2017. Formerly social chair at Harvard Effective Altruism, facilitator for Arete Fellowship, and founder of the DC Slate Star Codex meetup.
Robi Rahmanđ¸
Karma: 1,581
Resilience as in reducing x-risk by improving our chances of surviving a catastrophe, as opposed to reducing x-risk by decreasing the chances of catastrophes occurring.
I have multiple project proposals that could use funding. Is there a way to submit another application, besides making another account?
consider this story. Youâre a fairly thoughtful longtermist who one day comes across the idea of cluelessness. Youâre skeptical, but you find yourself grudgingly agreeing that predictions about the far future are mostly made-up. Claims like âthis intervention will increase expected total welfare across the cosmosâ start to seem fake. Still, surely some actions have better or worse consequences than others? Soon enough you wonder if that thought is fake too. You realize that even, say, kicking a puppy could prevent human extinction. Maybe this act of pointless cruelty sparks moral outrage in some onlooker, galvanizing them to work a bit harder at their AI safety job. Yes, this possibility is clearly far-fetched. And you can easily imagine stories pushing in the opposite direction. But you canât come up with a good argument that the expected long-run effects point one way or the other, or that they precisely cancel out. (That is, your cluelessness is âcomplexâ, not âsimpleâ.[1]) The sign of the EV of kicking the puppy seems to be: shrug. You walk away feeling like this reasoning is too clever by half, even if you canât say where exactly it went wrong. So you decide to file cluelessness under âyeah I get the arguments, but this is a bridge too farâ. And you go back to thinking about how to solve AI alignment.
More generally, hereâs what this tale illustrates. As argued in Mogensen (2020) and this sequence, impartial consequentialists[2] canât say whether any intervention has higher or lower expected value than inaction.
I hope Iâm not strawmanning cluelessness advocates, but they seem to constantly make this basic error. I havenât seen this point accounted for anywhere but if it is, perhaps you can point me to it.
The sign of the EV of kicking the puppy seems to be: shrug.
...no, itâs not? The EV of kicking the puppy is still a small negative amount, after incorporating all of the cluelessness-possibilities described above.
If the expected long-run effects precisely cancel out, then you shouldnât kick the puppy, same as you already wouldnât before accounting for highly uncertain long-term effects.
If the expected long-run effects donât precisely cancel out, then you incorporate them into the EV calculation and act based on that.
Cluelessness doesnât change the EV of kicking the puppy from â5 to 0. It changes the EV of the puppy from â5 to â5 Âą 1000000. But it doesnât move puppy-kicking up or down in the rank-ordering of possible actions you could take. (You could argue for risk aversion and say this favors neartermist work, but thatâs a very different claim from the radical cluelessness stuff.)
Do cluelessness advocates have a response to this?
âmanagersâ time is also scarce and high opportunity costâ doesnât seem like enough in my opinion to warrant âmillions will just sit in an account for another yearâ
Yeah, if the problem is scarcity of manager time then @lukeprog should be hiring more managers so the managers can manage more grantmakers.
comparing cluelessness world preferences with non-cluelessness world preferences
To decide whether you should act based on your cluelessness world preferences or your non-cluelessness world preferences, youâd have to use evidence (either empirical or theoretical) for which world youâre in.
But an ordinary consequentialist decision-making process would already take this evidence into account. (I think cluelessness-believers sometimes strawman cluelessness-skeptics and assume that they havenât thought of this or wouldnât update on such evidence.)
I think a more interesting crux is that cluelessness advocates believe that cluelessness undermines consequentialism while leaving deontology or virtue ethics relatively unscathed, but it seems to me that all the cluelessness arguments against consequentialism also undermine other decision-making systems as well.
I disagree that youâre justified in doing that
Even if itâs true that itâs unjustified, thatâs not relevant. If the decision-making process canât justify any actions, you can do whatever you want. Might as well act as if it can and do whatever I would have done anyway. (It would be less justified, but⌠shrug)
Iâd be interested in reading the case for it
Conveniently, itâs not necessary to justify it and try to increase anyoneâs credence that itâs true, because the alternative doesnât affect what decision they should make, so they should behave as if itâs true regardless of the probability, anywhere from 0 to 100%.
Yes, I understand that youâre saying thatâs a problem that you canât assign a probability-weighted expectation. Iâm pointing out that if you canât do that, then cluelessness doesnât support any claims that any other action (besides the highest-EV action based on the premise that you can assign a non-arbitrary probability-weighted expectation) is higher-EV, so you should do the previously-chosen highest-EV action anyway. Therefore cluelessness is irrelevant.
But itâs not answered there. If youâre pointing at the thing about how UEV isnât a point estimate but a range, thatâs still irrelevant: even if true, the preferred action is still decided by the scalar value that comes from taking your probability-weighted expectation of that range.
Are you assuming that the previously preferred action would still have some normative force behind it?
Yes.
If cluelessness is false, then you have some preferred action.
If cluelessness is true, then your preferred action is unjustified.
Thereâs some level of credence in cluelessness. The worst case for your otherwise-preferred action is when p(cluelessness)=100%. In that case it doesnât matter what you do.
So you might as well do whatever you were going to do anyway.
Thereâs no case where cluelessness is relevant to your decision process.
Cluelessness makes the belief that the preferred action has good consequences weaker, but it doesnât raise any other action above the otherwise-preferred action in terms of expected consequences. So itâs irrelevant to decision-making, and you should still do whatever you were going to do anyway.
Let me know if you have any arguments indicating otherwise. I was hoping this post would provide some.
It doesnât follow from âwe donât know the net direction of the consequences weâre unaware ofâ that we should regard the positives and negatives as precisely symmetric. One reason symmetry is implausible: If we become aware of a new possible consequence, this should update our beliefs about the others weâre unaware of, breaking the symmetry.
Thatâs great news. Youâre saying that we have information about which direction our estimates are biased? Then cluelessness is false! We should update our estimates in the direction indicated by the information we become aware of, making a new, more accurate estimate, and act upon that.
But thatâs false. If cluelessness is true, then you canât tell what you should do in that case. But you still know what you should do in the case where cluelessness is false (at least, as well as you did before hearing the cluelessness argument) so you should do whatever you were going to do anyway. So the justified action is still the same as before, but with lower confidence.
This woulnât be the case if youâre 100% certain that cluelessness is true, but if cluelessness is true then itâs not justified to be 100% confident that cluelessness is true.
Iâm out of the loop, was Ridglan Farms doing scientific research? Or just breeding beagles to sell as pets?
Is there any reason why cluelessness is decision-relevant? Even if itâs true, why shouldnât you just do the same thing you should do if itâs not true?
First, the unawareness argument doesnât imply that ânothing we do mattersâ all things considered. It only implies that impartial altruism, or any very far-reaching value system, isnât action-guiding. Other values and moral norms still matter to us, for example, rules like avoiding dishonesty or virtues like compassion. These can be action-guiding even if weâre clueless about total consequences.
No, the first sentence in this paragraph is wrong.
If cluelessness is correct then you also canât justifiably believe that rules like avoiding dishonesty or virtues like compassion matter, or are good to embody. For all you know, telling the truth in your personal life will cause there to be more dishonesty everywhere else in the universe, so you ought to lie instead.
But thatâs obviously ridiculous, for the same reason that the cluelessness argument against impartial altruism is ridiculous.
To use an analogy from accounting: recruiting and fundraising are operating expenditure. You need to make the investment every year to get the same returns. Community building is more like capital expenditureâyouâre investing in assets from which you expect future value to flow.
Surely recruiting is a capital expenditure in this framework?
itâs full of dumb criticisms and bitter, jealous people
Really? Itâs full of this stuff? Iâve been reading it for 5+ years and barely seen this. Can someone who agrees with this provide examples?
The point of UU is to have certain differences from traditional religion. UU encourages questioning traditional religion, but it discourages questioning of UUâs own positions.
EA encourages questioning of EAâs principles. I could get hundreds of upvotes by writing a post about why ITN is a bad framework and should be ditched, whereas youâd get tarred and feathered for suggesting that a UU congregation should drop one of their seven principles.
shrug thatâs fine, I donât mind the downvotes, but can we also enforce epistemic standards along with niceness? The above comment is refuting a strawman while not engaging with the rate of increase of AI capabilities which are the crux of the post.
Thereâs a field where applicants can paste previous applications as text, but can you add a way to upload attachments?