Data scientist working on AI governance at MIRI, previously forecasting at Epoch and the Stanford AI Index. GWWC pledge member since 2017. Formerly social chair at Harvard Effective Altruism, facilitator for Arete Fellowship, and founder of the DC Slate Star Codex meetup.
Robi Rahmanđ¸
Karma: 1,580
âmanagersâ time is also scarce and high opportunity costâ doesnât seem like enough in my opinion to warrant âmillions will just sit in an account for another yearâ
Yeah, if the problem is scarcity of manager time then @lukeprog should be hiring more managers so the managers can manage more grantmakers.
comparing cluelessness world preferences with non-cluelessness world preferences
To decide whether you should act based on your cluelessness world preferences or your non-cluelessness world preferences, youâd have to use evidence (either empirical or theoretical) for which world youâre in.
But an ordinary consequentialist decision-making process would already take this evidence into account. (I think cluelessness-believers sometimes strawman cluelessness-skeptics and assume that they havenât thought of this or wouldnât update on such evidence.)
I think a more interesting crux is that cluelessness advocates believe that cluelessness undermines consequentialism while leaving deontology or virtue ethics relatively unscathed, but it seems to me that all the cluelessness arguments against consequentialism also undermine other decision-making systems as well.
I disagree that youâre justified in doing that
Even if itâs true that itâs unjustified, thatâs not relevant. If the decision-making process canât justify any actions, you can do whatever you want. Might as well act as if it can and do whatever I would have done anyway. (It would be less justified, but⌠shrug)
Iâd be interested in reading the case for it
Conveniently, itâs not necessary to justify it and try to increase anyoneâs credence that itâs true, because the alternative doesnât affect what decision they should make, so they should behave as if itâs true regardless of the probability, anywhere from 0 to 100%.
Yes, I understand that youâre saying thatâs a problem that you canât assign a probability-weighted expectation. Iâm pointing out that if you canât do that, then cluelessness doesnât support any claims that any other action (besides the highest-EV action based on the premise that you can assign a non-arbitrary probability-weighted expectation) is higher-EV, so you should do the previously-chosen highest-EV action anyway. Therefore cluelessness is irrelevant.
But itâs not answered there. If youâre pointing at the thing about how UEV isnât a point estimate but a range, thatâs still irrelevant: even if true, the preferred action is still decided by the scalar value that comes from taking your probability-weighted expectation of that range.
Are you assuming that the previously preferred action would still have some normative force behind it?
Yes.
If cluelessness is false, then you have some preferred action.
If cluelessness is true, then your preferred action is unjustified.
Thereâs some level of credence in cluelessness. The worst case for your otherwise-preferred action is when p(cluelessness)=100%. In that case it doesnât matter what you do.
So you might as well do whatever you were going to do anyway.
Thereâs no case where cluelessness is relevant to your decision process.
Cluelessness makes the belief that the preferred action has good consequences weaker, but it doesnât raise any other action above the otherwise-preferred action in terms of expected consequences. So itâs irrelevant to decision-making, and you should still do whatever you were going to do anyway.
Let me know if you have any arguments indicating otherwise. I was hoping this post would provide some.
It doesnât follow from âwe donât know the net direction of the consequences weâre unaware ofâ that we should regard the positives and negatives as precisely symmetric. One reason symmetry is implausible: If we become aware of a new possible consequence, this should update our beliefs about the others weâre unaware of, breaking the symmetry.
Thatâs great news. Youâre saying that we have information about which direction our estimates are biased? Then cluelessness is false! We should update our estimates in the direction indicated by the information we become aware of, making a new, more accurate estimate, and act upon that.
But thatâs false. If cluelessness is true, then you canât tell what you should do in that case. But you still know what you should do in the case where cluelessness is false (at least, as well as you did before hearing the cluelessness argument) so you should do whatever you were going to do anyway. So the justified action is still the same as before, but with lower confidence.
This woulnât be the case if youâre 100% certain that cluelessness is true, but if cluelessness is true then itâs not justified to be 100% confident that cluelessness is true.
Iâm out of the loop, was Ridglan Farms doing scientific research? Or just breeding beagles to sell as pets?
Is there any reason why cluelessness is decision-relevant? Even if itâs true, why shouldnât you just do the same thing you should do if itâs not true?
First, the unawareness argument doesnât imply that ânothing we do mattersâ all things considered. It only implies that impartial altruism, or any very far-reaching value system, isnât action-guiding. Other values and moral norms still matter to us, for example, rules like avoiding dishonesty or virtues like compassion. These can be action-guiding even if weâre clueless about total consequences.
No, the first sentence in this paragraph is wrong.
If cluelessness is correct then you also canât justifiably believe that rules like avoiding dishonesty or virtues like compassion matter, or are good to embody. For all you know, telling the truth in your personal life will cause there to be more dishonesty everywhere else in the universe, so you ought to lie instead.
But thatâs obviously ridiculous, for the same reason that the cluelessness argument against impartial altruism is ridiculous.
To use an analogy from accounting: recruiting and fundraising are operating expenditure. You need to make the investment every year to get the same returns. Community building is more like capital expenditureâyouâre investing in assets from which you expect future value to flow.
Surely recruiting is a capital expenditure in this framework?
itâs full of dumb criticisms and bitter, jealous people
Really? Itâs full of this stuff? Iâve been reading it for 5+ years and barely seen this. Can someone who agrees with this provide examples?
The point of UU is to have certain differences from traditional religion. UU encourages questioning traditional religion, but it discourages questioning of UUâs own positions.
EA encourages questioning of EAâs principles. I could get hundreds of upvotes by writing a post about why ITN is a bad framework and should be ditched, whereas youâd get tarred and feathered for suggesting that a UU congregation should drop one of their seven principles.
shrug thatâs fine, I donât mind the downvotes, but can we also enforce epistemic standards along with niceness? The above comment is refuting a strawman while not engaging with the rate of increase of AI capabilities which are the crux of the post.
And when a strong candidate turns down a CG offer, the result is often not âa slightly-less-good grantmaker,â itâs just one fewer grantmaker. We routinely close rounds with fewer hires than weâd planned for.
Why? Shouldnât you make an offer to the runner-up?
Ultimately, it is better to debate the merits of specific interventions than general vibes.
Huh? Werenât you quitting a job to start an org to fix the vibes?
This is a very impressively daft comment.
Revenue and benchmarks and razor blades aside, letâs look at the object level. Have you used a frontier model recently? How smart were they three years ago and how smart are they now?
I hope Iâm not strawmanning cluelessness advocates, but they seem to constantly make this basic error. I havenât seen this point accounted for anywhere but if it is, perhaps you can point me to it.
...no, itâs not? The EV of kicking the puppy is still a small negative amount, after incorporating all of the cluelessness-possibilities described above.
If the expected long-run effects precisely cancel out, then you shouldnât kick the puppy, same as you already wouldnât before accounting for highly uncertain long-term effects.
If the expected long-run effects donât precisely cancel out, then you incorporate them into the EV calculation and act based on that.
Cluelessness doesnât change the EV of kicking the puppy from â5 to 0. It changes the EV of the puppy from â5 to â5 Âą 1000000. But it doesnât move puppy-kicking up or down in the rank-ordering of possible actions you could take. (You could argue for risk aversion and say this favors neartermist work, but thatâs a very different claim from the radical cluelessness stuff.)
Do cluelessness advocates have a response to this?