Lazily pursuing earn-to-give. Very excited about AI Safety, GHD, and the weirder parts of animal welfare.
dan.pandori šø
Karma: 840
Sure, if you can directly measure what you actually want (positive impact), that is of course best :)
Thereās lots of talk about how more EA-aligned funding will attract grifters. The Robin Hanson in me wondered what expensive or hard-to-fake signals exist for (human) alignment.
The first ideas that came to mind:
donating a kidney
vegan/āvegetarian
non-profit work
donating a lot to effective charities
community engagement
I donāt know whether folks should weight these highly in funding decisions. Probably they all collapse under sufficient pressure.
I feel like Iāve got a moral obligation to have true beliefs insofar as that leads to me having a positive impact.
If I had lower agency or power, I would whole heartedly endorse being ignorant & happy. Folks can of course choose for themselves here.
Thanks for the thoughtful reply Anthony.
-
Fair point that the sequences do not strongly default towards inaction. I think the evidence gains from action mean that it āisā favored, but this is an empirical prediction about the world, which I think you disagree with.
-
If increased information reduces our unawareness, that decreases the relevance of cluelessness arguments, no? In particular, the set of actions with overlapping credal sets would be reduced, and so there would be more dominated actions we could rule out for consideration. Unless youāre making an argument that the information learned is too small to meaningfully constrain our predictions in expectation. In which case sure, thatās an empirical claim I disagree with, but it is logically consistent.
-
I think most importantly we have an empirical disagreement about both how clueless we are of the longterm impacts of our actions, and how difficult it is to reduce that cluelessness. Iām trying to think about cruxes that might resolve that.
I think you disagree that increased predictive capability in shorter time horizons generalizes to long-time horizons [1]. For me, seeing bad short & medium term predictions āwouldā cause me to give more credence to unawareness arguments. But you seem to have prevented the opposite evidence from swaying you in the other direction (unless Iām misunderstanding).
Like, what is something you could see happen tomorrow that would make you say āwhoa, we actually can know the long-term consequences of our actionsā? Would it take something as extreme as https://āāwww.lesswrong.com/āāposts/āā6FmqiAgS8h4EJm86s/āāhow-to-convince-me-that-2-2-3?
-
If you want to give it a read. IMO funnier than the Anthropic parody (although some parts miss), less funny than Open Asteroid Impact.
OOC, have you asked Fable to try to write a satire with āOpen Asteroid Impactā as a reference class? I had it do one for ALLFED, and IMO it was pretty funny. Much funnier than āMETR Time Horizon 2.0ā (although Iām not sure how hard this post was optimizing for humor), pretty comparable to Open Asteroid Impact.
If youāve tried this yourself with Fable and still feel it comes up short, that would be a little surprising to me.
Happy to share the example if youād like, IMO it had a good degree of joke substructure. IE, strongly funny individual lines, good references, relevant puns.
Itās also very possible that the humor is of a variety that quickly saturates & youāre already saturated on it. Similar to how most folks find āCards Against Humanityā pretty funny the first time you play it, and then it goes downhill extremely sharply.
āThe graph lists Microsoft Excel 1.0, but the benchmarks must have been run on a modern version of Excel. The worksheet limit of Excel 1.0 was only 16,384 rows rather than 1,048,576, plus it lacked many of the functions of modern Excel, so it would presumably fail more tasks.ā
This is beautiful. Thank you.
Personally I thought this was a B- shitpost. If it was just a quick take of exactly the tldr text I would give it a B+. So IMO not bad, but I think I also read less LLM text than you so Iām probably not annoyed as quickly by it.
Thanks for putting this together! IMO it is very helpful, and encouraged me to look at IFPās proposal program.
Do you see cluelessness to be decreasing, steady, or increasing?
As in, are we getting better are predicting longterm outcomes or do you think we are no better than centuries ago?
One vote for decreasing cluelessness: improved epistemic practices such as RCTs, peer review, more advanced statistics, and prediction markets. There used to be many actions which we were deeply unsure about the near-term consequences of (ex. is bleeding this patient a good idea), and now our best predictions seem justifiably more confident. The evidence for improved long-term predictions seems less obvious, but still directionally improving.
Thanks! And very much my bad for missing the instructions, apologies!
Ah thanks! Presumably I can just submit this via the form then, or am I SOL?
I think making predictions (and learning how they go) for the short & medium term future will help us be much more calibrated on our predictions for the long-term future. So all the work on prediction markets etc probably give us better insight there.
Iām not sold on some sort of Great Reflection style pause, but it also doesnāt seem like a crazy idea.
I frankly have not thought hard about how acausal influence changes how we should act. I donāt have good recommendations. Being kind & having a wide circle of empathy feels pretty robustly good, but I wonāt pretend to have justified that from the lens of acausal negotiations.
Yeah thatās a totally fair take. I generally agree that reading AI writing triggers an āoh brotherā feeling for me these days.
I asked Fable to be more terse & uploaded the edited essay. I personally find this triggers less of my āoh brotherā feelings, but it might still be annoying to you.
I am highly conflicted about posting an entirely AI generated post like this. I decided to post as-is because:
IMO the basic argument (taking actions is how you learn about the world) is correct. It is worth posting for that alone.
I could change up the prose & re-write to hide the AI generated nature, but for what purpose? Fable wrote this, not me. Changing the prose feels intellectually dishonest, even if the AI-isms sometimes bother me.
I read many of the Longtermist essay submissions, and IMO this is better than many. So I donāt think Iām meaningfully diluting the quality of the submission pool.
Its explicitly allowed by the contest rules, so YOLO.
Iām a software engineer, and the norms around AI generated works have rapidly moved much more favorably to them. IE, AI code generally follows local stylistic norms, tests itself, and is locally correct. IMO this generalizes to prose.
Iād be very curious to hear from folks who believe that this submission is lower than average quality, and have specific reasons why (other than just that it was AI generated).
The LearnĀing Trap: What SiĀmuĀlated ClueĀless Agents ReĀveal About the UnawareĀness Argument
[6] If strange situations result in me saving money having more net-expect-impact this year than donating, Iām allowed to do so. Didnāt expect this to come up. Less than a 1% increase in savings, I wonāt beat myself up about it.
OP addresses this in the āWhy Cluelessness Mattersā section.
I agree my claim for tighter error estimates is very weak.
I could say that looking at the estimate you get by aggregating many different folksā together reduces variance (assuming you believe the estimates have some amount of uncorrelated signal). Individual estimates are noisy, but aggregate estimates are less noisy. This is basically the point discussed in the āWhy do different groups have the same rankingsā section of OPās post.
But frankly Iām largely making a vibes claim (ie, model gives silly results ā model probably wrong).
I admire the hustle and hope you succeed.
But as many others have said, this is a recipe for burnout. If a friend said they intended to do this, I would try quite hard to talk them down.