I am a rising junior at the University of Chicago (co-president of UChicago EA and founder of Rationality Group). I am mostly interested in philosophy (particularly metaethics, formal epistemology, and decision theory), economics, and entrepreneurship.
I also have a Substack where I post about philosophy (ethics, epistemology, EA, and other stuff). Find it here: https://substack.com/@irrationalitycommunity?utm_source=user-menu.
Reach out to me via email @ dnbirnbaum@uchicago.edu
Yea, unclear if these self-reports will be reliable, but I agree that this could be true (and I briefly mention something like it: “Broadly, AW has high tractability, enormous current scale, and stronger evidence of sentience—at least for now, since future experiments or engineering relevant to digital minds could change this.”
I think having new places to make more hits based bets (within orgs or creating new orgs) is going to be really important.
My sense is that we have somewhat of a grantmaker shortage now (though correct me if I’m wrong), and this problem will become worse.
I guess that I am just not very confident that this is a good proxy.
The allocation is probably not what it should be, but I’m not yet convinced that animal stuff is being underfunded.
This seems to assume that EA funds ought to be distributed “democratically” by people who identify as EAs or EA leaders. I don’t buy that.
If the goal is good resource allocation, we want funding decisions to track competence about cause prioritization, not representativeness. Randomly sampled EAs—many of whom haven’t spent much time thinking seriously about cause prioritization—don’t seem like the right decision-makers.
And it’s also not obvious that “EA leaders,” as such, are the right allocators either. The relevant property isn’t status or identity, but judgment, track record, and depth of thinking about cause prioritization.
I think one can reasonably ask this question of consciousness/welfare more broadly: how does one have access to their consciousness/welfare?
One idea is that many philosophers think one, by definition, has immediate epistemic access to their conscious experiences (though whether those show up in reports is a different question, which I try to address in the piece). I think there are some phenomenological reasons to think this.
Another idea is that we have at least one instance where one supposedly has access to their conscious experiences (humans), and it seems like this shows up in behavior in various ways. While I agree with you that our uncertainty grows as you get farther from humans (i.e. to digital minds), I still think you’re going to get some weight from there.
Finally, I think that, if one takes your point too far (there is no reason to trust that one has epistemic access to their conscious states), then we can’t be sure that we are conscious, which I think can be seen as a reductio (at least, to the boldest of these claims).
Though let me know if something I said doesn’t make sense/if I’m misinterpreting you.
I would love to hear any updated takes on this post from Ryan.
I agree this is a super hard problem, but I do think there are somewhat clear steps to be made towards progress (i.e. making self reports more reliable). I am biased, but I did write this piece on a topic that touches on this problem a bit that I think is worth checking out.
Vasco, you are the king of cross-posting good content onto the EA forum.
Yea, I agree with this a lot.
One thing to warn, though: one can be inclined to say something like “I’m an actor, so I don’t have to do any of the accuracy-based forecasting, etc,” which I think is definitely wrong.
One should choose which direction they act based on some somewhat accuracy/EV + vibes. Otherwise, there are infinite ways to act and most of them, without any foresight, fail even with the type of agency you’re describing.
In addition, once you are in the project you are acting in, you (ofc) shouldn’t constantly be doing the accuracy/EV thing. Sometimes, though, it’s probably worth taking a step back and doing it to consider opportunity cost (on some marked date every x amount of time to avoid decision/acting paralysis).
Dwarkesh (of the famed podcast) recently posted a call for new guest scouts. Given how influential his podcast is likely to be in shaping discourse around transformative AI (among other important things), this seems worth flagging and applying for (at least, for students or early career researchers in bio, AI, history, econ, math, physics, AI that have a few extra hours a week).
The role is remote, pays ~$100/hour, and expects ~5–10 hours/week. He’s looking for people who are deeply plugged into a field (e.g. grad students, postdocs, or practitioners) with high taste. Beyond scouting guests, the role also involves helping assemble curricula so he can rapidly get up to speed before interviews.
More details are in the blog post; link to apply (due Jan 23 at 11:59pm PST).
Hmm this is interesting.
Speculatively, I think there could actually just be convergence here, though, once you account for moral uncertainty and just very plausible situations where doing bad by everyone’s lights are as bad as, say, utilitarian nightmares but just easier to get others on board for (ie extreme power).
Yea, sorry about that. That really sucks.
Oh, sorry. I asked my friend, and they said it was Stop AI—not Pause AI. They basically protested in the middle of a talk with the CEA CEO saying things like he is a murderer, which I just think is pretty nuts. That is my bad, though. I have edited that part out of my initial comment.
I think I disagree with a bunch of what you said there: I don’t think good AIS involves being on good side of AI labs necessarily (tho I think there are good arguments for this), I think large movement building without getting a full pause would be a big win for PauseAI and that many EAs would agree with this (despite the fact that I and maybe them know little about social movements), and I do think making simple statements reflects negatively from a whole host of perspectives and should be taken pretty seriously.
I’d be interested in hearing how you/others at pause AI have tracked how much of this marginal improvements to advocacy you have been doing so far is. What are the wins? What were the costs? Happy to also get on a call with someone about this.
If there are real numbers on this (I know it’s hard in spaces like yours), I’d be curious about hearing why they aren’t often posted on the forum/LW, as that, I think, would be more helpful to people (and tracking the cost-effectiveness of AIS in general is super underrated, so this would be good).
If you don’t have the numbers, I would ask why you are so confident that this is actually the right approach.
Can you say more about what these EA assumptions are?
Disagreement is cool and awesome. Even intense disagreement (“I think this view is deeply misguided”). I really see no room for antagonism among two people that could be having an epistemically healthy conversation.
Sorry about your father.
I think there’s a much more mundane and much more epistemically healthy way to understand this disagreement.
Perhaps this is naive but my view currently just is: most people are disagreeing about a few concrete empirical and strategic parameters: the likely effectiveness and public reception of a Pause movement, how much meaningful safety work can be done from inside labs, and (probably) estimates of p(doom). Given how uncertain and high-stakes these questions are, it seems completely unsurprising that reasonable people would land in very different places.
It’s fine to worry about incentives and institutional bias — that could matter — but treating this as if the disagreement is obviously resolved, or as if it cleanly divides the world into “accelerationists” and “pause-ers,” strikes me as bad epistemics.
Thanks for the comment and good points.
What I meant is that they can be MORE politically charged/mainstream/subject to motivated reasoning. I definitely agree that current incentives around AI don’t perfectly track good moral reasoning.
Yep, I agree (though I’m not sure if I agree that the incentive is clearly in the negation; one could argue that a company may want to say that they are worried about sentience to increase hype the same way some argue that talking about the risks of AI increases hype). I just think there will be more when the issue is in the minds of the public.
I think there are some mainstream things about digital minds (Black Mirror comes to mind), but I don’t think it’s a thing that people yet take seriously in the real world.