I am a sophomore at the University of Chicago (co-president of UChicago EA and founder of Rationality Group). I am mostly interested in philosophy (particularly metaethics, formal epistemology, and decision theory), economics, and entrepreneurship.
I also have a Substack where I post about philosophy (ethics, epistemology, EA, and other stuff). Find it here: https://substack.com/@irrationalitycommunity?utm_source=user-menu.
From a philosophy standpoint, I find incommensurability pretty implausible (at least to act upon) for a couple reasons:
If two values are incommensurable, for every action that you take, there is some probability that you are making a trade-off between these actions. Given some version of expected value to be correct (where some probability of value is equivalent to trading off some lower value itself), this would mean that every action one takes is making a choice between two incommensurable goods. This seems to lock you into a constant state of decision paralysis (where every action you take is trading off two incommensurable goods), which, I believe, should just make incommensurable goods a non-viable option. (See this paper for more discussion)
Imagine you have some credence in two things being incommensurable (thereby making it such that you have no reasons to act in either way). Even if this is the case, however, you should still have some non-zero credence in these values / actions being commensurable as it is a contingent proposition. If the credence of incommensurability gives you no reason to act and the credence of commensurability does give you reason to act, this makes incommensurability irrelevant, making it so that your actions should entirely be informed by the case conditional on commensurability.
Happy to chat more about this, if you think that you’d find that helpful.
We’ve talked about this, but I wanted to include my two counterarguments as a comment to this post:
It seems like there’s a good likelihood that we have semi-mathusian constraints nowadays. While I would admit that one should be skeptical of total malthusianism (ie for every person dying another one lives because we are at max carrying capacity), I think it is much more reasonable to think that carrying constraints actually do exist and maybe its something like for every death you get .2 lives or something. If this is true, I think this argument weakens a bunch.
This argument only works if, conditional on existential risk not happening, we don’t hit malthusian constraints at any point in the future, which seems quite implausible. If we don’t get existential risk and the pie just keeps growing, it seems like we would just get super-abundance and the only thing holding people back would be malthusian physical constraints on creating happy people. Therefore, we just need some people to live past that time of super-abundance to have massive growth. Additionally, even if you think those people wouldn’t have kids (which I find pretty implausible—as one person’s preference for children would lead to many kids given abundance), you could talk about those lives being extremely happy which holds most of the weight. This also
Side note: this argument seems to rely on some ideas about astronomical waste that I won’t discuss here (I also haven’t done so much thinking on the topic), but it seems maybe worth it to frame around that debate.
Didn’t know that — thanks for sharing.
I think this is going to be hard for university organizers (as an organizer at UChicago EA).
At the end of our fellowship, we always ask the participants to take some time to sign up for 1-1 career advice with 80k, and this past quarter myself and other organizers agreed that we felt somewhat uncomfortable doing this given that we knew that 80k was leaning a lot on AI—as we presented it as merely being very good for getting advice on all types of EA careers. This shift will probably make it so that we stop sending intro fellows to 80k for advice, and we will have to start outsourcing professional career advising to somewhere else (not sure where this will be yet).
Given this, I wanted to know if 80k (or anyone else) has any recommendations on what EA University Organizers in a similar position should do (aside from the linked resources like Probably Good).
Really insightful post. Thanks!
Note: I’m really unsure what I believe about the following comment, but I’m interested in hearing what others have to say about it.
Whenever we add an additional condition of the type of thing we want (say, diversity), we sacrifice some amount of the terminal aim (getting the best people). While there are good reasons to care about diversity (optics, founder effects, making people feel more comfortable), there are also ones that are more controversial (for instance—in some cases like grant-making, diversity of sex or race as a proxy for getting a “more diverse outlook” on a particular subject). Let’s call optics/ founder effects instrumental diversity and more diverse outlook diversity. Given this framing, I think two points are important:
Note: I understand that this framing is weird because the kind of diversity of knowledge/ experience is said to be good instrumentally—i still wanted to make a different conceptual category for it because 1) it’s more controversial and 2) some conditions might apply to it that may not apply to other constraints.
Some argue that diversity is a powerful meme and will be hard to resist once you take some of its premises—this type of thing seems particularly apt for value drift. Perhaps this means that EAs should be more hesitant to take on some of the diversity (as opposed to instrumental diversity) points into decision making when hiring and such.
Conditional that someone decides to give some weight to diversity, I think it should be made more clear that this is a diversity point rather than an instrumental diversity point, as the former is more controversial.
I’m interested in hearing what others have to say about this—especially if you think this comment overrates the amount that EAs care about diversity (vs instrumental diversity). I’m also interested in hearing if you think I’m underestimating the reasons for why diversity might be important that I might be missing.
I think the Encompass link is expired.
Good point. Will change this when it’s not midnight. Thanks!
Thanks for the nice comment. Yea, I think this was more of “laying out the option space.”
All very interesting points!
Enjoyed this article a lot, and I think the framing of the “root problem objection” is an underrated one!
Thanks for the response.
The part that I’m still stuck on is that this last part about the implicit tradeoff in one’s offset seems crucial. The degree of offsetting is entirely based on tradeoff (maybe with some risk aversion under diff moral theories), but if you put that much into offsetting than it seems like you either have a major moral or epistemic disagreement with those that are donating in the first place. If that is the case, one person has got to give (either they don’t offset near this much or they don’t donate to AMF).
While I’m here, I also wanted to thank you for writing this post. Super interesting, thoughtful, and I’ve shared with a bunch of people already!
Perhaps I’m misunderstanding something, so please correct me if I’m wrong:
If one accepts all these assumptions, why would the best course of action be to offset AMF donations rather than to avoid donating to AMF in the first place?
If ITNs cause vastly more harm to mosquitoes than they help humans, wouldn’t this imply that AMF is not just a weak investment, but actually a net-negative intervention? It seems like these numbers, if taken seriously, suggest AMF should be deprioritized rather than merely balanced with shrimp welfare donations.
I assume that this is mostly about hedging against uncertainty under diff moral theories, but it seems like making this tradeoff of offset compared to counterfactual more money to AMF implies a certain tradeoff that you’re okay with such that you should never make the initial investment.
I’m confused about what sorta epistemic/ moral uncertainty theory someone would need to be offsetting the way you propose. Tbh I’ve already confused myself with this comment, but I hope it’s helpful(?)
Appreciate the response! This is very helpful so thanks.
Very interesting points. Here are a few other things to think about:
1. I think there are very few people whose primary motivation is helping others, so we shouldn’t empirically expect them to be doing the most good because they represent a very small portion of the population. This is especially true if you think (which I do) that the vast majority of people who do good are 1) (consciously or unconsciously) signaling for social status or 2) not doing good very effectively (the people who are are a much smaller subgroup because doing non-effective good is easy). It would be very surprising, however, if those who try to do good effectively aren’t doing much better than those who aren’t, as individuals, on average, but it seems unlikely to me (though feel free to throw some stats that will change my mind!).
2. I’m very skeptical that “the defensibility of morality as the pursuit of greatness depends on how sophisticated our cultural conceptions of greatness are.” Could you give more reason for why you think this?
3. I’m skeptical that 1) searching for equanimity is truly the best thing and 2) that we have good and tractable methods of achieving it. Perhaps people would be better off as being more Buddhist on the margin, but, to me, it seems like (thoughtfully!) getting the heavy positive tail end results and be really careful and thoughtful about negatives leads to a much better off society.
Let me know what you think!
Yep, I think this is true. The point is that, given AI stays aligned which is stated there, the best thing for a country to do would be to accelerate capabilities. You’re right, however, that its not an argument against AI being an existential threat (I’ll make a note to make this more clear) — it’s more a point for acceleration.
Thank you for doing this — this is super helpful from a university organizer perspective.
One question: Will you be able to handle the capacity of all the participants (UChicago, for example, is around 13 per quarter, excluding summer) after a university intro fellowship?