CEO of Rethink Priorities
Marcus_A_Davis
Karma: 2,074
Hey Vasco,
We agree having more low moral weight options for animals could be valuable but the Donor Compass is a simplification. We made some tradeoffs for simplification that reduced complexity given feedback from initial testers but may need to recalibrate if current users don’t find this the right tradeoff. Generally, if you want to do something more complex on animal weights (or just given the fact you have detailed opinions about the moral weights of animals), the quiz version of the Donor Compass probably isn’t the right tool and you should probably use the advanced mode.
Hey Nick,
I’m headed off to two weeks of conferencing in in a few hours so very likely won’t respond further after this but I do want to say a few things.
I take very seriously that you (or anyone else) believes we’ve come off as defensive or not open to changing our mind. I definitely think we haven’t always lived up to our communication ideals and could stand do better. I’m really sorry if you don’t think we aren’t open-minded but I don’t think this is true:
When criticism comes in, the response is polite well reasoned refutation—very good arguments for maintaining status-quo. I don’t think I’ve seen a response along the lines of “hey you have a good point there, let’s look into that” or even “yes that decision was tricky and we did X because...”.
I don’t have the time to do a throughout search and systematic weighing but I believe this comment section contains at least one example to the contrary. I think just a few comments down someone raises questions about what went into the AI discount and I stated why it’s shaky, what changing it does to the output, and how I hope to see it improve.
You’ve said that “personel is policy” carries weight, but haven’t suggested how we should approach examining that? My Claude ranking method isn’t the best for sure, what other way you would suggest? From an RP organisational perspective, I think there is a risk that the org’s work could be undermined somewhat by people who might feel like a bunch of animal welfare/GCR folks might have disproportionate sway over a cross-cause prioritisation process like this.
I’m not really sure what the best method is as a I said I think doing this type of counting is a very blunt, limited tool. From my experience, the worry about our staff and outcome bias basically cuts all directions as some GCR people consider us too animal friendly, some animal people consider us too GCR and GHD friendly, and some GHD people (like yourself) consider us too animal and GCR friendly. I have personally heard all of these concerns over the past several months while working on this project. This doesn’t invalidate any of the concerns (we really could be biased one of these ways, not all concerns are equally valid) but I think it can bring some perspective to it. I think one inherent limitation to this enterprise is we typically get staff who are willing to work on multiple areas and often have explicitly done so prior to joining RP. But that typically means few if any are perceived from the outside as a true “member” of any cause area.
But I think an additional reason it’s not super useful to assess our staff this way and why I still lean away from the utility of type of exercise, is the actual projects RP does are reflective most of all of what projects we think are valuable. WIT in particular, has done roughly four large projects and those projects were about moral weight, risk and uncertainty in giving, quantitative tools for cross cause prioritization, and digital minds. But the reason we selected those projects—which I believe are then counted as part of what cause areas they’ve focused on in the Claude search—is because we thought there was a useful contribution to make that we could do well. To put it mildly, I think there’s far less big intellectual gaps in the GHD space than in AW and GCR space, or in cause prioritization itself. So, that we filled in gaps outside of GHD that we think are needed to improve cause prioritization may be being used as evidence we don’t care about GHD.
Outside of general competence, here’s what I actually prioritize on staff working on cause prio projects (not necessarily in this order):
Willingness and ability to be truthful to the arguments and the facts
Willingness to seek out and engage alternative views, and get expert feedback
Philosophical/empirical knowledge and breadth of experience working on the specific types of problems at hand
Of these I think (3) is partially visible from the outside, and under certain circumstances (2). (1) can be seen in what choices and arguments are actually advanced. But none of this maps very well onto “cause area” background, and largely I think different types of methodological background and experience have more to do with the output that area inclination. (I also don’t think cause areas are the relevant level of crux, cause areas contain bundles of interventions that are extremely heterogenous but that’s a story for another day).
Perhaps within RP there’s much agreement but I would argue there will be a heavy “groupthink” element built there over time. That there are 10+ moral theories in this model illustrates the extreme diversity.
I actually want more. I think we didn’t capture everything I think is relevant about moral theories and hope to do better on this in the future. We had to start somewhere though.
I agree it would be great to have other groups working on this stuff, a poll a while ago was overwhelmingly in favour. The reality is though, you guys are it for now. There aren’t other groups working on moral weights and cross-cause prioritisation. I think this means that there’s perhaps more responsibility for balance within the organisation. The situation is more “Aristotle” dominating philosophy than Friedman and Stiglitz. I think with our current levels of information, thre is far more shared ground in economics than in cross-cause prioritisatoin.
I get that we are largely it for now and take this responsibility very seriously. Suffice it to say this project has dominated my thinking for months, and my desire to do the best we can with our initial offering has been my primary concern for several months. Hopefully despite your concerns, future editions of this project and others can do more to win your trust and live up to what we need to be be if we’re going to be the only game in cause prioritization town.
Hey Nick, I don’t know if we are that far apart on the conceptual issue of potential bias but I think we are approaching this differently.
However, I really like the Stieglitz and Friedman analogy and think it is useful in many ways. Simply, there are lots of decision points here. And I really wish there were some other groups working on building such work so we could compare and contrast our work with theirs like you can in that context. If that were so, I think this type of meta-debate would be either less likely to occur, or more productive because we could point to more specific things.
At the same time, I think “count up the cause area backgrounds of the staff who worked on this” is misleading even if I agreed with the characterization of our staff, and I don’t. This is for a number of reasons and, if you’d permit, I’ll largely extend the economics and social science analogy to raise my points.
First, in some simple sense, I think looking at our staff’s background like this is like looking at the subfields GiveWell’s staff worked in prior to joining and if a disproportionate number worked on malaria using that to argue this is evidence of potential bias for why multiple of their top donation opportunities overall involve malaria. It’s not that this bias is not possible, it’s just a very blunt guide, at best, and the causal arrow may point the other way. That is, GiveWell may employ a lot of people who have backgrounds in malaria to help create their final estimates because they are particularly well suited to the task and/or malaria is an important area they have to cover.
Second, for this model many of the relevant choice points come from fields that have less ideological valence on the cause area lines (i.e. what do you think of risk attitudes) and potential bias in those areas is likely just as relevant to reaching conclusions as thinking at the level of cause area. Further, some inputs have effects difficult to pin down in the overall model, which limits the ability of people to even implicitly put their thumb on the scale because they don’t know what changing that input would change in the result. There are a few areas where it is obvious (i.e. make the animal moral weights higher or lower, reduce the cost-effectiveness of a given area) but those are the areas precisely where anyone looking at our model can see what we choose and object if they disagree.
But suppose you were convinced that Stiglitz tends to be biased in a liberal direction, and Friedman in a conservative one. What’s the best way to demonstrate that? I think it would often be to point to a specific assumption or choice they made that is questionable. This is why I asked you to point to something specific that you think is wrong. Not because I think it can’t be the case that we’re biased or that it’s inherently illegitimate to bring up the possibility, but because it’s the specifics that demonstrate that we are biased. I really do agree that too often some people in EA think everything they do is objective. I wrote this just last month and stand behind this as applying to basically everyone:
As Keynes observed about a parallel dynamic in economics: “practical men, who believe themselves to be quite exempt from any intellectual influences, are usually the slaves of some defunct economist.” The same applies here. Views that people describe as mere intuitions or common sense are often downstream of philosophical ideas promoted in their environments. When someone thinks such intuitions and implicit stances are indisputable or obviously correct, it’s more often than not evidence they haven’t examined or tested those ideas at all.
If someone doesn’t have the time to investigate an area, it can be at times reasonable to check for this kind of bias and discount appropriately (I think this is most useful as a guide when a group or individual has shown repeated bias in the past) but I also think this can at times serve as a shortcut to dismissing anything that doesn’t already ideologically align with yourself. Balancing these two competing forces can be a constant struggle, particularly in areas outside of your expertise and I know it’s something constantly in my mind when I read about politics.
Finally, to continue the analogy with economics, often the field of experts are debating a much more narrow range of opinions than exists in the public debate. For example, in immigration the economist debate about the impact of immigration on wages and employment is far more narrow than the general public discussion of the same topic. Stiglitz and Friedman (assuming you mean Milton Friedman) might have disagreed a lot but it was often within a largely shared framework. I’m pretty sure neither of them thought mass central planning would work, or that tariffs were largely paid by the exporting countries. I think many of the opinions people give about cause prioritization may fall into this category of something very few who think about the topic carefully dare to defend but, because those outside haven’t put in the time, they are unaware of this (in some sense through no fault of their own, not everyone can be an expert or very informed about every topic).
I think this type of distinction is likely to hold among serious efforts to build cross cause models.
Hi Nick,
I think there is not a single EA organization I would consider unbiased on this question, including ourselves (despite our ongoing efforts not to be). That is exactly why we publish so much of our methodology and our assumptions openly. One of the main motivations for this work is concern about the effect of bias when assumptions and models are implicit or hidden. We would welcome more experts with broader backgrounds being involved in drafting and improving these estimates, which is part of what we hope this kind of public methodology enables.
On conflicts of interest: we aim to be transparent about potential or perceived conflicts of interest, and we state all present COIs in the post above and on the website (here and here), and we will update these when and if they change. RP is recommending these grants independently and no one outside of RP was involved in choosing the funds for inclusion.* You are welcome to review our methodology on how the funds were chosen and provide critique.
On team composition and priorities specifically, a few points: First, the researchers you list are not, in fact, the principal contributors to the project and it’s harder to pin us down even if you stuck to that methodology. For example, Carmen van Schoubroeck was the project lead for all this work, and started her EA career as a global health and development researcher. Some of the specific inputs that in theory could be biased here (like the choice of aggregation methods) are mine and my background involves starting a global health charity, being an animal welfare fund manager, and incubating and providing operational support to a large number of AI projects. And obviously RP works across all of these areas because I, personally, think it’s a good idea. Secondly, Claude does not have access to the work our researchers have done that’s not published, nor what they actually prioritize in their own giving. Third, our global health and development researchers gave feedback on the modeling assumptions, in the same way we involved researchers with other expertise on other cause areas. Having that option is one of the benefits of being a research organization with teams across causes.
More broadly, I would not consider Claude’s opinions about the priorities of our researchers to be a good method of gauging the quality of our work. I think all of the following:
The old adage that “personnel is policy” is relevant to any exercise that’s as complex as this one
I’m confident that there are ways to improve this work, and specific improvements to be gained by bringing in diverse perspectives (I’d be happy to discuss options here in more detail)
Despite (1) and (2) there are larger areas of improvement or concerns about this work than trying to infer cause area orientation from public work of staff and use that to infer potential bias in our choices for the model
Specifically with regard to (1), I can never tell you with certainty no one who worked on this was not subtly unconsciously biased, but one of the reasons people were chosen to work on this project at all is because of my perception of their lack of bias. This is a considerable filter to working on any cross cause work at RP and I stand behind the credibility of every person who worked on this was doing their best to be unbiased. I think this filtering combined with the process we had of having multiple people review work and getting input from funds and cause area experts should lead to less bias. I could be wrong, of course, but I think it would be more productive to make specific critiques of our work and choices because the specific choices are there to see. It’s not costless in time to do this, so I don’t begrudge anyone for not engaging, but you don’t have to infer if you think our animal moral weights are too high, or the cost-effectiveness of an area is too low, you can see what we chose and say what you think is wrong and why.
I truly wish for myself and RP to do the impartial good but we only ever achieve doing so imperfectly. But, primarily, the way we find out is through specific things we get wrong. So, if you have specific issues with inputs or choices that you think privileged one area over others, please point them out. We’d be happy to revise them if they should be improved. The current model surely isn’t perfect (as we acknowledge) and can benefit from specific, thoughtful criticism about our methodology, which we’ve described and linked to on this site.
*Despite the complete lack of involvement of former staff in our decisionmaking, in general I favor more transparency in EA so ill note you are correct that RP has done research on lead exposure before, and in fact, two former RP staff now work on the Lead Exposure Action Fund. I think our ability to model lead was improved because we’d done work on it and this was a pro for us including it, as we felt more confident we could accurately do so by the time of launch. I was also a fund manager for the EA Animal Welfare Fund. We also have former staff at Longview, and some at other parts of Coefficient Giving unrelated to LEAF. We also have former staff or board members at a number of the funds we list that we plan to consider including Astralis, the AI Safety Tactical Opportunities Fund, Giving Green, and Macroscopic Ventures.
How was the 40% figure calculated?
This is definitely an approximation, not based on a rigorous underlying model. I informally personally asked several people in the cause prio and AI spaces, including some at labs, to answer the question about what type of discount seemed appropriate and, along with my best judgment, settled on 40%.
I would very much like to improve this input in the future, perhaps through formal surveys or, perhaps, a Delphi panel-type process. Though I think my weakly held intuition here is a bigger possible change isn’t the precise value (which in the current model doesn’t change the result dramatically), but distinguishing between reduced effectiveness overall and probability of an effect going decreasing, with consideration of different time periods when those effects happen in each cause area.
Fascinating. Is biorisk depreciating because some of those projects have a long payoff time?
My intuition is that biorisk talent will be more easily redeployable in a world undergoing an AI transition than a lot of the animal talent (more sticky).
We included biorisk in the AI discount not just because of payoff times but because of the same type of uncertainty you raised about AW. That is, it seems possible the actions people are taking today could be mooted by changes in AI development.
It does seem possible that biorisk talent could potentially be more easily redeployable than AW. I would also add a couple more related additional considerations in what discount to use for biorisk: (i) some pathways to very negative outcomes for AI run through biorisk and (ii) it seems plausible to me that some civilizational hardening measures (i.e. more widely available PPE) seem perhaps more robust to AI uncertainty than some interventions in other cause areas because they act on AI risk itself given (i).
This is why in future versions of this model, I could imagine both nukes and biorisk having a different AI discount since there are clear interactions here between AI and these other GCRs.
Hi Chris,
The way it practically works is a blanket cut on the overall cost-effectiveness. That could be because there is an x-risk, or because the effect of non-AI work is reduced or eliminated. For the current inputs, a 40% discount is applied, which I think is large enough to account for all those possibilities.
But perhaps the phrase “regular animal welfare” is potentially another source of disagreement? The funds we’re donating to are aware of AI’s impact on the world and seem likely to take steps to use AI to improve their outcomes. The current allocation is based on what specific interventions the AW funds are funding but it’s also worth explicitly noting that giving money to AW funds or AW groups isn’t a commitment to pursue the same interventions that exist today indefinitely (this same reasoning also applies to GHD funds). Perhaps current interventions become moot (that’s why we are discounting them) but it’s also possible, for example, that AW work could be more effective in the future because AI makes monitoring welfare vastly cheaper or, say, boosts the efficiency of animal groups.
All that said, if you think there should be a higher discount, you can apply it, but that doesn’t substantially change the results regarding what goes to AW. Indeed, if you increase the discount to 90%, you don’t get much more to AI (and less overall to GCRs as a whole, likely because the discount also applies to biorisk and nuclear weapons). Generally, this is because there are interaction effects between risk preferences, diminishing returns, overall cost-effectiveness, and how the worldviews are aggregated. Making everything else less cost-effective doesn’t change, say, that you have to have a certain risk attitude to take longer-shot bets at success.
That understanding of our AI teams is roughly correct.
The primary reason for IAPS spinning out is IAPS and RP thought they could be more impactful if they spun out. This was for a variety of reasons including, among other things, different operational support needs.
In terms of why start a new team, there is still plenty of other work on AI that should be done that isn’t DC-centered AI policy in the way that IAPS is primarily engaged in. RP may have a comparative advantage in other areas, especially around incubating new groups, and exploring new research frontiers related to navigating transformative AI.
Rethink Priorities: 2025 Results, 2026 Plans and Funding Needs
Hey Lukas,
Thanks for the detailed reply. You raise a number of different interesting points and I’m not going to touch on all of them, given a lack of time but there are a few I want to highlight.
the discussion under “An outside view on having strong views” would benefit from discussing how much normative ethics is analogous to science and how much it is anologous to something more like personal career choice (which weaves together personal interests but still has objective components where research can be done—see also my post on life goals).
While I can see how you might make this claim, I don’t really think ethics is very analogous to personal career choice. Analogies are always limited (more on this later) but I think this analogy probably implies too much “personal fit” in career choice which are often a lot about “well, what do you like to do?” so much as they are “this is what will happen if you do that?”. I think you’re largely making the case more for the former, with some part of the latter and for morality I might push for a different combination, even assuming a version of anti-realism. But perhaps all this breaks down on what you think of career choice, where I don’t have particularly strong takes.
I think anti-realism is obviously true, but I don’t mean the “anything goes” type of anti-realism, so I’m not unsympathetic to your overall takeaway.
Still, even though I agree with your response to the “anything goes” type of anti-realism, I think you’d ideally want to engage more with metaethical uncertainty and how moral reflection works if (the more structure-containing) moral anti-realism is true.
You’re right I haven’t engaged here about what normative uncertainty means in that circumstance but I think, practically, it may look a lot like the type of bargaining and aggregation referenced in this post (and outlined elsewhere), just with a different reason for why people are engaged in that behavior. In one case, it’s largely because that’s how we’d come to the right answer but in other cases it would be because there’s no right answer to the matter and the only way to resolve disputes is through aggregating opinions across different people and belief systems.
That said, I believe–correct me if I’m wrong–your posts are arguing for a particularly narrow version of realism that is more constrained than typical and that there’s a tension between moral realism and moral uncertainty.
Stepping back a bit, I think a big thrust of my post is that you generally shouldn’t make statements like “anti-realism is obviously true” because the nature of evidence for that claim is pretty weak, even if the nature of the arguments for you reaching that conclusion were clear and are internally compelling to you. You’ve defined moral realism narrowly so perhaps this is neither here nor there but, as you may be aware, most English-speaking philosophers accept/lean towards moral realism despite you noting in this comment that many EAs who have been influential have been anti-realists (broadly defined). This isn’t compelling evidence, but it is evidence against the claim that anti-realism is “obviously correct” since you are at least implicitly claiming most philosophers are wrong about this issue.
What this means is that moral uncertainty almost by necessity (there’s a trivial exception where your confidence in moral realism is based on updating to someone’s else’s expertise but they have not yet told you the true object-level morality that they believe in) implies either metaethical uncertainty (uncertainty between moral realism and moral anti-realism) or confident moral anti-realism.
I’ve read your post on moral uncertainty and moral realism being in tension (and the first post where you defined moral realism) and I’m not sold on the responses you provide to your challenge. Take this section:
Still, I think the notion of “forever inaccessible moral facts” is incomprehensible, not just pointless. Perhaps(?) we can meaningfully talk about “unreachable facts of unknown nature,” but it seems strange to speak of unreachable facts of some known nature (such as “moral” nature). By claiming that a fact is of some known nature, aren’t we (implicitly) saying that we know of a way to tell why that fact belongs to the category? If so, this means that the fact is knowable, at least in theory, since it belongs to a category of facts whose truth-making properties we understand. If some fact were truly “forever unknowable,” it seems like it would have to be a fact of a nature we don’t understand. Whatever those forever unknowable facts may be, they couldn’t have anything to do with concepts we already understand, such as our “moral concepts” of the form (e.g.,) “Torturing innocent children is wrong.”
I could retort here that it seems totally reasonable to argue that there’s a fact of the matter about what caused the Big Bang or how life on Earth began. What caused these could conceivably be totally inaccessible to us now but still related to known facts. Nothing about not knowing how these things started commits us to say–what I take to be the equivalent in this context–that the true nature of those situations has nothing to do with concepts we understand like biology or physics. Further, given what we know now in these domains, I think it’s fair to rule out a wide range of potential causes of them and constrain things to a reasonable set of targets that it may have caused them.
The analogy here seems reasonable enough with morality to me that you shouldn’t rule this type of response out.
Similarly, you say the following to branch two of possible responses to your claim:
To summarize, the issue with self-evident moral statements like “Torturing innocent children is wrong” is that they don’t provide any evidence for a moral reality that covers disagreements in population ethics or accounts of well-being. To be confident moral realists, we’d need other ways of attaining moral knowledge and ascertaining the parts of the moral reality beyond self-evident statements. In other words, we can’t be confident moral realists about a far-reaching, non-trivial, not-immediately-self-evident moral reality unless we already have a clear sense of what it looks like.
I don’t fully buy this argument for similar reasons to the above. This seems more like an argument that to be confident moral realists who assert correct answers to most/all the important questions we need strong evidence of moral realism in most/all domains than it is an argument that we can’t be moral realists at all. One way I might take this (not saying you’d agree) would be to say you think moral realism that isn’t action guiding on the contentious points isn’t moral realism worth the name because all the value of the name is in the contentious points (and this may be particularly true in EA). But if that phrasing of the problem is acceptable, then we may be basically only arguing about the definition of “moral realism” and not anything practically relevant. Or, one could say we can’t be confident moral realists given the uncertainty about what morality entails in a great many cases and I might retort “we don’t need to be confident in order to choose among the plausible options so long as we can whittle things down to restricted set of choices and everything isn’t up for grabs.” This would be for basically the same reasons a huge number of potential options aren’t relevant for settling on the correct theory of abiogenesis or taking the right scientific actions given the set of plausible theories.
But perhaps a broader issue is I, unlike many other effective altruists, am actually cool with (in your words) “minimalist moral realism” being fine and using aggregation methods like those mentioned above to come to final takes about what to do given the uncertainty. This is quite different from confidently stating “the correct answer is this precise version of utilitarianism, and here’s what it says we need to do…”. I don’t think what I’m comfortable saying obviously qualifies as an insignificant moral realism relative to such a utilitarian even if the reasons for reaching the suggested actions differed.
But stepping back, this back and forth looks like another example of the move I criticized above because you are making some analogies and arguing some conclusion follows from those analogies, I’m denying those analogies, and therefore denying the conclusion, and making different analogies. Neither of us has the kind of definitive evidence on their side that prevails in science domains here.
So, how confident am I that you’re wrong? Not super confident. If the version of moral anti-realism you say is true and it results in something like your life-goals framework as the best way to decide ethical matters, then so be it. But the question is what to do given uncertainty that this is the correct approach, and that assuming it’s the correct approach we know what it recommends differs from how we’d otherwise behave. I don’t think it’s clear to me meta-ethical uncertainty about realism or anti-realism is a highly relevant factor in deciding what to do unless, again, someone is embracing a “anything goes” kind of anti-realism which neither of us are endorsing.
We should be more uncertain about cause prioritization based on philosophical arguments
Rethink Priorities’ funding needs
Hey Saulius,
I’m very sorry that you felt that way – that wasn’t our intention. We aren’t going to get into the details of your resignation in public, but as you mention in your follow up comment, neither this incident, nor our disagreement over WAW views were the reason for your resignation.
As you recall, you did publish your views on wild animal welfare publicly. Because RP leadership was not convinced by the reasoning in your piece, we rejected your request to publish it under the RP byline as an RP article representative of an RP position. This decision was based on the work itself; OP was not at all a factor involved in this decision. Moreover, we made no attempt to censor your views or prevent them from being shared (indeed I personally encouraged you to publish the piece if you wanted).
To add some additional context without getting into the details of this specific scenario, we can share some general principles about how we approach donor engagement.
We have ~40 researchers working across a variety of areas. Many of them have views about what we should do and what research should be done. By no means do we expect our staff to publicly or privately agree with the views of leadership, let alone with our donors. Still, we have a donor engagement policy outlining how we like to handle communication with donors.
One relevant dimension is that we think that if one of our researchers, especially while representing RP, is sending something to a funder that has a plausible implication that one of the main funders of a department should seriously reduce or stop funding that department, we should know they are planning to do so before they do so, and roughly what is being said so that we can be prepared. While we don’t want to be seen as censoring our researchers, we do think it’s important to approach these sorts of things with clarity and tact.
There are also times when we think it is important for RP to speak with a unified voice to our most important donors and represent a broader, coordinated consensus on what we think. Or, if minority views of one of our researchers that RP leadership disagrees with are to be considered, this needs to be properly contextualized and coordinated so that we can interact with our donors with full knowledge of what is being shared with them (for example, we don’t want to accidentally convey that the view of a single member of staff represents RP’s overall position).
With regard to cause prioritization, funders don’t filter or factor into our views in any way. They haven’t been involved in any way with setting what we do or don’t say in our cause prioritization work. Further, as far as I’m aware, OP hasn’t adopted the kind of approach we’ve suggested on any of our major cause prioritization on moral weight or as seen in the CURVE sequence.
We mean to say that the ideas for these projects and the vast majority of the funding were ours, including the moral weight work. To be clear, these projects were the result of our own initiative. They wouldn’t have gone ahead when they did without us insisting on their value.
For example, after our initial work on invertebrate sentience and moral weight in 2018-2020, in 2021 OP funded $315K to support this work. In 2023 they also funded $15K for the open access book rights to a forthcoming book based on the topic. In that period of 2021-2023, for public-facing work we spent another ~$603K on moral weight work with that money coming from individuals and RP’s unrestricted funding.
Similarly, the CURVE sequence of WIT this year was our idea and we are on track to spend ~$900K against ~$210K funded by Open Phil on WIT. Of that $210K the first $152K was on projects related to Open Phil’s internal prioritization and not the public work of the CURVE sequence. The other $58K went towards the development of the CCM. So overall less than 10% of our costs for public WIT work this year was covered by OP (and no other institutional donors were covering it either).
Hey Vasco, thanks for the thoughtful reply.
I do find fanaticism problematic at a theoretical level since it suggests spending all your time and resources on quixotic quests. I would go one further and say I think if you have a series of axioms and it proposes something like fanaticism, this should at least potentially count against that combination of axioms. That said, I definitely think, as Hayden Wilkinson pointed out in his In Defence of Fanaticism paper, there are many weaknesses with alternatives to EV.
Also, the idea that fanaticism doesn’t come up in practice doesn’t seem quite right to me. On one level, yeah, I’ve not been approached by a wizard asking for my wallet and do not expect to be. But I’m also not actually likely going to be approached by anyone threatening to money-pump me (and even if I were I could reject the series of bets) and this is often held as a weakness to EV alternatives or certain sets of beliefs. On another level, in some sense to the extent I think we can say fanatical claims don’t come up in practice it is because we’ve already decided it’s not worth pursuing them and discount the possibility, including the possibility of going looking for actions that would be fanatical.* Within the logic of EV, even if you thought there weren’t any ways to get the fanatical result with ~99% certainty, it would seem you’d need to be ~100% certain to fully shut the door on at least expending resources seeing if it’s possible you could get the fanatical option. To the extent we don’t go around doing that I think it’s largely because we are practically rounding down those fanatical possibilities to 0 without consideration (to be clear, I think this is the right approach).
All the other problems attributed to expected utility maximisaton only show up if one postulates the possibility of unbounded or infite value, which I do not think makes sense
I don’t think this is true. As I said in response to Michael St. Jules in the comments, EV maximization (and EV with rounding down unless it’s modified here too) also argues for a kind of edge-case fanaticism, where provided a high enough EV if successful you are obligated to take an action that’s 50.000001% positive in expectation even if the downside is similarly massive.
It’s really not clear to me the rational thing to do is consistently bet on actions that would impact a lot of possible lives but, say ~0.0001% chance of making a difference and are net positive in expectation but have a ~49.999999% chance of causing lots of harm. This seems like a problem even within a finite and bounded utility function for pure EV.
I am confused about why RP is still planning to invest significant resources in global health and development… Maybe a significant fraction of RP’s team believes non-hedonic benefits to be a major factor?
I’ve not polled internally but I don’t think non-hedonic benefits issue is a driving force inside RP. Speaking for myself, I do think hedonism is makes up for at least more than half of what makes things valuable at least in part for the reasons outlined in that post.
The reasons we work across areas in general are because of differences in the amount of money in the areas, the number of influenceable actors, the non-fungibility of the resources in the spaces (both money and talent), and moral and decision-theoretic uncertainty.
In this particular comparison case of GHD and AW, there’s hundreds of millions more of plausibly influenceable dollars in the GHD space than in the AW space. For example, GiveWell obviously isn’t going to shift their resources to animal welfare, but they still move a lot of money and could do so more effectively in certain cases. GiveWell alone is likely larger than all of the farm animal welfare spending in the world by non-governmental actors combined, and that includes a large number of animal actors I think it’s not plausible to affect with research. Further, I think most people who work in most spaces aren’t “cause neutral” and, for example, the counterfactual of all our GHD researchers isn’t being paid by RP to do AW research that influences even a fraction of the money they could influence in GHD.
Additionally, you highlight that AW looks more cost-effective than GHD but you did not note that AMF looked pretty robustly positive across different decision theories and this was not true, say, of any of the x-risk interventions we considered in the series and some of the animal interventions. So, one additional reason to do GHD work is the robustness of the value proposition.
Ultimately, though, I’m still unsure about what the right overall approach is to these types of trade-offs and I hope further work from WIT can help clarify how best to make these tradeoffs between areas.
*A different approach is to resist this conclusion is to assert a kind of claim that you must drop your probability in claims of astronomical value, and that this always balances out increases in claims of value such that it’s never rational within EV to act on these claims. I’m not certain this is wrong but, like with other approaches to this issue, within the logic of EV it seems you need to be at ~100% certainty this is correct to not pursue fanatical claims anyway. You could say in reply the rules of EV reasoning don’t apply to claims about how you should reason about EV itself, and maybe that’s right and true. But these sure seem like patches on a theory with weaknesses, not clear truths anyone is compelled to accept at the pain of being irrational. Kludges and patches on theories are fine enough. It’s just not clear to me this possible move is superior to, say, just biting that you need to do rounding down to avoid this type of outcome.
Thanks for the engagement, Michael.
I largely agree with your notes and caveats.
However, on this:
Expected utility maximization can be guaranteed to avoid fanaticism while satisfying the standard EUT axioms (and countable extensions), with a bounded utility function and the bounds small enough or marginal returns decreasing fast enough, in relative terms… In my view, expected utility with a bounded utility function (not difference-making) is the most instrumentally rational of the options, and it and boundedness with respect to differences seem the most promising, but have barely been discussed in the sequence (if it all?). I would recommend exploring these options more.
I’m definitely in for exploring a variety of more options. We didn’t explore all possible options in this series, and I think we could, in theory, spend a lot more time investigating possible options including some of the combinations of theories, and more edge case versions of particular views like WLU you lay out.
However, I think while it is plausible EV could avoid some version of fanaticism that way, it still seems vulnerable to a very related issue like the following.
It seems there are actually two places for EV where rounding down or bound setting needs to happen to avoid issues with particularly risky gambles. (1) For really low probabilities (i.e. 1 in 100 trillion) with really high outcomes and (2) around the 50% line distinguishing actions that lean net positive from those that are neutral or negative in expectation. Conceptually, these are very similar but practically there may be different implications for doing them.
While it seems a bounded EV function with a function that assigns marginal returns a really steep decline could avoid the fanaticism of (1) (though this itself creates counterintuitive results), it doesn’t seem like this type of solution alone would resolve the issue where the the decision point is whether something is lean net positive but possibly only barely of (2).That is, there are many choices about actions where the sign of the action is uncertain and this applies, among other things, to x-risk interventions that have the possibility of having a very large expected utility if the action succeeds. Practically, it seems these types of choices are likely very common for charitable actors.
If despite a really large expected utility in your bounded function, you don’t think we should always take an action that is only, say, 50.0001% positive in expectation you wind up in a very similar place with regard to being “mugged” by high value outcomes that are not just unlikely to pay out but almost equally as likely to cause harm, then you think something has gone awry in EV. And it doesn’t seem reasonable bounds designed for avoiding really low probabilities but high EV outcomes will help you avoid this.
To be clear, I haven’t reasoned this out entirely, and I will just preemptively grant it’s possible you could create a different “bound” that would act on not just small probabilities, but also on these edge-cases where EU suggests taking these types of gambles. But if you do that this looks a lot like what you are doing is introducing a difference-making criteria to your decision theory. To the extent you may think this type of modified EU is viable, it is because it mimics the aversion of these other theories to certain types of uncertainty.
Basically, I’m actually not confident that this type of modification should matter much for us. The axiom choices matter here for which theory to put the most weight in but I’m unsure this type of distinction is buying you much practically if, say, after you make them you still end up with a set of theoretical options that look in practice like pure EV vs EV with rounding down vs something like WLU vs something like REU.
EDIT: grammar fix.
How Rethink Priorities is Addressing Risk and Uncertainty
Rethink Priorities’ Cross-Cause Cost-Effectiveness Model: Introduction and Overview
How Rethink Priorities’ Research could inform your grantmaking
In trying to convince people to support global health charities I don’t think I’ve ever gotten the objection “but people in other countries don’t matter” or “they matter far less than Americans”, while I expect vegan advocates often hear that about animals.
I have gotten the latter one explicitly and the former implicitly, so I’m afraid you should get out more often :).
More generally, that foreigners and/or immigrants don’t matter, or matter little compared to native born locals, is fundamental to political parties around the world. It’s a banal take in international politics. Sure, some opposition to global health charities is an implied or explicit empirical claim about the role of government. But fundamentally, not all of it as a lot of people don’t value the lives of the out-group and people not in your country are in the out-group (or at least not in the in-group) for much of the world’s population.
First, I think GiveWell’s research, say, is mostly consumed by people who agree people matter equally regardless of which country they live in.
GiveWell donors are not representative of all humans. I think a large fraction of humanity would select the “we’re all equal” option on a survey but clearly don’t actually believe it or act on it (which brings us back to revealed preferences in trades like those humans make about animal lives).
But even if none of that is true, were someone to make this argument about the value of the global poor, the best moral (I make no claims about what’s empirically persuasive) response is “make a coherent and defensible argument against the equal moral worth of humans including the global poor”, and not something like “most humans actually agree that the global poor have equal value so don’t stray too far from equality in your assessment.” If you do the latter, you are making a contingent claim based on a given population at a given time. To put it mildly, for most of human history I do not believe we even would have gotten people to half-heartedly select the “moral equality for all humans” option on a survey. For me at least, we aren’t bound in our philosophical assessment of value by popular belief here or for animal welfare.
Hey Vasco, I replied to the last link here and I don’t have anything to add to Laura’s responses for your first two links.
In brief, the Donor Compass is streamlined, if you want more subtlety you probably should be using the advanced version of the tool. And I while I think we should aim to have secondary effects of interventions, I want to do it in a way that doesn’t unnecessarily penalize/reward areas where that data is/is not available, and to not have the effects of all interventions be dominated by deeply uncertain secondary impacts.