I research a wide variety of issues relevant to global health and development. I’m always happy to chat—if you think we have similar interests and would like to talk, send me a calendar invite at karthikt@berkeley.edu!
Karthik Tadepalli
Karma: 2,938
In essence, is this is an alteration of information hazard that accounts for a group’s propensity to commit harm? That seems like a useful concept. It seems small, but I think it’s important to differentiate between the groups that information is being spread to (e.g. spreading information on this forum probably has lower risk than spreading the same information on 4chan).
In regards to your question about whether EA should consider spirit hazards, I haven’t read deeply enough into the concept of information hazards to know if this is well-trodden ground (I just read the top forum posts from your link) but it seems like spirit hazards are only one half of the cost-benefit equation in sharing information. You suggest that risky information should be shared only with decisionmakers, but I can think of two scenarios in which that would not be ideal.
1. Accountability—when information is spread more broadly, decisionmakers can be held accountable for their actions (e.g. an AI company could be called out for unsafe practices if it’s broadly known what practices are unsafe and why).
2. Wisdom of crowds—spreading information about a risk allows many people to independently try to develop solutions to that risk, which can be more successful than a lone decisionmaker trying to develop solutions.
This whole balance is analogous to cybersecurity, where security researchers actively spread information on vulnerabilities, because that allows other security researchers to fix them + learn from them, to stay ahead of hackers. Of course, not every scenario calls for the same information spread—explosives experts would definitely not want to spread bomb recipes on internet forums in the hope of helping out other explosives experts.
Perhaps the difference between cybersecurity and explosives that makes information spread more beneficial in the former case is that hacking is much more accessible to malicious actors than building a bomb, so dangerous information will inevitably be discovered by someone. So spirit hazard depends on the likelihood of information spreading to bad actors anyway. In domains which have a higher cost of involvement (and thus are more centralized) this is less likely to be the case, so maybe spirit hazard is still high in cases that we generally care about.
Grantmakers aren’t just people with money—they are people with a bird’s eye view of the space of grant proposals. This may not be the same as topic expertise, but it’s still quite important for a person with a project trying to make it fit into the broader goals of the EA community.
This is really fascinating. I think a pretty close analogy is international water governance. Neighboring countries share access to a river but the more upstream country can seriously damage the more downstream country with damming/water pollution. It’s a high stakes issue between neighboring countries and has led to water treaties that countries have a strong incentive to stick to because escalation would be very damaging.
Not an answer, but I’m struck by the (probably correct) presumption that people will still see the “right” answer as bednets. It’s not just a matter of optics: temporary emergencies (like the coming famines) seem very likely to offer the most cost-effective interventions, conditional on not being neglected. The reason is that 1) in the absence of the emergency, people would otherwise live long and healthy lives, 2) temporary emergencies often cost less to address than systemic issues. This is why during the height of India’s COVID-induced oxygen shortages, oxygen interventions were (arguably) more cost-effective than GiveWell top recommendations, because most people who could be saved by an oxygen tank would have lived longer lives otherwise. So people should definitely not discount famine relief ex-ante as being less cost-effective.
I would imagine the forum has a record of the total voting power of all users, which is increasing over time, and the karma can be downscaled by this total.
I don’t think it’s normally useful to preemptively direct aid because of the predictability issues you point out (coming famines being an exception). My point is more that there is a large value to quickly reacting to emergencies that we will miss if we only focus on systemic issues.
Aid infrastructure costs are a valid point, and that pushes the limit of how useful my point is as an abstract claim. I imagine that in many emergencies, a sufficiently effective infrastructure already exists and can be augmented if people put in a lot of effort (again leaning on the India COVID emergency example). But that’s just my imagination and could definitely be wrong.
Sure, bednets was just a standin for systemic issues (including malnutrition). I think COVID was an outlier in EA responsiveness to a current crisis, and the more longstanding conventional wisdom is that emergencies are usually not neglected or tractable enough to be worth spending on compared to systemic issues. Doing Good Better definitely made that argument about the inefficacy of disaster relief.
Here are features of EA that are justifiable from a utilitarian perspective, but not from other moral frameworks:
1. DALY-based evaluations of policies. The idea of a DALY assumes that quality of life is interchangeable between people and aggregatable across people, which is not common sense and not true in a rights-based framework, because rights are not interchangeable between people.
2. Longtermism. Most arguments for longtermism are of the form “there will be far more people in the future, so the future is more important to preserve” which is a utilitarian argument. Maybe you could make a “future people have rights” argument, but that doesn’t answer why their rights are potentially more important than neartermist concerns—only a population-weighting view does that.
3. (Relatedly) Population ethics. Almost every non-utilitarian view entails person-affecting views: an act is only bad if it’s bad for someone. Creating happy lives is not a moral good in other philosophies, whereas (many though not all) EAs are motivated by that.
4. Animal welfare. Animal welfare concerns as we imagine them stem from trying to reduce animal pain. You could bend over backward to describe why animals have rights, but most rights-based frameworks derive rights from some egalitarian source (making them unsuitable to say animals have “less rights than people but still some rights”, which is the intuition most of us have). Moreover, even if you could derive animal rights, it would be super unclear what actions are best to support animal rights (how do you operationalize animal dignity?), whereas a utilitarian view allows you to say “the best actions are the ones that minimize the pain that animals experience” leading you to solutions like eliminating battery cages.
I don’t think you can reject utilitarianism without rejecting these features of EA. Utilitarianism could be “wrong” in an abstract sense but I think 70% of EAs see it as the best practical guide to making the world better. It often does conflict with common-sense ethics—the common sense of most people would suggest animal suffering doesn’t matter, and that future people matter significantly less than people alive today! Utilitarianism is not an unwanted appendage to EA that could hamper it in the future. It’s the foundation of EA’s best qualities: an expanding moral circle and the optimizing of altruistic resources.
Sorry to hijack this comment, but I’ve noticed a lot of people saying veg*n and I’m confused, what’s the reason to censor it?
Of course, regex is everywhere :) thanks for clarifying!
General equilibrium thinking
Yeah, I have seen different terms used for this (direct vs indirect, static vs dynamic, marginal vs equilibrium). Direct and indirect effects are probably simpler, albeit less clear about the source of “indirect” effects. For example, direct effects of malaria prevention could be living longer and having higher income, whereas indirect effects of malaria prevention could be that areas with high malaria prevention become richer because people can work more when malaria burden is lower. Any use of “indirect effects” that I have seen would include the latter, but both of them are “partial equilibrium effects” in this terminology. So I don’t think the concepts map exactly.
General equilibrium analysis is specifically about different markets, but I know a lot of economists use “general equilibrium effects” informally to refer to the effects of people re-optimizing. That’s why I used it in this context.
And the terminology doesn’t change the basic point, which rephrased would be “we tend to overweight direct effects and underweight + not explicitly talk about indirect effects, we should be more explicit about those when debating ways to do good.”
This is interesting, but it feels like the difference needs a little more grounding. After all, “continuity” is always just an approximation—we are not literally building a continuum of AIs, one for each point along the curve. Progress from GPT-2 to GPT-3 or DALL-E to DALL-E 2 is clearly discrete. It seems like the difference is in how discrete it is, which is more of a quantitative difference than a qualitative difference.
I’m skeptical of the section of your argument that goes “weak EA doesn’t suffer from totalization, but strong EA does, and therefore EA does.”
The presence of a weak EA does not undermine the logic of a strong EA. If EA’s fundamental goal is to achieve “as much [good] as possible”, its default position will always point towards totalisation.
Why do you take strong EA as the “default” and weak EA as something that’s just “present”? I could equally say
The presence of a strong EA does not undermine the logic of a weak EA. If EA’s fundamental goal of achieving as much good as possible is subject to various self-imposed exemptions, its default position does not point towards totalization.
Adjudicating between these boils down to whether strong EA or weak EA is the better “true representation” of EA. And in answering that, I want to emphasize—EA is not a person with goals or positions. EA is what EAs do. This is normally a semantic quibble because we use “EA has the position X” as a useful shorthand for “most EAs believe X, motivated by their EA values and beliefs”. But making this distinction is important here, because it distinguishes between weak EA (what EAs do) and strong EA (what EAs mostly do not do). If most EAs believe in and practice weak EA, then I feel like it’s the only reasonable “true representation” of EA.
You address this later on by saying that weak EA may be dominant today, but we can’t speak to how it might be tomorrow. This doesn’t feel very substantial. Suppose someone objects to utilitarianism on the grounds “the utilitarian mindset could lead people to do horrible things in the name of the greater good, like harvesting people’s organs.” They then clarify, “of course no utilitarian today would do that, but we can’t speak to the behavior of utilitarians tomorrow, so this is a reason to be skeptical of utilitarianism today.” Does this feel like a useful criticism of utilitarianism? Reasonable people could disagree, but to me it feels like appealing to the future is a way to attribute beliefs to a large group even when almost nobody holds them, because they could hold those views.
Moreover, I think future beliefs and practices are reasonably predictable, because movements experience a lot of path-dependency. The next generation of EAs is unlikely to derive their beliefs just by introspecting towards the most extreme possible conclusions of EA principles. Rather, they are much more likely to derive their beliefs from a) their pre-existing values, b) the beliefs and practices of their EA peers and other EAs who they respect. Both of these are likely to be significantly more moderate than the most extreme possible EA positions.
Internalizing this point moderates your argument to a different form, “EA principles support a totalitarian morality”. I believe this claim to be true, but the significance of that as “EA criticism” is fairly limited when it is so removed from practice.
The important statistic for cost effectiveness is really the cost effectiveness of the marginal grant. If the marginal grant is very cost effective, then money is being left on the table and we should be awarding more grants! Conversely, if the marginal grant is very cost ineffective, then grant making is inefficiently large even if the average grant is cost effective. In that situation we could improve cost-effectiveness by eliminating some of those marginal grants.
The distance between the marginal grant and the average grant is increasing in the fat-tailedness of the distribution, so for very fat tailed distributions, this difference is extremely important.
This would be a much more interesting post if you elaborated on any of these ideas. In particular most of your ideas center on the “altruistic value”, but you do not explain what that metric is. You mention that it is a relative and subjective metric, which makes me imagine that it is not “utility” or some QALY-esque measure of happiness created by ones actions.
This is a great exercise. I am definitely concerned about the endogenous nature of predictions: I think you are definitely right that people are more likely to offer predictions on the aspects of their project that are easier to predict, especially since the prompt you showed explicitly asks them in an open-ended way. A related issue is that people may be more comfortable making predictions about less important aspects of the project, since the consequences of being wrong are lower. If this is happening, then this forecasting accuracy wouldn’t generalize at all.
Both of these issues can be partly addressed if the predictions are solicited by another person reading the writeup, rather than chosen by the writer. For example, Alice writes up an investigation into human challenge trials as a strategy for medical R&D, Bob reads it and asks Alice for some predictions that he feels are important to complementing the writeup e.g. “will human challenge trials be used for any major disease by the start of 2023?” and “will the US take steps to encourage human challenge trials by the start of 2024?”
This obviously helps avoid Alice making predictions about only easier questions, and it also ensures that the predictions being made are actually decision-relevant (since they are being solicited by someone else who serves the role of an intelligent layperson/policymaker reading the report). Seems like a win-win to me.
Expert opinion has always been a substitute for object level arguments because of deference culture. Nobody has object level arguments for why x-risk in the 21st century is around 1/6: we just think it might be because Toby Ord says so and he is very credible. Is this ideal? No. But we do it because expert priors are the second best alternative when there is no data to base our judgments off of.
Given this, I think criticizing an expert’s priors is functionally an object level argument, since the expert’s prior is so often used as a substitute for object level analysis.
I agree that a slippery slope endpoint would be bad but I do not think criticizing expert priors takes us there.
I think a very healthy epistemic habit is to not be completely caught up in the thoughtspace of any one community. Thinking is a garbage-in-garbage-out process. If all your inputs are from EA/rationalism, then you end up regurgitating EA/rationalism without adding new insights.
One of my favorite EA Forum posts has been this cause investigation for violence against women and girls, which the author mentions is inspired by their work as a healthcare worker and observing violence against women on the ground. Part of the reason this is so valuable is because it is extremely unlikely that EAs would have been exposed to this otherwise. The author combined external knowledge with EA reasoning/cost-effectiveness analysis to create something that neither the average EA nor the average healthcare worker could have created.
Deference to EA/rationalist conventional wisdom is useful in the many domains where you are not an expert. But EA did not get to be this way through parochial thinking. It got this way through new ideas continually being introduced so that good ones float to the top.
In summary: you should be able to point to a sizable number of intellectual influences that are not EA or EA-adjacent. EA is a question, not an answer, and our ability to give good answers is contingent on our continually bringing new perspectives and insights to the marketplace of ideas.
Are we allowed to submit multiple candidate cause areas for evaluation? If so, would that be as separate proposals?