AI alignment research, with a focus on moral progress and CEV.
Babel
Karma: 136
Here’s the link!
I’d also like to reiterate the arguments Larry Temkin gave against international aid, since the post doesn’t cover them. I’m not sure if I’m convinced by these arguments, but I do find them reasonable and worth serious consideration.
Opportunity cost of local human resources: International aid agencies tend to hire competent local people in the country they operate in (e.g. Sub-Saharan African countries), but these competent people could otherwise serve in important roles for the development of the local society.
Corruption: Lots of international aid funds are corrupted by government officials and local thugs (and such corruption is often covered up). Often, the corrupted funds then go on to fuel the agenda of thugs or malevolent political actors, which is harmful to the people of the aided nation.
In addition to the two empirical arguments above, he also gave normative arguments, but those arguments are commonly raised and rather well-known by EAs, so I don’t repeat them here.
Comment from author: Note that I lean slightly towards the term “animal advocacy”, so it’s possible that my analysis contains a slight bias towards this term.
I like this idea of using structured discussion platforms to aggregate views on a topic.
However, there is a cost for an individual to switch to new platforms, so perhaps the harder task is to get a large number of EAs to use this platform.
A counter-argument: Here it is argued that the research supporting the 3.5% figure may not apply to the animal advocacy context.
I think building epistemic health infrastructure is currently the most effective way to improve EA epistemic health, and is the biggest gap in EA epistemics.
I elaborated on this in my shortform. If the suggestion above seems too vague, there’re also examples in the shortform. (I plan to coordinate a discussion/brainstorming on this topic among people with relevant interests; please do PM me if you’re interested)
(I was late to the party, but since Nathan encourages late comments, I’m posting my suggestion anyways.)
Suggestion: I think building epistemic health infrastructure is currently the most effective way to improve EA epistemic health, and is the biggest gap in EA epistemics.
I elaborated on this in my shortform. If the suggestion above seems too vague, there’re also examples in the shortform. (I plan to coordinate a discussion/brainstorming on this topic among people with relevant interests; please do PM me if you’re interested)
(I was late to the party, but since Nathan encourages late comments, I’m posting my suggestion anyways. I’m posting the comment also under this post besides the previous one, because Nathan said that “maybe add them to both”; please correct me if that’s my misunderstanding)
Proposal: I think building epistemic health infrastructure is currently the most effective way to improve EA epistemic health, and is the biggest gap in EA epistemics.
My definition of epistemic health infrastructure: a social, digital, or organizational structure that provides systematic safeguards against one or more epistemic health issues, by regulating some aspect of the intellectual processes within the community.
They can have different forms (social, digital, organizational, and more) or different focuses (individual epistemics, group epistemology, and more), but one thing that unites them is that they’re reliable structures and systems, rather than ad hoc patches.
(note: in order to keep the shortform short I tried to be curt when writing the content below; as a result the tone may come out harsher than I intended)
We talk a lot about epistemic health, but have massively underinvested in infrastructure that safeguards epistemic health. While things like EA Forum and EAG and social circles at EA hubs are effective at spreading information and communicating ideas, to my knowledge there has been no systematic attempt at understanding (and subsequently improving) how they affect epistemic health.
Examples of things not-currently-existing that I consider epistemic health infrastructure:
(Not saying that these are the most valuable ones; just that they fall into this category. They are examples and are only examples.)
mechanisms to poll/aggregate community opinions (e.g. a more systematized version of Nathan Young’s Polis polls) on all kinds of important topics, with reliable mechanisms to execute actions according to poll results
something like the CEA community health team, but focusing on epistemic health, with better-defined duties and workflows, and with better transparency
EA forum features and (sorting/recommending/etc.) algorithms aiming to minimize groupthink/information cascades
(Many proposals from other community members about improving community epistemic health also fall into this category. I won’t repeat them here.)
I plan to coordinate a discussion/brainstorming on this topic among people with relevant interests. Please do PM me if you’re interested!
- What specific changes should we as a community make to the effective altruism community? [Stage 1] by (5 Dec 2022 9:04 UTC; 58 points)
- 's comment on What specific changes should we as a community make to the effective altruism community? [Probably gonna delay this a bit] by (7 Dec 2022 13:54 UTC; 1 point)
Apologies for posting four shortforms in a row. I accumulated quite a few ideas in recent days, and I poured them all out.
Summary: When exploring/prioritizing causes and interventions, EA might be neglecting alternative future scenarios, especially along dimensions orthogonal to popular EA topics. We may need to consider causes/interventions that specifically target alternative futures, as well as add a “robustness across future worlds” dimension to the ITN framework.
Epistemic status: low confidence
In cause/intervention exploration, evaluation and prioritization, EA might be neglecting alternative future scenarios, e.g.
alternative scenarios of the natural environment: If the future world experienced severe climate change or environmental degradation (which has serious downstream socioeconomic effects), what are the most effective interventions now to positively influence such a world?
alternative scenarios of social forms: If the future world isn’t a capitalist world, or is different from the current world in some other important aspect, what are the most effective interventions now to positively influence such a world?
...
This is not about pushing for certain futures to realize. Instead, it’s about what to do given that future. Therefore, arguments against pushing for certain futures (e.g. low neglectedness) do not apply.
For example, an EA might de-prioritize pushing for future X due to its low neglectedness, but if they think X has a non-trivial probability to realize, and its realization has rich implications for cause/intervention prioritization, then whenever doing prioritization, they need to think about “what I should do in a world where X would be realized”. This could mean:
finding causes/interventions that are robustly impactful across future scenarios, or
finding causes/interventions that specifically target future X.
In theory, the consideration of alternative futures should be captured by the ITN framework, but in practice it’s usually not. Therefore it could be valuable to add one more dimension to the ITN framework: “robustness across future worlds”.
Also, there’re different dimensions on which futures can differ. EA tends to have already considered the dimensions that are related to EA topics (e.g. which trajectory of AI is actualized), but tends to ignore the dimensions that aren’t. But this is unreasonable, as EA-topic-related dimensions aren’t necessarily the dimensions in which futures have the largest variance.
Finally, note that in some future worlds, it’s easier to have high altruistic impact than in other worlds. For example in a capitalist world, altruists seem to be at quite a disadvantage to profit-seekers; in some alternative social forms, altruism plausibly becomes much easier and more impactful, while in some other social forms, it may become even harder. In such cases, we may want to prioritize the futures that have the most potential for current altruistic interventions.
Epistemic status: I only spent 10 minutes thinking about this before I started writing.
Idea: Funders may want to pre-commit to awarding whoever accomplished a certain goal. (e.g. maybe some funder like Open Phil can commit to awarding a pool of money to people/orgs who reduce meat consumption to a certain level, and the pool will be split in proportion to contribution)
Detailed considerations:
This can be seen as a version of retroactive funding, but it’s special in that the funder makes a pre-commitment.
(I don’t know a lot about retroactive funding/impact markets, so please correct me if I’m wrong on the comparisons below)
Compared to other forms of retroactive funding, this leads to the following benefits:
less prebuilt infrastructure is needed
provides stronger incentives to prospective “grantees”
better funder coordination
better grantee coordination
… but also the following detriments:
much less flexibility
perhaps stronger funding centralization
potentially unhealthy competition between grantees
Compared to classical grant-proposal-based funding mechanisms, this leads to the following benefits:
better grantee coordination
stronger incentives for grantees
more flexibility (i.e. grantees can use whatever strategy that works, rather than whatever strategy the funder likes)
… but also the following detriments:
lack of funds to kickstart new projects that otherwise (ie if without funding) wouldn’t be started
perhaps stronger funding centralization
potentially unhealthy competition between grantees
Important points:
The goals should probably be high-level but achievable, while being strategy-agnostic (i.e. you can use whatever morally acceptable strategies to achieve the goal). Otherwise, you lose a large part of the value from pre-committed awards—sparkling creativity from prospective grantees.
If your ultimate goal is too large and you need to decompose it into subgoals and award the subgoals, make sure your subgoals are dispersed across a diverse range of tracks/strategies. For example, if your ultimate goal is to reduce meat consumption, you may want to set subgoals on the alt protein track, as well as on the vegan advocacy track, and various other tracks.
Explicitly emphasize that foundation-building work will be awarded, rather than awarding only the work that completed the one last step to the goal.
Attribute contribution using an open and transparent research process. Maybe crowdsource opinions from a diverse group of experts.
Such research will be hard. This is IMO one of the biggest barriers to this approach, but I think it applies to other versions of retroactive funding/impact markets too.
Four podcasts on animal advocacy that I recommend:
Freedom of Species (part of 3CR radio station)
Covers a wide range of topics relevant to animal advocacy, from protest campaigns to wild animal suffering to VR. More of its episodes are on the “protest campaigns” end which is less popular in EA, but I think it’s good to have an alternative perspective, if only for some diversification.Knowing Animals (hosted by Josh Milburn)
An academic-leaning podcast that focuses on Critical Animal Studies, which IMO is like the academic equivalent of animal advocacy. Most guests are academics in philosophy, humanities and social sciences. (and btw, one episode discussed wild animal suffering, and I liked that episode quite a lot)The Sentience Institute Podcast
EA-aligned. Covers topics ranging from alt proteins to animal-focused impact investing to local animal advocacy groups to digital sentience.Animal Rights: The Abolitionist Approach Commentary (by Gary L. Francione)
A valuable perspective that’s not commonly seen in EA. Recommended for diversification.
Off-topic: I also recommend the Nonlinear Library podcasts; they turn posts on EA Forum and other adjacent forums (LW, AF) to audio. There’re different versions that form a series, including a version containing all-time top posts of EA Forum. There’s also a version containing the latest posts meeting a not-very-high karma bar—I use that version to keep track of EA news, and it saved me a lot of time.
Statement: This shortform is worth expanding into a top-level post.
Please cast upvote/downvote on this comment to indicate agreement/disagreement to the above statement. Please don’t hesitate to cast downvotes.
If you think it’s valuable, it’ll be really great if you are willing to write this post, as I likely won’t have time to do that. Please reach out if you’re interested—I’d be happy to help by providing feedback etc., though I’m no expert on this topic.
Summary: This is a slightly steelmanned version of an argument for creating a mass social movement as an effective intervention for animal advocacy (which I think is neglected by EA animal advocacy), based on a talk by people at Animal Think Tank. (Vote on my comment below to indicate if you think it’s worth expanding into a top-level post)
link to the talk; alternative version with clearer audio, whose contents—I guess—are similar, but I’m not sure. (This shortform doesn’t cover all content of the talk, and has likely misinterpreted something in the talk; I recommend you to listen to the full talk)
Epistemic status: An attempt at steelmaning the arguments, though I didn’t really try hard—I just wrote down some arguments that occur to me.
The claim: Creating a mass social movement around animals, is more effective than top-to-bottom interventions (e.g. policy) and other interventions like vegan advocacy, at least on current margins.
This is not to say policy work isn’t important. Just that it comes into the picture later.
My impression is that the track record of mass movements in creating change is no less impressive than that of policy reforms, but EA seems to have completely neglected the former.
A model of mass movements:
Analogous to historic movements like the civil rights movement in the US, and recent movements like Extinction Rebellion. Both examples underwent exponential growth, which will be explained in the next bullet point.
You start with a pool of people in the movement, and these people go out and try to grab attention for the movement, using tactics like civil disobedience and protests. Exposure to the ideas leads to more people thinking about them, which in turn leads to more people joining. With the enlarged people pool, you start the cycle again. This then leads to an exponentially growing pool.
After the movement is large enough and has enough influence, policy reforms and other interventions aimed at the top of society will become viable.
Research showed that few, if any, movements failed after reaching a size threshold of 3.5% of the entire population.
Many movements died down because their base number of exponentiation is smaller than 1, but successful movements can have much higher base number.
Other interventions like vegan outreach and policy work may also have similar exponential growth, but it’s plausible that their base numbers are much less likely to be >1 (or to be very high) when compared with mass social movements.
Strategies for mass movements:
Strategy is super important!
Start from your ultimate goal (e.g. stop animal exploitation), and then set milestones for achieving this goal, and then design concrete actions and campaigns in service of milestones.
One key point is “escalation”—how to make the movement grow exponentially starting from the initial pool
You need to be momentum-driven: convert the attention you get to new movement members, seize more attention with your enlarged membership, and repeat the cycle
You need to force people in the general public to take sides, possibly by non-violent disruptions and making salient sacrifices (e.g. arrests)
You may need to show concrete demands (rather than abstract ones) that resonate with people
Another key point is “absorption”—when large numbers of new members join the movement, how to rapidly and effectively absorb them
Decentralized movements can absorb more rapidly. (e.g. people trained can go off independently and train other new people)
There’s no silver bullet; we still need deep thinking and discussions and coordination to guide our strategy.
Do you think it’s worth expanding into a top-level post? Please vote on my comment below.
Hypothesis: in the face of cluelessness caused by flow-through effects, “paving path for future progress” may be a robust benefit of altruistic actions.
Epistemic status: off-the-cuff thoughts, highly uncertain, a hypothesis instead of a conclusion
(In this short-form I will assume a consequentialist perspective.)
Take slavery abolition as an example. The abolition of slavery seems obviously positive at the object level. But when we take into account second-order effects, things become less clear (e.g. the meater-eater problem). However, I think the bad second-order effects (if any) can plausibly be outweighed by one big second-order benefit: that the abolition of slavery paves the way for future moral progress, including (but not limited to) those around our treatment of animals. For example, it seems likely to me that in a world with slavery, it would be much harder to advocate for the rights of human minorities, of animals, and of digital sentience.
I guess this applies to many other cases too, including cases irrelevant to moral progress but relevant to some other kind of progress. This hypothesis might not change how we act by much, as we usually tend to ignore hard-to-evaluate second-order effects. This hypothesis may provide a reason why sometimes an action is justified despite of seemingly negative second-order effects, but I also worry that it may be abused, as a rationalization for ignoring flow-through effects.
My personal approach:
I no longer think of myself as “a good person” or “a bad person”, which may have something to do with my leaning towards moral anti-realism. I recognize that I did bad things in the past and even now, but refuse to label myself “morally bad” because of them; similarly, I refuse to label myself “morally good” because of my good deeds.
Despite this, sometimes I still feel like I’m a bad person. When this happens, I tell myself: “I may have been a bad person, so what? Nobody should stop me from doing good, even if I’m the worst person in the world. So accept that you have been bad, and move on to do good stuff.” (note that the “so what” here doesn’t mean I’m okay with being bad; just that being bad has no implication on what I should do from now on)
It doesn’t mean it’s okay to do bad things. I ask myself to do good things and not to do bad things, not because this makes me a better person, but because the things themselves are good or bad.
The past doesn’t matter (except in teaching you lessons), because it is already set in stone. The past is a constant term in your (metaphorical) objective function; go optimize for the rest (i.e. the present and the future).
Don’t know how to use Airtable, but a quick googling led me to this. The last reply (by kuovonne) in the linked thread seems useful.
I’m excited to see this post, thank you for it!
I also think much more exploration and/or concrete work needs to be done in this “EA+AI+animals” (perhaps also non-humans other than animals) direction, which (I vaguely speculate) may extend far beyond the vicinity of the Project CETI example that you gave. Up till now, this direction seems almost completely neglected.
I’ll be giving some critique below, but nevertheless, thank you for the idea and the analysis!
I think the animal welfare section of this post would benefit from more rigor. (not sure about the other sections; haven’t read them yet)
healthy: “oysters, mussels, scallops, and clams are good for you. They’re loaded with protein, healthy fats, and minerals like iron and manganese.”
Neither the linked article nor the quote sounds very credible or scientifically convincing to me.
Eating bivalves causes less suffering than an equivalent amount of chickens, pigs, cows, and most other animals.
To me this seems highly non-obvious. Maybe explain why you think so?
Also, I suspect this depends a lot on one’s moral weights assigned to different species, which (I guess) varies hugely across different people.
Depending on what it substitutes for, it would also reduce crop farming and associated rodent/insect deaths, which are more sentient than bivalves.
It’s good that field deaths are included in the analysis.
But one may also want to count the second-order effects of bivalve aquaculture (note that I have no knowledge about this and don’t know if this will significant change the conclusions).
Non-EAs are receptive to a proposal to substitute bivalves for other meat.
This also seems non-obvious to me.
Therefore, bivalves are the most effective way to reduce overall animal suffering.
This is a really bold claim and would deserve much more argumentation. Consider, for example, doing a cost-effectiveness comparison with the popular EA animal welfare interventions, if you’d like to argue for this.
Again, thank you for the post, and please don’t take this comment as an attempt of dismissal; just pointing out where I think it could be improved :)
How does “practicing compassion and generosity with those around us” get operationalized in the EA community?
The most salient example that comes to mind may be going vegetarian/vegan (for ethical and/or climate reasons), which (a little less than) half of the community members claimed to have done, according to a survey.
Apart from that there’s also everyday altruism, e.g. helping granny cross the street.
Nothing more comes up, though I had only thought about this for twenty seconds so I have probably missed something.
Value lock-in is the central variable. If AGI leads to lock-in of current human values, then humans may suvive while animals keep suffering.
If by “AGI goes well” we also include the continuation of things like moral progress (which current AI existential safety work does NOT address!), then the two are indeed aligned.