Iâm a software engineer on the CEA Online team, mostly working on the EA Forum. We are currently interested in working with impactful projects as contractors/âconsultants, please fill in this form if you think you might be a good fit for this.
You can contact me at will.howard@centreforeffectivealtruism.org
Separate from your comment, I have seen comments like this elsewhere (albeit also mainly from Bob and other RP people), so I still think itâs interesting additional evidence that this is a thing.
It seems like some people find it borderline inconceivable that neuron counts correspond to higher experience intensity/âsize/âmoral weight, and some people find it inconceivable the other way. This is pretty interesting!
Hereâs a screenshot (open in new tab to see it in slightly higher resolution). Iâve also made a spreadsheet with the individual voting results, which gives all the info that was on the banner just in a slightly more annoying format.
We are also planning to add native way to look back at past events as they appeared on the site :), although this isnât a super high priority atm.
Iâm curating this post. I see this and @NickLaingâs post as the best in class on the topic of moral weights from the AW vs GH debate week, so Iâm curating them as a pair[1].
I was impressed by titotal doing the fairly laborious work of replicating everyoneâs calculations and finding the points where they diverged. As discussed by the man himself, there were lots of different numbers flying around, and even if AW always comes out on top it matters by how much:
You might think this doesnât matter, but the difference between 1500 times and 35 times is actually quite important: if youâre at 1500 times, you can disagree with a few assumptions a little and still be comfortable in the superiority of AW. But if itâs 35 times, this is no longer the case, and as we shall see, there are some quite controversial assumptions that these models share.
See also the other curation comment
Iâm curating this post. The issue of moral weights turned out to be a major crux in the AW vs GH debate, and Iâm excited about more progress being made now that some live disagreements have been surfaced. Iâm curating this post and @titotalâs (link) as the âbest in classâ from the debate week on this topic.
On this post: The post itself does a good job of laying out some reasonable-to-hold objections to RPâs moral weights. Particularly I think the point about discounting behavioural proxies is important and likely to come up again in future.
I think comment section is also very interesting, there are quite a few good threads:
1. @David MathersđžâČ comment which raises the point that the idea of estimating intensity/âsize of experience from neuron counts doesnât come up (much) in the academic literature. This was surprising to me!
2. @Bob Fischerâs counterpoint making the RP case
3. This thread which gets into the issue of what counts as an uninformed prior wrt moral weights
Digression but I would recommend reading about Thompson sampling :) (wikipedia, inscrutable LessWrong post). Itâs a good model to have for thinking about explore-exploit tradeoffs in general.
This is a problem we have thought about, and personally I think it is quite bad, in that it causes a lot of randomness in which posts get past ~10 karma. We agree that essentially showing new posts to more people is the answer (even if they start to get downvoted).
The fairly standard solution in recommendation algorithms is to view this as an explore/âexploit problem, and to add some randomness to tune the tradeoff between then (see Bandits for Recommender Systems). This would mean each user would get a slightly different ordering of the list (stable per user) in order to make sure each post gets an appropriate number of views (based on the quality signal we have so far).
Hereâs an internal doc with our plans on this. We may not get to it that soon, because of giving season and other stuff, but it is one of our priorities after that.
Hide the vote score of all new posts
Iâm not that optimistic about this. For algorithm purposes the thing that matters is the number of opportunities a post gets to be voted on (~views) before dropping off the frontpage. Presumably the theory here is that seeing post has low karma makes you less likely to view it and/âor biased towards also downvoting it.
Based on my own experience I donât think this is the case. If I see a 0 to negative karma post on the frontpage I think Iâm more likely to click it out of morbid curiosity, and then if itâs at least ok Iâll upvote it because I think the current score is overly harsh. I know a lot of people apply this logic about voting (correcting the current score rather than applying a fully independent judgement), and we generally endorse this.
Some consideration should also go to engagementâsome metric related to either number of votes or comment count
I think this is a good idea (thinking about comment counts specifically), although I agree it could be prone to going wrong, e.g. making doom threads even doomier. One solution I can think of is to give a boost for comments but only up to a fairly low cutoff (say, 6 comments). Thoughts on this?
For a ~20 karma post I think having 5+ comments is a good signal that its a valuable-but-niche post.
Weâve just shipped an update to the debate week banner, so you can expand the columns to see more of the distribution, as itâs getting a bit squashed. You just have to click on one of the â+Nâ circles.
(Feel free to downvote this quick take if it hangs around on the Frontpage for too long)
Iâm also a big fan, I was just saying to @Toby Tremlettđč earlier how Iâm always excited when he comments on my stuff
I was assuming that a reduction in agriculture would result in an overall reduction in the biomass (and âneuron countâ[1]) of birds and mammals, because:
Currently the biomass of farmed birds + mammals is about 10x that of farmed birds + mammals (source, not sure how marine mammals are counted but only the ballpark is needed), and this is only using 45% of the habitable land as you say
Logically it makes sense that farming aims to efficiently convert land area into animal biomass, and has more of a top down ability to achieve this than nature does. The animals that are most widely farmed are partly chosen for having food chains of only one step, and not needing to run around a lot expending energy.
A point against this is that animals are slaughtered earlier than their natural lifespan, which would result in fewer days experienced per unit of feed input. But given the numbers above I donât think this is an offsetting factor
Of course by number of individuals it would go the other way, which is presumably why you are concerned about reducing farming from a suffering-focused perspective. So I think this comes back to the issue of moral weights for small animals (as usual đ).
...
Iâm now trying to inhabit a position that I donât exactly believe, but is interesting and that I do find somewhat persuasive.
From a bigger picture perspective, you can imagine someone trying to derive the optimal arrangement of civilisation according to hedonic utilitarianism, where they accept something closer to my end of the suffering-focused and logarithmic-intensity axes[2]. Suppose they have a good model of evolutionary theory and economics but lack the details of how life on earth currently looks.
They might think something like the following: âIn order to expect any kind of top down control over the outcome you need an intelligent species + culture that can coordinate over the use of large areas (geographical, or in whatever relevant space). This species will probably be high maintenance, because they will need to have had very complex and demanding needs and wants in order for them to develop the necessary culture in the first place.
The ideal scenario would be for a relatively small population of this high maintenance species to act as stewards for a much larger population of creatures that are very low maintenance, in order to achieve a high total utility with the resources available. These low maintenance creatures should be chosen to not require a lot of energy, be easily satisfied with simple and cheap pleasures, and have simple social structures such that you can scale the number of individuals without too many side effects.
Of course, this is just a pipe dream, because this would require the advanced species to have some kind of intrinsic preference for stewarding this larger population. Among the set of all goals it seems unlikely they would have this specific oneâ.
If you look at the actual world it is quite striking how close it is to this vision. Humanity does maintain large populations of low maintenance animals, using a large proportion of the resources that are available to do so, at minimum economic cost. The difference is that we currently torture them.
If you were to accept the vision above, it looks like an easier move from âmaintaining large population and torturing themâ to âmaintaining large population and trying to give them happy livesâ, than it is from âlarge population + torturing themâ to â90% smaller population of domesticated animalsâ to âlater maybe we make the population large again for morally motivated reasonsâ.
...
Anyway, sorry for getting on a tangent from directly replying to your comment, but this long term picture is the thing that makes me actually uneasy about going hard on interventions to end factory farming. That is, on the margin currently Iâm pretty happy with a best guess of it being positive expected value to reduce the amount of animal farming, but would be more hesitant about ending farming overnight because of the potential for irreversible effects.
I would expect that if non-animal protein sources become clearly superior than animal sources, then this would result in a very rapid collapse in the number of farmed animals, and that once this has happened it could be a lot harder to move towards the âhigh population, high welfareâ world (because we would start using all the land for something else, and the idea of using a large fraction of the land on earth for managed populations of animals would become seen as weird).
I think itâs not widely conceptualised that potentially âPTC-dominant alternative protein â >50%[3] collapse in the welfare-range-weighted population of creatures within 10 yearsâ.
Used as a stand-in for some more accurate proxy for sentience, but which scales predominantly with brain size/âcomplexity rather than number of individuals
I.e. they think extremely bad experiences are not orders of magnitude worse than simply quite bad experiences
Using â>50%â as a stand-in for âa quite surprising amount of the total fractionâ and welfare-range-weighted as a stand-in for âweighted by the delta in welfare that humans could reasonably expect to achieve with some degree of confidence (e.g. without it being in animals that are so different from humans that their sentience is highly questionable)â
Thanks Vasco, I did vote for animal welfare, so on net I agree with most of your points. On some specific things:
You could donate to organisations improving instead of decreasing the lives of animals
This seems right, and is why I support chicken corporate campaigns which tend to increase welfare. Some reasons this is not quite satisfactory:
It feels a bit like a âhelping slaves to live happier livesâ intervention rather than âfreeing the slavesâ
Iâm overall uncertain about whether animals lives are generally net positive, rather than strongly thinking they are
Iâd still be worried about donations to these things generally growing the AW ecosystem as a side effect (e.g. due to fungibility of donations, training up people who then do work with more suffering-focused assumptions)
But these are just concerns and not deal breakers.
Rethink Prioritiesâ median welfare range for shrimps of 0.031 is 31 k (= 0.031/â10^-6) times their welfare range based on neurons of 10^-6. For you to get to this super low welfare range, you would have to justify putting a very low weight in all the other 11 models considered by Rethink Priorities.
I am sufficiently sceptical to put a low weight on the other 11 models (or at least withhold judgement until Iâve thought it through more). As I mentioned Iâm writing a post Iâm hoping to publish this week with at least one argument related to this.
The gist of that post will be: itâs double counting to consider the 11 other models as separate lines of evidence, and similarly double counting to consider all the individual proxies (e.g. âanxiety-like behaviourâ and âfear-like behaviourâ) as independent evidence within the models.
Many of the proxies (I claim most) collapse to the single factor of âdoes it behave as though it contains some kind of reinforcement learning system?â. This itself may be predictive of sentience, because this is true of humans, but I consider this to be more like one factor, rather than many independent lines of evidence that are counted strongly under many different models.
Because of this (a lot of the proxies looking like side effects of some kind of reinforcement learning system), I would expect we will continue to see these proxies as we look at smaller and smaller animals, and this wouldnât be a big update. I would expect that if you look at a nematode worm for instance, it might show:
âTaste-aversion behaviourâ: Moving away from a noxious stimulus, or learning that a particular location contains a noxious stimulus
âDepression-like behaviourâ: Giving up/âputting less energy into exploring after repeatedly failing
âAnxiety-like behaviourâ: Being put on edge or moving more quickly if you expose it to a stimulus which has previously preceded some kind of punishment
âCuriosity-like behaviourâ: Exploring things even when it has some clearly exploitable resource
It might not show all of these (maybe a nematode is in fact too small, I donât know much about them), but hopefully you get the point that these look like manifestations of the same underlying thing such that observing more of them becomes weak evidence once you have seen a few.
Even if you didnât accept that they were all exactly side effects of âa reinforcement learning type systemâ (which seems reasonable), still I believe this idea of there being common explanatory factors for different proxies which are not necessarily sentience related should be factored in.
(RPâs model does do some non-linear weighting of proxies at various points, but not exactly accounting for this thing⊠hopefully my longer post will address this).
On the side of neuron counts, I donât think this is particularly strong evidence either. But I see it as evidence on the side of a factor like âtheir brain looks structurally similar to a humanâsâ, vs the factor of âthey behave somewhat similarly to a humanâ for which the proxies are evidence.
To me neither of these lines of evidence (âbrain structural similarityâ and âbehavioural similarityâ) seems obviously deserving of more weight.
Farmed animals are neglected, so I do not think worldview diversidication would be at risk due to moving 100 M$ to animal welfare
I definitely agree with this, I would only be concerned if we moved almost all funding to animal welfare.
As far as Iâm aware itâs a coincidence, but Iâm v happy about this :)
My personal reasons favouring global health:
Iâm sceptical of Rethinkâs moral weight numbers[1], and am more convinced of something closer to anchoring on neuron counts (and even more convinced by extreme uncertainty). This puts animal charities more like 10x ahead rather than 1000 or 1 million times. Iâm also sceptical of very small animals (insects) having a meaningful probability/âdegree of sentience.
I am sceptical of suffering focused utilitarianism[2], and am worried that animal welfare interventions tend to lean strongly in favour of things that reduce the number of animals, on the assumption that their lives are net negative. Examples of this sort of mindset include this, this, and this.
Not all of these actively claim the given animalsâ lives must be net negative, but Iâm concerned about this being seen as obviously true and baked into the sorts of interventions that are pursued. Iâm especially concerned about the idea that the question of whether animalsâ lives are net-negative is not relevant (see first linked comment), because the way in which it is relevant is that it favours preventing animals from coming into existence (this is more commonly supported than actively euthanising animals).
Farmed animals are currently the majority of mammal + bird biomass, and so ending the (factory) farming of animals is concomitant with reducing the total mammal + bird population[3] by >50%, and this is not something that I see talked about as potentially negative.
That said, if pushed I would still fairly strongly predict that farmed chickens lives are net negative at least, which is why on net I support the pro animal welfare position.
I think something like worldview diversification is essentially a reasonable idea, for reasons of risk aversion and optimising under expected future information. The second is an explore/âexploit tradeoff take (which often ends up looking suspiciously similar to risk aversion đ§).
In the case where there is a lot of uncertainty on the relative value of different cause areas (not just in rough scale, but that things we think are positive EV could be neutral or very negative), it makes sense to hedge and put a few eggs into each basket so that you can pivot when new important information arises. It would be bad to, for instance, spend all your money euthanising all the fish on the planet and then later discover this was bad and that also there is a new much more effective anti-TB intervention.
Of course, this more favours doing more research on everything than it does pouring a lot of exploit-oriented money into Global Health, but in practice I think some degree of trying to follow through on interventions is necessary to properly explore (plus you can throw in some other considerations like time preference/âdiscount rates), and OpenPhil isnât spending money overall at a rate that implies reckless naive EV maximising (over-exploitation).
Some written-down ideas in this direction: We can do better than argmax, Tyranny of the Epistemic Majority, In defense of more research and reflection.
I believe something like âpartiality shouldnât be a (completely) dirty wordâ. When taken to extremes, most people accept some concessions to partiality. For instance itâs generally considered not a good strategic move to pressure people into giving so much of their income that they canât live comfortably, even though for a sufficiently motivated moral actor this would likely still be net positive. Most people also would not jump at the chance to be replaced by a species that has 10% higher welfare.
I think itâs wrong to apply this logic only at the extremes, and there should be some consideration of what the market will bear when considering more middle of the road sacrifices. For instance a big factor in the cost effectiveness of lead elimination is that it can be happily picked up by more mainstream funders.
(I realise a lot of these are not super well justified, Iâm just trying to get the main points across).
Iâm planning to publish a post this week addressing one small part of this, although itâs a pretty complicated topic so I donât expect this to get that far in justifying the position
Not meant in a very technical sense, just as the idea that there is probably more suffering relative to positive wellbeing, or that itâs easier to prevent it. Again, this is for reasons that are beyond the scope of this post. But two factors are:
1) I think common sense reasoning about the neutral point of experience is overly pessimistic
2) I am sceptical of the intensity of pain and pleasure being logarithmically distributed (severe pain ~100x worse than moderate pain), and especially of this being biased in the negative direction. One reason for this is that I find the âfirst storyâ for interpreting Weberâs law in this post much more intuitive, i.e. that logarithmically distributed stimuli get compressed to a more linear range of experience
Weighted by biomass obviously. The question of actual moral value falls back to the moral weights issue above. A point of reference on the high-moral-weights-sceptical end of the spectrum is this table @Vasco Grilođž compiled of aggregate neuron counts (although, as mentioned, I donât actually think neuron counts are likely to hold up in the long run)
Iâm curating this post. Reading this post was a turning point for me from taking counting arguments seriously to largely rejecting them without a strong reason to think the principle of indifference holds. I thought the reductio arguments at the start were really well chosen to make the conclusion seem obvious (at least against the strict form of the argument) without leaving room for ML-specific nitpicks.
Non-moderator nudge: Given that most of the comments here are created via voting on the banner, Iâd like to discourage people from downvoting comments below zero just for being low effort. I think itâs still useful to leave a quick note in this case, so people can see them when browsing the banner. Hopefully positive karma will still do the job of sorting really good ones to the top.
The basic case for chickens is very strong, even under views that are sceptical of small animals having a high chance/âdegree of sentience, because itâs so easy to affect their lives cheaply compared to humans, and their lives seem v easy to improve by a lot
$100m in total is not a huge amount (equiv to $5-10m/âyr, against a background of ~$200m). I think concern about scaling spending is a bit of a red herring and this could probably be usefully absorbed just by current interventions
I thought about it more, and I am now convinced that the paper is right (at least in the specific example I proposed).
The thing I didnât get at first is that given a certain prior over , and a number of iterations survived, there are âmore surviving worldsâ where the actual is low relative to your initial prior, and that this is exactly accounted for by the Bayes factor.
I also wrote a script that simulates the example I proposed, and am convinced that the naive Bayes approach does in fact give the best strategy in Jackâs case too (I havenât proved that there isnât a counterexample, but was convinced by fiddling with the parameters around the boundary of cases where always-option-1 dominates vs always-option-2).
Thanks, this has actually updated me a lot :)
I no longer endorse this, see reply below:
I donât think this does away with the problem, because for decision making purposes the fact that a random event is extinction-causing or not is still relevant (thinking of the Supervolcano vs Martians case in the paper). I didnât see this addressed in the paper. Hereâs a scenario that hopefully illustrates the issue:
A game is set up where a ball will be drawn from a jar. If it comes out red then âextinctionâ occurs, the player loses immediately. If it comes out green then âsurvivalâ occurs, and the player continues to the next round. This is repeated (with the ball replaced every time) for an unknown number of rounds with the player unable to do anything.
Eventually, the game master decides to stop (for their own unknowable reasons), and offers the player two options:
Play one more round of drawing the ball from the jar and risking extinction if it comes out red
Take a fixed 10% chance of extinction
If they get through this round then they win the game.
The game is played in two formats:
Jack is offered the game as described above, where he can lose before getting to the decision point
Jill is offered a game where rounds before the decision point donât count, she can observe the colour of the ball but doesnât risk extinction. Only on the final round does she risk extinction
Letâs say they both start with a prior that P(red) is 15%, and that the actual P(red) is 20%. Should they adopt different strategies?
The answer is yes:
For Jack, he will only end up at the decision point if he observes 0 red balls. Assuming a large number of rounds are played, if he naively applies Bayesian reasoning he will conclude P(red) is very close to 0 and choose option 1 (another round of picking a ball). This is clearly irrational, because it will always result in option 1 being chosen regardless of the true probability and of his prior[1]. A better strategy is to stick with his prior if it is at all informative
For Jill, she will end up at the decision point regardless of whether she sees a red ball. Assuming a large number of practice rounds are played, in almost all worlds applying naive Bayesian reasoning will tell her P(red) is close to 20%, and she should pick option 2. In this case the decision is sensitive to the true probability, and she only loses out in the small proportion of worlds where she observes an unusually low number of red balls, so the naive Bayesian strategy seems rational
The point is that the population of Jacks that get the opportunity to make the decision is selected to be only those that receive evidence that imply a low probability, and this systematically biases the decision in a way that is predictable beforehand (such that having the information that this selection effect exists can change your optimal decision).
I think this is essentially the same objection raised by quila below, and is in the same vein as Jonas Mossâs comment on Tobyâs post (Iâm not 100% sure of this, Iâm more confident that the above objection is basically right than that itâs the same as these two others).
Itâs quite possible Iâm missing something in the paper, since I didnât read it in that much detail and other people seem convinced by it. But I didnât see anything that would make a difference for this basic case of an error in decision making being caused by the anthropic shadow (and particularly I didnât see how observing a larger number of rounds makes a difference).
A way to see that this is common-sense irrational is to suppose itâs a coin flip instead of a ball being drawn, where itâs very hard to imagine how you could physically bias a coin to 99% heads, so you would have a very strong prior against that. In this case if you saw 30 heads in a row (and you could see that it wasnât a two-headed coin) it would still seem stupid to take the risk of getting tails on the next round
The linked blog post says that starting a collaboration with other funders was one of OPâs goals for this year (quote from the section on 2024 goals from another blog post):
Weâre also aiming to experiment with collaborating with other funders by creating a multi-donor fund in an area that we think is particularly ripe for it. Weâll have more news to share on that later this year.
Which, from the wording and the timeline I assume was essentially referring to LEAF project. Is this a direction OP (perhaps inspired by this argument about PEPFAR?) increasingly wants to go in with other projects? And do you know if there are other collaborations like this in the pipeline?
Weâre thinking of moving the Forum digest, and probably eventually the EA Newsletter to Substack. Weâre at least planning to try this out, hopefully starting with the next digest issue on the 23rd. Hereâs an internal doc with our reasoning behind this (not tailored for public consumption, but you should be able to follow the thread).
Iâm interested in any takes people have on this. Iâm not super familiar with Substack from an author perspective so if you have any crucial considerations about how the platform works that would be very helpful. General takes and agree/âdisagree (with the decision to move the digest to Substack) votes are also appreciated.