Fascinating discussion between the two of you here, thanks.
I have one comment: I donât think their welfare being exactly 0 should have negligible probability. If we consider an animal like the soil nematode, I think there should be a significant probability assigned to the possibility that they are not sentient, unless Iâm missing something?
Outsourcing the welfare estimates to Gemini seems like a risky move to me. Itâs a key part of the whole analysis, but is an extremely challenging question to begin answering. Whatâs the reason to expect Gemini to be able to do a good job of this, given the blind spots we know current AI models still have?
I tried pasting your prompt into ChatGPT, with research mode, and the 3 value estimates it gave were all positive, rather than negative: https://ââchatgpt.com/ââshare/ââ683f34e8-8088-8006-8ba4-b719d025ac45
If Iâm understanding right, this would flip all your conclusions on their head, and instead of trying to eliminate wild animal habitats, the top priority would be to increase them?
Such extreme sensitivity to highly uncertain quantities strikes me as a strong reductio ad absurdum argument against this approach to decision making on this kind of question. Otherwise we find ourselves oscillating wildly between âdestroy all natureâ and âdestroy all humansâ on the basis of each piece of new information, never being especially confident in either.
I like Bayesianism and expected value maximization as a framework for decision making under uncertainty, but when considering situations with enormous amounts of value described by extremely speculative probability estimates, I think we probably need to approach things differently (or at least adapt our priors so as to be less sensitive to these kind of problems). Something like Holden Karnofskyâs approach here (which Anthony DiGiovanni shared with me on a recent post on insect suffering).
Edit: Anthony DiGiovanni doesnât actually endorse Holden Karnofskyâs approach (see Anthonyâs comment below).
If you care about maximizing total welfare, then you will think adding extra people to the population is better until the point where the harm they do to others (via using up scarce resources) outweighs the good from creating the extra person. I donât know where that point is, but itâs not obvious that it is a point where âeveryone is suffering and starvingâ.
If you care about total welfare, then it is true that for any concievable state of the world, there is always a hypothetical better state of the world in which everyoneâs lives are only barely worth living, but the population is so large that the total amount of welfare is still higher. This is the repugnant conclusion.
The repugnant conclusion is a problem for people who claim to care about total welfare. But you can reject the repugnant conclusion without being forced to conclude that saving people makes the world worse. For example, person affecting views hold that an act can only be good or bad if it is good or bad for someone. Stopping someone from being born is not bad for anyone, because the hypothetical child does not exist. On the other hand, saving someone who already exists might still be seen as very good. The two things do not have to be treated as equivalent.
Person-affecting views have different problems (the non-identity problem, and giving non-transitive ranking of outcomes). But they avoid the repugnant conclusion without implying that saving people is bad.
Or perhaps Iâm missing the point of your question completely, and it is more practical than theoretical. Are you just getting at the practical concern that saving lives will increase the current population, and that at its current level this is a bad thing? Like Ian Turner said, this then becomes a complicated empirical question about what the actual effect of saving lives on the population is. I donât know if the answer to that is clear.
Also, even if you think adding an extra person by creating a new life does more harm than good on the margin in todayâs world, and you believe that saving a life increases the population, it does not necessarily follow that saving a life also does more harm than good, because saving lives also has other big effects (e.g. if someone who already exists dies then the family and friends who are left behind suffer greatly).
I think a non-paradoxical answer is: âa world with 10 billion people is better, but probably not twice as goodâ.
The reason it might not be quite twice as good is because of the resource issues you mention, that might decrease average welfare.
If the claim is that every moral theory is equivalent to ârule consequentialismâ, maybe you have more of a case. But âact consequentialismâ is very distinct I think.
I disagree, I think the difference is substantive.
A utilitarian form of consequentialism might tell Alice to save the drowning child in front of her, while it tells Bob to donate to the AMF, but despite acting differently, both Alice and Bob are pursuing the same ultimate agent-neutral aim: to maximize welfare. The agent-relative âaimsâ of saving the child or making the donation are merely instrumental aims. They exist only as a means to an end, the end being the fundamental agent-neutral aim that both both Alice and Bob have in common.
This might sound like semantics, but I think the difference can be made clearer by considering situations involving conflict.
Suppose that Alice and Bob are in complete agreement about what the correct theory of ethics is. They are also in complete agreement on every question of fact (and wherever they are uncertain about a question of fact, they are in agreement on how to model this uncertainty e.g. maybe they are Bayesians with identical subjective probabilities for every concievable proposition). This does not imply that they will act identically, because they may still have different capacities. As you point out, Alice might have greater capacity to help the child drowning in front of her than Bob does, and so any sensible theory will tell her to do that, instead of Bob. But still, there is an important difference between the case where they are consequentialists and the case where they are not.
If Alice and Bob subscribe to a consequentialist theory of ethics, then there can be no conflict between them. If Alice realises that saving the child is going to interfere with Bobâs pursuit of donating, or vice versa, then they should be able to figure this conflict out between them and come up with an agreed way of coordinating to achieve the best outcome, as judged by their common shared ultimate aims. This is possible because their shared aims are agent-neutral.
But if Alice and Bob subscribe to a non-consequentialist theory (e.g. one that says we should give priority to our own family) then it is still possible for them to end up in conflict with one another, despite being in complete agreement on the answer to every normative and empirical question. For example, they might each pursue an outcome which is best for their respective families, and this may involve competing over the same resources.
If i recall correctly, in Reasons+Persons, Parfit examines this difference around conflicts in detail. He considers the particular case of prioners-dilemma style conflicts (where each party acting in their own interests leaves them worse off than if they had cooperated) and claims this gives a decisive argument against non-consequentialist theories which do not at least switch to more agent-neutral recommendations in such circumstances (and he argues this includes âcommon-sense moralityâ).
I really appreciate you taking the time to write such a detailed reply to my comment, thank you! And for sharing additional reading material on these questions. What you say makes a lot of sense. I think this research is really important and fascinating work, and Iâm excited to hear about what progress you can make.
I understand you might not have time to engage in further back-and-forth on this, but I just wanted to elaborate a bit on the one part of my comment that I think is maybe not directly answered by your reply. This is the issue around how we can compare the utility functions of different individuals, even in principle.
Suppose we know everything there is to know about how the brain of a fly works. We can figure out what they are thinking, and how these thoughts impact their behaviour. Maybe you are right that one day we might be able to infer from this that âfly experience Aâ is roughly 10x more painful than âfly experience Bâ (based purely on how it influences their choices among trade-offsâwithout relying on any assumptions around signal strength). But once we have determined the utility function for each separate individual (up to an undetermined constant factor), this still leaves open the problem of how to compare utility functions between different individuals, e.g. how to compare âfly experience Aâ to âhuman experience Câ. The vN-M approach does not tell you how to do that.
With humans, you can fairly easily get around this limitation in practice. You just need to find one reference experience that you expect to be similar for everyone (which could be as simple as something like stubbing your toe) and then with that one bridge between individual utility functions established, a standard âunitâ of utility, all other comparisons follow. I can even imagine doing something like this to compare the welfare of humans with other non-human mammals or birds.
But by the time we consider insects, I find it hard to imagine how we would approach picking this âreferenceâ experience that will allow us to do the inter-species welfare comparisons that we want to do. Iâm still not sure Iâm convinced that the methods outlined here will ultimately enable us to do that. Itâs not just that the âsignal strengthâ assumption is âunprovenâ, it is that I am struggling to wrap my head around what a proof of that assumption would even look like. The correlation between pain intensity and signal strength in vertebrates presumably is based on within-individual comparisons, not between-individual comparisons, and assuming that a correlation in the first-type of comparison implies a correlation in the other still seems like a big extrapolation to me.
Haha, ok, fair enough, I was not expecting that response!
Your solution (and Karnofskyâs) sound very interesting to me. But Iâll need to read both links in more depth to properly wrap my head around it.
A few questions though:
Karnofskyâs worked example for applying their multi-model technique leads with: âdoes this action deviate greatly from ânormality?ââ Why is this not just a more formalized version of the appeal to absurdity heuristic?
Not everyone is a galaxy-brain philosopher who can come up with complex blogposts like those to explain why giving their wallet to a Pascal mugger is wrong, yet everyone gets the correct (presumably) answer to this thought experiment anyway. And I think most are getting there by using some kind of absurdity heuristic? I think that should count in favour of the usefulness of the appeal to absurdity heuristic! Really feels like thereâs a good galaxy-brain meme in this. (I get Iâm rolling back here on my early suggestion that we could abandon the absurdity heuristic as soon as just one person could come up with a solution to the problem of pascalâs mugger).
Back to the actual subject of this post: Do you think the approach outlined in your 2 links could be used as an argument against the overwhelming importance of insect suffering, at least for someone who was extremely uncertain about the likelihood of insect sentience or its intensity?
Would you give your wallet to a pascal mugger?
If yes: Guess what? I am a sorcerer from a parallel universe who has the ability to conjure arbitrary numbers of sentient beings into existence at will, and subject them to extreme torture. You tell me how unlikely you think this claim is. I will then threaten 10x the reciprocal of that number, unless you give me ÂŁ100. I can send you my details and we can arrange the transfer.
If no: How do you explain this other than by an appeal to absurdity? I would love to know the solution to this problem.
Unless or until we have a better solution to this problem than âthatâs absurdâ, then I think we have to allow appeals to absurdity, especially when used against an argument that bears some resemblance to this pascal mugger example, at least superficially.
This is a fascinating analysis, thank you for sharing on the forum!
Thereâs something that bothers me whenever I read this kind of argument though. And that is: you take for granted that it is possible to put units on pain, and that we should all already know what that means. My question to this is:
If I say âExperience A is 13x as intensely painful as Experience Bâ, then what does this statement mean?
Where does the number 13 come from and how can I interpret it? It is one thing to put an ordering on different experiences in terms of how painful they are, but it is something very different to put a scale on that ordering so that quantitative comparisons like the above make sense.
I really like the discussion of information theory and complexity in this post, considering the question: how many distinct pain states is it possible for an organism to distinguish, given their complexity? But as you acknowledge, even if you can determine this, you are still left with the problem of deciding what the range is between the bottom and top end of those distinguishable states. I agree with you that itâs not clear that a capacity to distinguish more states should necessarily translate into a capacity for experiencing more intense pain.
When you do discuss the intensity question and what that means, you assume that higher intensity pain corresponds to a greater âsignal strengthâ and will require âadditional process unitsâ, but this seems like a massive leap to me! Whatâs the basis for that assumption? Why couldnât one organism represent extremely intense pain with less processing energy than another organism? And I think we canât even start evaluating this question properly until weâve answered the âwhat do these units even mean?â question that I posed at the start of my comment.
If you define your units to be related to âsignal strengthâ or âprocessing unit countâ, then sure, your methodology is sound. But it would then no longer be clear that units defined in this way have anything to do with ethics.
The only way I know to rigorously define what âunitsâ of pain (or more broadly, of utility) mean, is the von Neumann-Morgensten utility theorem. Here, the definition of âExperience A is 13x as painful as Experience Bâ is that a rational decision maker would be indifferent between a 1â13 chance of receiving Experience A, and a certainty of receiving Experience B.
This is a definition that can be formalized, and most importantly for thinking about ethics, it is the definition that has most decision relevance. But it is also extremely difficult to figure out how we would go about applying this to primitive sentient organisms. The vNMM utility theorem doesnât even tell you how to calibrate the respective utility functions of two individual humans (it is undetermined up to linear rescalings), let alone how you would go about doing that between a human and a fly.
When comparing humans, we can plausibly assume that similar kinds of pain have similar value for different individuals, and calibrate their respective utility scales that way. When considering insects, Iâm not sure I know where to start, even after reading this fantastic post.
Iâm not sure if I agree with you or not, but I donât know why you were getting so downvoted for this comment (before I strong up-voted, just to balance things out).
I thought the karma system was supposed to be independent of agreement/âdisagreement? I want to see your side of the discussion explored in the comments. I donât think people should be downvoting this kind of objection!
Your point 1 seems like a very good question to me, and I would be interested to read the authorâs reply.
Your second point also seems like a reasonable response to the piece, and Iâm sure represents what a lot of people would feel, especially if not familiar with EA. The author did a good job of anticipating and responding to lots of potential objections, but I donât think directly addressed this âdoesnât this lead to absurd conclusions?â objection.
The whole argument does feel like it resembles a Pascalâs mugging, in the same vein as strong-longtermism. When you try to do expected value maximization using Bayesian subjective probabilities (e.g. around extinction risk or likelihood of insect sentience or intensity of insect experience), and then start considering situations with huge amounts of potential value, it does seem like a recipe for decision paralysis: âbut look how big these numbers are, you canât be that certain they donât matter, surely??â
Would be interested to hear from those whoâve disagreed with this, since I think Iâm just pointing out a mathematical mistake? Interested to be corrected if Iâve got something wrong.
Perhaps would help to give some example numbers. Suppose someone assigns, for an insect:
P(react vigorously given pain experienced) = 1
P(react vigorously given no pain experienced) = 0.5
(These numbers seem defensible to me)
This gives you a Bayes factor of 2, when updating your probability that pain is experienced after seeing evidence that insects react vigorously to some negative stimulus. This is not a âstrongâ update.
Interesting, thanks!
Even if youâre certain that AGI is only 5 years away and will eradicate all diseases, a lot of children are going to die of malaria in those 5 years. Donating to malaria charities could reduce that number.
Sure, I think Iâve seen that comment before, and Iâm aware Chollet also included loads of caveats in his initial write up of the o3 results.
But going from zero fluid intelligence to non-zero fluid intelligence seems like it should be considered a very significant milestone! Even if the amount of fluid intelligence is still small.
Previously there was a question around whether the new wave of AI models were capable of any fluid intelligence at all. Now, even someone like Chollet has concluded they are, so it just becomes a question of how easily those capabilities can scale?
Thatâs the way Iâm currently thinking about it anyway. Very open to the possibility that the nearness of AGI is still being overhyped.
I think this is a misapplication of Bayes rule.
What matters is not that the 1st scenario is much more lilely than the 2nd under the hypothesis that pain is experienced (it clearly is). The relevant question is whether the 1st scenario is much more likely under the hypothesis that pain is experienced than under the hypothesis that pain is not experienced (itâs relation to the second scenario is irrelevant, a red herring). And whether this is actually the case is much less clear.
This is what your footnote equation says too, so Iâm not disagreeing with that, but I think the way you presented the argument in the text hides this, and might lead someone to misunderstand what it is they are being asked to judge is âmuch more likelyâ.
You can make an evolutionary argument for why we would expect an animal to react âvigorouslyâ to sustaining damage, and it is not clear why this evolutionary explanation requires the pain to be âexperiencedâ. So someone could make an argument that the likelihood of scenario 1 is high under both hypotheses, in which case it should only cause a small change in your priors.
I thought the post was really interesting, thank you for sharing it! It has updated me towards thinking that thereâs a higher chance insects might be sentient. But I think things are still a lot more complicated than suggested by this reply.
Thank you for the detailed reply Jared!
It makes sense that including outcome_2 would risk controlling away much of any effect of veganuary on outcome. And your answers to those pre-empted follow up questions make sense to me as well!
But does that then mean my original concern is still valid..? There is still a possibility that a statistically significant coefficient for veganuary_2 in the model might not be causal, but due to a confounder? Even a confounder that was actually measured, like activism exposure?
This is a fantastic , clearly written, post. Thank you for writing up and sharing!
In the 3 models, why is outcome_2 not included as a predictor?
Iâm just trying to wrap my head around how the 3-wave separation works, but canât quite follow how the confounders will be controlled for if the treatment is the only variable included from wave 2.
For example, in the first model:
Suppose âactivismâ was a confounder for the effect of âveganuaryâ on âoutcomeâ (so âactivismâ caused increased âveganuaryâ exposure, as well as increased âoutcomeâ).
Suppose we have 2 participants with identical Wave 1 responses.
Between wave 1 and wave 2, the first participant is exposed to âactivismâ, which increases both their âveganuaryâ and âoutcomeâ values, and this change persists all the way through to Wave 3.
The first participant now has higher outcome_3 and veganuary_2 than the second participant, with all other predictors in the model equal, so this will lead to a positive coefficient for veganuary_2, even though the relationship between veganuary and outcome is not causal.
I can see how this problem is avoided if outcome_2 is included as a predictor instead (or maybe as well as..?) outcome_1. So maybe this is just a typo..? If so I would be interested in the explanation for whether you need outcome_1 and outcome_2, or if just outcome_2 is enough. Iâm finding that quite confusing to think about!
Thanks for sharing the original definition! I didnât realise Turing had defined the parameters so precisely, and that they werenât actually that strict! I
I probably need to stop saying that AI hasnât passed the Turing test yet then. I guess it has! Youâre right that this ends up being an argument over semantics, but seems fair to let Alan Turing define what the term âTuring Testâ should mean.
But I do think that the stricter form of the Turing test defined in that metaculus forecast is still a really useful metric for deciding when AGI has been achieved, whereas this much weaker Turing test probably isnât.
(Also, for what itâs worth, the business tasks I have in mind here arenât really âcomplexâ, they are the kind of tasks that an average human could quite easily do well on within a 5-minute window, possibly as part of a Turing-test style setup, but LLMs struggle with)
Sorry, you did say this in the other thread as well and I should have made that clear in my comment originally. Have now edited.