Thanks. This is a challenging response to reply to. (3) risks “proving too much” but it seems like a valid argument on its face.
ben.smith
I arrived here from Jay Shooster’s discussion about the EA community’s attitude to eating animals.
I wasn’t aware of the current scientific consensus about consciousness; this article was a good primer on the state of the field for me in terms of which theories are preferred. I do like your though and I think it’s an interesting challenge or way to approach thinking about consciousness in machines. I’ve typed out/deleted this reply several times as it does make me re-evaluate what I think about panpsychism. I believe I like your approach and think it is useful for thinking about consciousness at least in machines, but am not sure that “panpsychism” as a theory adds much.
Psychological or neurological theories of consciousness are implicitly prefaced on studying human or non-human animal systems. Thus, though they reckon with the cognitive building blocks of consciousness, there’s less examination of just how reductive your system could get and still have consciousness. Whether you’re taking a GWT, HOT, or IIT approach, your neural system is made up of millions of neurons arranged into a number of complex components. You might still think there needs to be some level of complexity within your system to approach a level of valenced conscious experience anything like that which you and I are familiar. Even if there’s no arbitrary “complexity cut-off”, for “processes that matter morally” do we care about elemental systems that might have, quantitatively, a tiny, tiny fraction of the conscious experience of humans and other living beings?
To be a bit more concrete about it (and I suspect you agree with me on this point): when it comes to thinking about which animals have valenced conscious experience and thus matter morally, I don’t think panpsychism has much to add—do you? To the extent that GWT, HOT, or IIT ends up being confirmed through observation, we can then proceed to work out how much of each of those experiences each species of animal has, without worrying how widely that extends out to non-living matter.
And then proceeding squarely on to the question of non-living matter. Even if it’s true that neurological consciousness theories reduce to panpsychism, we can still observe that most non-living systems fail to have anything but the most basic similarity to the sorts of systems we know for a fact are conscious. Consciousness in more complex machines might be one of the toughest ethical challenges for our century or perhaps the next one, but I suspect when we deal with it, it might be through approaches like this, which attempt to identify building blocks of consciousness and see how machines could have them in some sort of substantive way rather than in a minimal form. Again, whether or not an electron or positron “has consciousness” doesn’t seem relevant to that question.
Having said that, I can see value in reducing down neurological theories to their simplest building blocks as you’ve attempted here. That approach really might allow us to start to articulate operational definitions for consciousness we might use in studying machine consciousness.
It’s worth checking out this very much ongoing twitter thread with Lamme about related issues.
https://mobile.twitter.com/VictorLamme/status/1258855709623693325
Yes, I tend to think that any one individual’s impact on the world around them probably balances out roughly neutral.
So I don’t use the argument that your own children might do a lot of good for the world and therefore you should raise children. That seems too speculative. And so the more known direct impact of having children on your own happiness and their happiness balances out the very speculative, almost entirely uninformed prior of the indirect effects having children might lead to.
Where you have a clear idea of a high and direct impact career that would be difficult to pursue were you to have children, then yes that might win out. Again, direct impacts are important, indirect impacts I think are so speculative that they probably don’t count for much.
As for earning to give, this is another challenge to my argument. I am sceptical that someone who really wants to have children will be happy in the long term sacrificing that for earning to give and this I’m sceptical that their commitment will be sustained and thus it may not be particularly impactful anyway, vs some compromise between personal desires and earning to give that is sustainable over decades.
That’s pretty speculative on my part but maybe borne out by observations made by 80k on people who enter morally neutral, high impact careers just to earn to give.
A few random thoughts from a researcher with a background in psychology research:
One driver for preferences for LSTs or eudaimonia frameworks for SWB is an intuition that solely focusing our well-being concerns on happiness or affect would lead us to conclude that happiness wireheading as a complete and final solution, and that’s intuitively wrong for most people.
Because psychologists are empiricists, they don’t spend too much time worrying about whether affect, life satisfaction, or eudamonia are more important in a philosophical or ethical sense. They are more concerned about how they can measure each of these factors, and how environmental (or behavioral or genetic) factors might be linked to SWB measures. To the extent there is psychological literature on the relative value of SWB measures, I think most of it is simply just trying to justify that it is worth measuring and talking about eudamonia at all, as eudamonia is probably the least accepted of the three SWB measures.
Working out the relative importance of SWB measures seems to me to be solely a question of values, for moral philosophy and not psychology, so I am glad that you, as a moral philosopher, are considering the question!
Finally, a bit of an aside, but another area where I would like to see more moral philosophers, psychologists, and neuroscientists talking is the relative importance of positive vs. negative affect. From a neuropsychological point of view, positive and negative affect are qualitatively different. Often, for convenience, researchers might measure a net difference between them, but I think there are very good empirical reasons that they should be considered incommensurable. All positive affect shares certain physical neuroscientific characteristics (almost always nucleus accumbens activity, for instance) but negative affect activates different systems. If these are really incommensurable, again we need to look to moral philosophers to be think about which is more important. This could be important for questions in moral philosophy (e.g., prior existence vs. total view) and in EA particularly: a strong emphasis on the moral desirability of positive affect might lead us towards a total view (because more people means more total positive affect) whereas balancing negative and positive affect could lead us towards a prior existence view (fewer people means less negative affect but also less positive affect), and a strong focus on avoidance of negative affect could even lead to a preference for the extinction of sentient life.
Yes you’re right.
I will try a slightly different claim that links neuropsychology to moral philosophy then. If you think maximizing well-being is the key aim of morality, and you do this with some balance of positive and negative affect, then I predict your balance of positive and negative affect at least as an empirical matter will change your ideal number of people to populate the Earth and other environments with in the total view.
Maybe it’s too obvious: if we’re totally insensitive to negative affect, then adding any number of people who experience any level of positive affect is helpful. If we’re insensitive to positive affect then total view would lead to advocating the extinction of conscious life (would Schopenhauer almost have found himself endorsing that view if it was put to him?). And there would be points all along the range in the middle that would lead to varying conclusions about optimal population. It might go some way to making total view seem less counterintuitive.
I second this question. Intuitively, your argument makes sense and you have something here.
But I would have more confidence in the conclusion if a False Discovery Rate correction was applied. This is also called a Benjamini-Hochberg procedure (https://en.wikipedia.org/wiki/False_discovery_rate#Controlling_procedures).
In R, the stats package makes it very easy to apply the false discovery rate correction to your statistics—see https://stat.ethz.ch/R-manual/R-devel/library/stats/html/p.adjust.html. You would do something like
p.adjust(p, method = “fdr”, n = length(p))
where p is a vector/list of all 55 of your uncorrected p-values from your t-tests.
Right now, the field is focusing on doing its empirical work better—the “open science” movement. I think that social scientists do engage in what we call “theoretical” work, but it is generally simply theories about how things empirically work (e.g., if religion is unique in its ability to produce high eudamonia for a large number of people, how can we conceptualize it as a eudamonia-producing system? Or which systems in the brain are responsible for production of pain experience; how is physical pain related to other forms of emotional pain?).
A fair number of us are probably logical positivists to a degree, in that we don’t want to go near a theoretical question with no empirical implications. That is a real shame. But to me, it just seems like theoretical values questions are outside of the domain of “social science” and in the domain of “humanities”. And one good reason to continue specialising/compartmentalizing like that is that many social scientists are just crap at formulating a clearly-articulated logical argument (try to read a theory in a psychology paper in the latter half of the Intro where they formulate hypotheses from their theory; compare the level of logical rigor and clarity with that from your philosophy papers). Collaborations between philosophers and psychologists are great (have you listened to Very Bad Wizards by Tamler Sommers and David Pizzaro? I only cite a podcast because honestly, I can’t think of actual research project collaborations) and collaborations should happen more, but honestly, it’s just difficult for me to even conceive of a psychologist trying to answer the question “what really matters more: eudamonia or net positive and negative affect?” because it seems to me at that point they’re doing humanities, not science.
I suppose there’s a whole history of that too; BF Skinner’s ‘behavioral turn’ really focused the field on what we can measure to the exclusion of anything that can’t be measured; it took a few decades just for the field to creep into thinking about things that could be in principle measured, or only indirectly measured (the ‘cognitive turn’) let alone thinking about entirely non-measurable values questions like “what ultimate moral end should we prefer?” Prior to Skinner, there was Freud and Jung and related theorists who did do theory, but I am not sure it was very good or useful theory.
To focus what I am trying to say: is there something we could gain from social scientists (particularly moral psychologists) theorising more about values that is unique or distinct from or would add to what philosophers (particularly moral philosophers) are already doing?
Sounds like a great attempt to fill a very salient gap! We will be discussing your project at the EA Auckland meetup tomorrow night (Tuesday 6.30pm utc+13). Let me know if you have any interest in zooming into chat.
Awesome, will love to have you! I’ll message you direct with a couple of details.
Hi Michael, I was searching for demandingness discussion on the forum here and found your comment.
Are you are aware of any discussion on this before or since your comment?
One recent article is: https://faculty.wharton.upenn.edu/wp-content/uploads/2020/02/Effectiveness-and-Demandingness.pdf which claims “EAs must endorse the view that well off people have at least fairly demanding unconditional obligations” to donate money to effective charities.It’s not my prior view at all. I think the most good will be done by people partaking in activities that are not particularly demanding at all (e.g., AGI Alignment research, plant-based meat research, well-being research, etc) rather than giving a substantial portion of income or making other demanding sacrifices. In order for the EA community to incentivize or show approval of such activity, people willing to do that research should be welcomed into the EA community whether or not they take a giving pledge or partake in any other more demanding activities.
But...those are just my private half-baked thoughts to date. I’d be interested in a conversation on this topic.
James Lindsay has already created something like this, except he is very much “anti-woke” and his dictionary reflects his perspective. https://newdiscourses.com/translations-from-the-wokish/
James Lindsay has already created something like this, except he is very much “anti-woke” and his dictionary reflects his perspective. https://newdiscourses.com/translations-from-the-wokish/
I think the main solution is to develop strong and resilient institutions. Areas for improvement could be:
Distributing power over more individuals rather than less
Making office-holding unappealing for people with narcissistic or sadistic intentions or tendencies by increasing penalties for abuses of office
More transparency in government to make it harder to abuse the office
More checks and balances
Educating the electorate and building a healthier society so that people don’t want to elect a narcissist
Thanks for your remarks. I’m looking forward to her full article being published, because I agreed that as it is, she’s been pretty vague. The full article might clear up some of the gaps here.
From what you and others have said, the most important gap seems to be “why we should not be consequentialists”, which is much bigger than just EA! I think there is something compelling; I might reconstruct her argument something like:
EAs want to do “the most good possible”.
Ensuring more systemic equality and justice is good.
We can do things that ensure systemic equality and justice; doing this is good (this follows from 2), even if it’s welfare-neutral.
If you want to do “the most good” then you will need to do things that ensure systemic equality and justice, too (from 3).
Therefore (from 1 and 4) it follows that EAs should care about more than just welfare.
You can’t quantify systemic equality and justice.
Therefore (from 5 and 6) if EAs want to achieve their own goals they will need to move beyond quantifications.
Probably consequentialists will reply that (3) is wrong; actually if you improve justice and equality but this doesn’t improve long-term well-being, it’s not actually good. I suppose I believe that, but I’m unsure about it.
Her choice to use multiple, independent probability functions itself seems arbitrary to me, although I’ve done more reading since posting the above and have started to understand why there is a predicament.
Instead of multiple independent probability functions, you could start with a set of probability distributions for each of the items you are uncertain about, and then calculate the joint probability distribution by combining all of those distributions. That’ll give you a single probability density function on which you can base your decision.
If you start with a set of several probability functions, with each representing a set of beliefs, then calculating their joint probability would require sampling randomly from each function according to some distribution specifying how likely each of the functions are. It can be done, with the proviso that you must have a probability distribution specifying the relative likelihood of each of the functions in your set.
However, I do worry the same problem arises in this approach in a different form. If you really do have no information about the probability of some event, then in Bayesian terms, your prior probability distribution is one that is completely uninformative. You might need to use an improper prior, and in that case, they can be difficult to update on in some circumstances. I think these are a Bayesian, mathematical representation of what Greaves calls an “imprecise credence”.
But I think the good news is that many times, your priors are not so imprecise that you can’t assign some probability distribution, even if it is incredibly vague. So there may end up not being too many problems where we can’t calculate expected long-term consequences for actions.
I do remain worrying, with Greaves, that GiveWell’s approach of assessing direct impact for each of its potential causes is woefully insufficient. Instead, we need to calculate out the very long term impact of each cause, and because of the value of the long-term future, anything that affects the probability of existential risk, even by an infinitesimal amount, will dominate the expected value of our intervention.
And I worry that this sort of approach could end up being extremely counterintuitive. It might lead us to the conclusion that promoting fertility by any means necessary is positive, or equally likely, to the conclusion that controlling and reducing fertility by any means necessary is positive. These things could lead us to want to implement extremely coercive measures, like banning abortion or mandating abortion depending on what we want the population size to be. Individual autonomy seems to fade away because it just doesn’t have comparable value. Individual autonomy could only be saved if we think it would lead to a safer and more stable society in the long run, and that’s extremely unclear.
And I think I reach the same conclusion that I think Greaves has, that one of the most valuable things you can do right now is to estimate some of the various contingencies, in order to lower the uncertainty and imprecision on various probability estimates. That’ll raise the expected value of your choice because it is much less likely to be the wrong one.
> Hope this helps.
It does, thanks—at least, we’re clarifying where the disagreements are.
If you think that choosing a set of probability functions was arbitrary, then having a meta-probability distribution over your probability distributions seems even more arbitrary, unless I’m missing something. It doesn’t seem to me like the kind of situations where going meta helps: intuitively, if someone is very unsure about what prior to use in the first place, they should also probably be unsure about coming up with a second-order probability distribution over their set of priors .
All you need to do to come up with that meta-probability distribution is to have some information about the relative value of each item in your set of probability functions. If our conclusion for a particular dilemma turns on a disagreement between virtue ethics, utilitarian ethics, and deontological ethics, this is a difficult problem that people will disagree strongly on. But can you even agree that these each bound, say, to be between 1% and 99% likely to be the correct moral theory? If so, you have a slightly informative prior and there is a possibility you can make progress. If we really have completely no idea, then I agree, the situation really is entirely clueless. But I think with extended consideration, many reasonable people might be able to come to an agreement.
Upon immediately encountering the above problem, my brain is like the mug: just another object that does not have an expected value for the act of giving to Malaria Consortium. Nor is there any reason to think that an expected value must “really be there”, deep down, lurking in my subconscious.
I agree with this. If the question is, “can anyone, at any moment in time, give a sensible probability distribution for any question”, then I agree the answer is “no”.
But with some time, I think you can assign a sensible probability distribution to many difficult-to-estimate things that are not completely arbitrary nor completely uninformative. So, specifically, while I can’t tell you right now about the expected long-run value for giving to Malaria Consortium, I think I might be able to spend a year or so understanding the relationship between giving to Malaria Consortium and long-run aggregate sentient happiness, and that might help me to come up with a reasonable estimate of the distribution of values.
We’d still be left with a case where, very counterintuitively, the actual act of saving lives is mostly only incidental to the real value of giving to Malaria Consortium, but it seems to me we can probably find a value estimate.
About this, Greaves (2016) says,
averting child deaths has longer-run effects on population size: both because the children in question will (statistically) themselves go on to have children, and because a reduction in the child mortality rate has systematic, although difficult to estimate, effects on the near-future fertility rate. Assuming for the sake of argument that the net effect of averting child deaths is to increase population size, the arguments concerning whether this is a positive, neutral or a negative thing are complex.
And I wholeheartedly agree, but it doesn’t follow from the fact you can’t immediately form an opinion about it that you can’t, with much research, make an informed estimate that has better than an entirely indeterminate or undefined value.
EDIT: I haven’t heard Greaves’ most recent podcast on the topic, so I’ll check that out and see if I can make any progress there.
EDIT 2: I read the transcript to the podcast that you suggested, and I don’t think it really changes my confidence that estimating a Bayesian joint probability distribution could get you past cluelessness.
So you can easily imagine that getting just a little bit of extra information would massively change your credences. And there, it might be that here’s why we feel so uncomfortable with making what feels like a high-stakes decision on the basis of really non-robust credences, is because what we really want to do is some third thing that wasn’t given to us on the menu of options. We want to do more thinking or more research first, and then decide the first-order question afterwards.
Hilary Greaves: So that’s a line of thought that was investigated by Amanda Askell in a piece that she wrote on cluelessness. I think that’s a pretty plausible hypothesis too. I do feel like it doesn’t really… It’s not really going to make the problem go away because it feels like for some of the subject matters we’re talking about, even given all the evidence gathering I could do in my lifetime, it’s patently obvious that the situation is not going to be resolved.
My reaction to that (beyond I should read Askell’s piece) is that I disagree with Greaves that a lifetime of research could resolve the subject matter for something like giving to Malaria Consortium. I think it’s quite possible one could make enough progress to arrive at an informative probability distribution. And perhaps it only says “across the probability distribution, there’s a 52% likelihood that giving to x charity is good and a 48% probability that it’s bad”, but actually, if the expected value is pretty high, it’s still strong impetus to give to x charity.
I still reach the problem where we’ve arrived at a framework where our choices for short-term interventions are probably going to be dominated by their long-run effects, and that’s extremely counterintuitive, but at least I have some indication.
There is an argument from intuition that carry some force by Schoenfield (2012) that we can’t use a probability function:
(1) It is permissible to be insensitive to mild evidential sweetening.
(2) If we are insensitive to mild evidential sweetening, our attitudes cannot be represented by a probability function.
(3) It is permissible to have attitudes that are not representable by a probability function. (1, 2)...
You are a confused detective trying to figure out whether Smith or Jones committed the crime. You have an enormous body of evidence that to evaluate. Here is some of it: You know that 68 out of the 103 eyewitnesses claim that Smith did it but Jones’ footprints were found at the crime scene. Smith has an alibi, and Jones doesn’t. But Jones has a clear record while Smith has committed crimes in the past. The gun that killed the victim belonged to Smith. But the lie detector, which is accurate 71% percent of time, suggests that Jones did it. After you have gotten all of this evidence, you have no idea who committed the crime. You are no more confident that Jones committed the crime than that Smith committed the crime, nor are you more confident that Smith committed the crime than that Jones committed the
crime....
Now imagine that, after considering all of this evidence, you learn a new fact: it turns out that there were actually 69 eyewitnesses (rather than 68) testifying that Smith did it. Does this make it the case that you should now be more confident in S than J? That, if you had to choose right now who to send to jail, it should be Smith? I think not.
...
In our case, you are insensitive to evidential sweetening with respect to S since you are no more confident in S than ~S (i.e. J), and no more confident in ~S (i.e. J) than S. The extra eyewitness supports S more than it supports ~S, and yet despite learning about the extra eyewitness, you are no more confident in S than you are in ~S (i.e. J).
Intuitively, this sounds right. And if you went from this problem trying to understand solve the crime on intuition, you might really have no idea. Reading the passage, it sounds mind-boggling.
On the other hand, if you applied some reasoning and study, you might be able to come up with some probability estimates. You could identify the conditioning of P(Smith did it|an eyewitness says Smith did it), including a probability distribution on that probability itself, if you like. You can identify how to combine evidence from multiple witnesses, i.e., P(Smith did it|eyewitness 1 says Smith did it) & P(Smith did it|eyewitness 2 says Smith did it), and so on up to 68 and 69. You can estimate the independence of eyewitnesses, and from that work out how to properly combine evidence from multiple eyewitnesses.
And it might turn out that you don’t update as a result of the extra eyewitness, under some circumstances. Perhaps you know the eyewitnesses aren’t independent; they’re all card-carrying members of the “We hate Smith” club. In that case it simply turns out that the extra eye-witness is irrelevant to the problem; it doesn’t qualify as evidence, so it it doesn’t mean you’re insensitive to “mild evidential sweetening”.
I think a lot of the problem here is that these authors are discussing what one could do when one sits down for the first time and tries to grapple with a problem. In those cases there’s so many undefined features of the problem that it really does seem impossible and you really are clueless.
But that’s not the same as saying that, with sufficient time, you can’t put probability distributions to everything that’s relevant and try to work out the joint probability.
----
Schoenfield, M. Chilling out on epistemic rationality. Philos Stud 158, 197–219 (2012).
- Why does (any particular) AI safety work reduce s-risks more than it increases them? by 3 Oct 2021 16:55 UTC; 48 points) (
- Why does (any particular) AI safety work reduce s-risks more than it increases them? by 3 Oct 2021 16:55 UTC; 48 points) (
- 7 Mar 2023 6:37 UTC; 4 points) 's comment on Model-Based Policy Analysis under Deep Uncertainty by (
- 6 Mar 2023 18:07 UTC; 3 points) 's comment on Model-Based Policy Analysis under Deep Uncertainty by (
Thanks! That was helpful, and my initial gut reaction is I entirely agree :-)
Have you had an opportunity to see how Hillary Greaves might react to this line of thinking? If I had to hazard a guess I imagine she’d be fairly sympathetic to the view you expressed.
I’ve been trying to evaluate career decisions about studying psychology and neuroscience. Do you think that studying motivation from a neuroscientific perspective is an effective way to contribute to AI alignment work? Do you think that-considering the scale of mental illnesses such as anxiety of depression-doing work on better understanding anxiety and depression is also highly effective?