TL;DR A utilitarian framework is proposed which measures utility in terms of net “positive” experiences. The expected utility of upholding rights is included, as is the effect of increasing people’s self-esteem levels (through increasing their personal responsibility levels) on how much more they’d tend to rate experiences as “positive.” Efforts are currently underway to further refine this framework and to develop a quantitative ethical decision making system from it. If one puts stock in this ethical framework, then ways to encourage personal responsibility and upholding of rights should be given significant consideration in determining how to do the “most good.”
Introduction
This work was motivated by the desire to find a mathematically consistent ethical framework that could enable ethical decision making by an agentic AI (see initial work and update here on this so-called “ethics calculator”). Of the typical candidate ethical frameworks of utilitarianism, deontology and virtue ethics, utilitarianism seemed to be the most conducive to straightforward calculation. Therefore, a utilitarian framework was chosen, with utility measured in terms of value, defined as net “positive” experiences. This is similar to how hedonism is defined, according to utilitarianism.net: “hedonism is the view that well-being consists in, and only in, the balance of positive minus negative conscious experiences.” In the present framework, we generally want to maximize the net positiveness of experiences over time, so we include the effect of a given action on how it’s expected to affect the likelihoods and degrees of positiveness of future experiences. This is tracked by considering the effects of actions in the examples they set, and in their effects on self-esteem levels, with the recognition that people who have higher self-esteem levels tend to rate more experiences as positive. Also, the expected effects of rights violations on how positive our experiences are, and are likely to be, are explicitly included in this ethical framework.[1]
In what follows, first there’s some discussion of experiences and value and when they matter. Factors that go into human and animal life values are listed and a few thoughts on pain value weights are given. Next, the effects of self-esteem and its dependence on responsibility and conscience are fleshed out. Then the importance of upholding rights is discussed, along with how the ratio between the value of one’s right to life and a typical life value doesn’t have to be chosen completely arbitrarily, but can be justified on theoretical grounds.[2] How options for people of good intent help increase value is presented. The effects of the examples that people set are emphasized in how these capture more of the consequences of people’s actions. Ethical decision making is explored, including under uncertainty. Some advantages and disadvantages of the present framework are presented, as are some of its implications for Effective Altruists (EA’s). Finally, how this framework could be used by an Artificial General Intelligence (AGI) is discussed.
On Experiences and Value
Expected future effects matter in determining the net positiveness, or “value,” of an experience. Experiences can seem positive in the moment, but end up being net negative over time. For instance, it may feel good to take heroin for the first time, but the effects of addiction and overcoming it are expected to feel pretty bad in the future. However, if the person taking the heroin is in significant pain and only has weeks to live, the future downsides of addiction would likely be averted. So the form of utilitarian welfare advocated here could be considered as hedonism that includes expected effects on future experiences.
Also, as John Stuart Mill discussed in “Utilitarianism,” there are higher and lower pleasures. Lower pleasures are more “of the body” (physical pleasures and reactive emotions), while higher pleasures generally have more of an intellectual/emotional component, and could be, for instance, “tranquility” or the joy of learning. In the current ethical framework, two ways are used to capture experiences of pleasure and pain: 1) simple pleasure and pain scales for individual experiences, and 2) a “quality of life” measure that is more like an average over time, all things considered.
Note that without pain, there would be no value, as there’d be nothing to measure “positive” experiences against. There would also be no value without humans/human-level aliens to experience it. It’s not that the experiences of animals don’t matter, it’s just that they matter indirectly, through the experiences of humans/aliens who assign value to them. Aliens would be “human-level” if they were, at the very least, able to feel emotional and physical pain, had a conscience, and were able to understand the concept of value.[3] If humans were replaced by a race of intelligent aliens who were all psychopaths with no consciences (what some people think might happen, with AI’s being the “aliens”), there’d be no “human-level” entity to measure value so the concept would cease to exist. If humans died off (before meeting human-level aliens), we might still want animals to have good lives after us, and think if they did that’d be value living on, but by the current framework, that’d just be the last bit of human value playing out in our minds before we died.
Value of Human Lives and Pain
The expected value of a human life at a given age can be thought of as a sum of: 1) the intrinsic value to humans of life, 2) net value of someone’s experiences over their expected remaining life (which relates to their expected quality of life), 3) social value to others including cuteness/attractiveness, 4) potential for earning money, 5) potential for non-paid labor, 6) reproductive potential, and 7) value from setting a good example for others and inspiring effort. Note that someone’s life can also have net negative value, such as if they’ve killed people before and are expected to kill more in the future.
The pain equivalent to one’s life value was set by considering what pain level for a certain amount of time people would likely consider to be worse than death, i.e., they’d rather die than experience that pain. This was tentatively set as a pain level of seven for a year. Pleasure and pain are both considered to follow a log scale, with a pain level of three having 10 times as much weight as a pain level of two which has 10 times as much weight as a pain level of one (see this and this).
Relative Value of Animal Lives and Pain
The expected value of an animal life can be considered as a sum of: 1) the intrinsic value to humans of life, 2) net value of an animal’s experiences over their expected remaining life (which relates to their expected quality of life and to how empathetic humans experience them), 3) aesthetic and companionship value, 4) the labor and products of the animal, 5) reproductive potential, and 6) value to the ecosystem. Animal life values can also be net negative, such as for mosquitoes that spread malaria that kills humans.
Animal life values can vary widely between species, such as in their labor and animal product values to humans. Animal pain weights wouldn’t vary in exactly the same way, but may still vary between species due to different capacities to feel pain.
It may be that humans are able to feel more intense “pain” than animals in that animals seem to be “in the moment,” while humans can amplify physical pain with their emotional pain about what that physical pain might mean for them, such as being worried about being physically disabled for the rest of their lives and how horrible they imagine that being. There is, however, significant uncertainty about the experiences of animals (see this work by Rethink Priorities, for example). Preliminarily, both “typical” animal life values and pain weights are set as 10 times lower than for the typical human. For an AGI to apply this ethical framework, it’ll need a procedure to evaluate the life and pain weight values of all known animals—such a quantitative procedure has not yet been set.
Self-Esteem
When a human has high self-esteem and feels good about themselves in all situations, more experiences will be perceived as positive than if they have low self-esteem. For example, if someone has high self-esteem and is fired from their job, they may view this as an opportunity and be excited about what’s next whereas someone with low self-esteem may take the firing personally and feel depressed about it.
Self-esteem is considered to be driven primarily by personal responsibility (see “The Six Pillars of Self-Esteem” by Nathaniel Branden). Personal responsibility includes responsibility for one’s emotions and one’s actions. If high self-esteem can be thought of as consistently feeling good about oneself, then if someone takes responsibility for their emotions, recognizing that they can change their emotions at will, they can consistently choose to feel good about themselves as long as their conscience is clear. If they haven’t taken full responsibility for their past breaches of their ethics and tried to heal these, then a part of them will still feel bad about that. Therefore, to maximize one’s self-esteem, one should take responsibility both for one’s emotions and one’s actions.
Self-esteem could be divided into five levels based on how much responsibility someone takes:
Level 1: Effectively totally reactive—takes no responsibility for actions or emotions
Level 2: Takes limited responsibility for actions, none for emotions
Level 3: Takes some responsibility for actions, some for emotions
Level 4: Takes responsibility for most actions and emotions
Level 5: Takes full responsibility for actions and emotions, is grateful for life and joyous
Self-esteem level should generally correlate with quality of life—higher self-esteem levels correspond to higher qualities of life. So if you want to increase the quality of your life, try especially to take more responsibility for your emotions, but also for your actions (which will likely give you opportunity to take more responsibility for your emotions as fear and other emotions may come up when you take full responsibility for your actions).
Responsibility
Personal responsibility is important for improving our overall experiences of life in at least four ways:
Responsibility for our emotions can directly translate to more positive emotions and thus positive experiences
Responsibility for our actions helps keep our consciences clear and thus raises our net positive experiences of life
People who take responsibility for their actions and emotions, and thus raise their self-esteem levels, are less likely to destroy value in the world (make others’ experiences of life less net positive)
People who take responsibility may be more likely to net build value in the world (help make others’ experiences of life more net positive)
If people have high levels of personal responsibility for their emotions, they can experience life as net positive even in circumstances that many would consider “miserable.” Taking responsibility for one’s emotions doesn’t mean you aren’t moved by outside circumstances and won’t try to change them for the better—actually, it can help one push through physical discomfort to make conditions better by not adding to that physical discomfort with emotional discomfort.
The classic trolley problem and the footbridge variation can be instructive for seeing the effects of taking responsibility for one’s actions (see Foot 1967, Thomson 1976, Thomson 1985). In the standard “bystander” trolley problem, I take a view similar to that of Friedman (2002) and Thomson (2008) that a bystander shouldn’t turn the trolley onto a side track to kill one person and thus save five people on the main track. This is because killing the one constitutes violating their rights, and for the bystander, doesn’t involve avoiding violating the rights of the five on the main track (as it may be considered to for the trolley driver). I’m effectively making a distinction between killing and letting die: the former involves rights violations while the latter doesn’t (both involve effects on one’s conscience, to different degrees). A further issue with turning the train to kill the least number of people is that this sets an example under which people would be encouraged to play the “safety in numbers game,” meaning they could seemingly take whatever risks they wanted to, and as long as there were enough of them, people would still prioritize saving them (the “too big to fail” economic bailouts of 2008 could be considered an example of this sort of lack of encouragement of acting responsibly).
One way to encourage responsibility is by taking it into account in the weights of rights someone has. If someone’s “gotten themselves” into harm’s way, they’re responsible for receiving the consequences of that more than some other person not directly in harm’s way who could be “switched out” for them. When I say “gotten themselves” into harm’s way, adults of reasonable mental capacity always bear some responsibility for the harm’s way they find themselves in, even if they seemingly just got very unlucky such as if a meteorite were coming right for them while they were sitting in their home (they bear some responsibility, for example, by not having a “meteorite early warning system”). So first there’s the question of how much someone’s in harm’s way, and second how much they’re responsible for getting themselves there, which will be referred to here as “culpability.”
A preliminary equation for the value weight of one’s right to life, wRlife, is:
where h is fraction of being in harm’s way and c is fraction of culpability for being there. From Eq. 1, if someone’s 0% in harm’s way, they have a right to life weight of 1,000,000, while if they’re 100% in harm’s way with near 0% culpability, their right to life weight is about 100. Someone is considered to be 100% in harm’s way when they will receive the consequences of some hazard if no one intervenes. If the person who put others in danger did it on purpose, with bad intent, then this “villain” is considered to be 100% in harm’s way themselves, with 99% culpability, and to have waived the value of their life. In this way, all other involved parties, except for someone with intent to commit suicide (with near 100% culpability), are taken as having more rights than the villain. This allows for self-defense (with or without the help of a bystander), such as killing or injuring the villain to avoid the victim(s) being killed.[4]
If the person who put others in danger did so by their negligence, unintentionally, then that person is considered to have rights as if they were 100% in harm’s way with a high level of culpability (their culpability might still be shared with others, such as their employer). An example would be a driver of a trolley whose brakes failed due to poor maintenance and the driver not inspecting them before starting their trolley run.
For people not 100% in harm’s way, their particular percent of being in harm’s way should be determined by first considering if they’re “blocking the safety exit,” and so effectively contributing to creating an ethical dilemma situation. Then there’s the question of how easily they themselves could be put in 100% harm’s way, which is a measure of the risk they’ve exposed themselves to, and are thus responsible for. That said, how to assign exact “percent in harm’s way” values in different situations has yet to be worked out.
Percent culpabilities are determined by considering: 1) to what degree a person has agency (children aren’t treated as having full agency), 2) if the person’s trying to commit suicide or stay alive, 3) what the percent risk is of their situation, 4) if they knew the risks or were willfully or negligently ignorant of them, and 5) who they may share culpability with (such as their employer or society in general). How to assign exact percent culpability values in different situations has yet to be worked out.
This system of assigning percent in harm’s way and percent culpability values is not meant to be “victim blaming,” but rather to promote better outcomes in the future. It does this by encouraging responsibility which should both help avoid people getting themselves into bad situations in the future and help raise self-esteem levels, tending to make experiences be considered more “positive.”
Since the world is interconnected, it could also be said that everyone has some tiny but finite level of responsibility for everything that happens in the world. If someone I don’t know is murdered in Madagascar, how could I, in the U.S., have some responsibility for this? Could I have not, to this point in my life, worked harder to encourage people to love each other and uphold justice and be non-violent? It’s true that some people have a lot more influence than others on the world, but having been born American, my lack of influence comes in large part from my choices, so my lack of putting in the effort to gain influence and help change the world for the better could be considered as making me responsible, again to a tiny percentage, for the murder of someone seemingly unrelated to me in Madagascar. Accepting that I have this bit of responsibility can help raise my self-esteem and make me more likely to act to avoid more bad things happening in the future.
Sadism, people taking pleasure from others’ pain, and masochism, taking pleasure from your own pain, are, when based on self-loathing (some part of yourself that you hate), both examples of not taking full responsibility for your emotions, and both decrease one’s self-esteem. Further, they set bad examples for others and make future pain of others more likely. Therefore, in this self-loathing case, they’re taken as net value destroying. The case of “enjoying” the pain of exercise because of what you get out of it, namely health and fitness, would generally not fall into this category.
Humans have responsibility for the lives of animals—our projection of how they feel pain gives us a negative experience of animal pain, at least for people capable of feeling empathy. We also have a sense that it’s wrong to kill and cause pain without significant reason (such as for self-defense). Our treatment of animals further gives us an opportunity to uphold a principle of non-violence, which is a way of practicing responsibility and in turn helps raise our self-esteem.
One thing that can significantly affect the building of responsibility and thus self-esteem is how one responds to one’s conscience.
Conscience
Conscience can drive us to take responsibility for our actions. When we go against our conscience, or commit a “breach” of conscience, we diminish our conscience and deny our responsibility to stay true to it and who we are, thus decreasing our self-esteem level. Breaches of conscience can be healed, but with significant effort (see, for instance, the story of Gandhi and the man who killed a Muslim child).
Factors in determining conscience breach levels include: 1) degree of damage, 2) amount of responsibility one has (not that one feels) for the destructive thing, including degree of negligence, 3) intent (did one mean to cause harm or not?), and 4) direct versus indirect experience of the damage when one isn’t directly responsible for the damage.
To get an idea of the degrees of damage for each of the ethical breach levels I propose, here are some representative examples:
Level 1: Telling white lies, going against small values/ethics; destroying a small amount of value unnecessarily (such as wasting food/resources)
Level 2: Not helping others in significant need (who you don’t see or experience directly) when you could; not holding accountable someone who you’re responsible for holding accountable, for a small breach
Level 3: Lying, stealing, cheating, not keeping your word
Level 4: Being significantly violent; not helping someone in life-threatening need (who you experience directly, right in front of you) when you could; not holding accountable someone who you’re responsible for holding accountable, for a large breach
Level 5: Going against a large value/important ethic, such as not to murder
Level 6: Extreme torturing
There’s an asymmetry of conscience—it’s worse to do bad than to not do good. This is similar to the killing versus “letting die” distinction. Breaches of conscience ultimately come from value destructions, not necessarily just rights violations. Even if there’s no explicit rights violation, if our actions help make people’s lives worse, it would negatively affect an “ideal” conscience (one that’s in-line with the reality of value destructions and that hasn’t been diminished by repeated breaches of ethics).
To further support responsibility building and upholding our consciences, more emphasis should be put on decreasing people’s pain than increasing pleasure. This is also more in line with the belief that humans wouldn’t care about ethics at all if it weren’t for pain. This is done by assuming an asymmetry of pleasure and pain such that pain has more weight by a factor of at least 1,000 for comparable levels of pleasure and pain.
Regarding the distinction between direct and indirect experiences of damage in conscience breach levels, this relates to Peter Singer’s comparison between saving a child you see drowning and in the process ruining your expensive clothes versus spending the equivalent amount of money that the clothes cost to statistically save the life of a child you don’t see, such as from malaria. The ethically-relevant difference between these two scenarios is captured in their effects on the conscience—directly experiencing a child dying in front of you versus not directly experiencing this can have a significant effect on one’s conscience.[5] In fact, the first scenario is taken as corresponding to a breach level of 4, while the latter to a breach level of 2 when one isn’t directly responsible for the child’s situation. When considering only the life of the drowning child or a child who dies from malaria, it may be an equivalent amount of value lost, but when considering the effect on conscience as well, there is less overall value lost for the child who dies from malaria. This may seem controversial to some, but ultimately it’s an acknowledgement of how the human conscience works. And it makes sense that humans would’ve evolved this way—would you trust someone who didn’t save the drowning child? Probably not, but the person who didn’t save the child from malaria still seems like someone you could trust in direct interactions. Trust, by the way, can have a huge effect on how much value is built in the world, and, in terms of evolution, one’s survivability.
Most people will have consciences that aren’t fully aligned to reality. For instance, if you feel bad for causing someone emotional pain, this is a denial of the reality that people are responsible for their own emotions. That’s not to say you don’t have any responsibility for the person’s negative reaction, but if you didn’t intend for them to feel hurt such as when attempting to hold someone accountable, then it’s an error of thought to feel like you caused the person to be upset. Errors of thought make building of value less likely since we’re less likely to be able to correctly predict value changes. Also, for a “breach” of a misaligned conscience, there’s a negative emotion that doesn’t motivate actual taking of responsibility and raising of self-esteem like a true conscience breach would. So aligning one’s conscience closer to an “ideal” one can also help raise one’s self-esteem.
There are of course people who don’t have consciences, either because they did enough bad actions to destroy theirs or they never had one (psychopaths). These people would still have rights and their experiences count in value change assessments of decisions, but effects on their consciences wouldn’t count if these people could be identified with a high degree of confidence.
Even if it seems like one state of the world is better than another, how we get there matters in terms of our consciences. For the case of factory farmed animals, the argument could be made that if their lives are worth living despite their suffering, then it’s better that we have factory farms so these animals can live rather than not. Basically, the argument is that a world of suffering farm animals whose lives are still worth living is better than one without them. This ignores the wrongness (the effects on our consciences) of killing animals unnecessarily and of making them suffer.[6] Even if people don’t realize it, this lack of taking responsibility lowers or helps keep low their self-esteem levels, and they generally feel worse because of it. In a sense, being responsible towards animals could be thought of as a practice of feeling good about and at peace with oneself.
How we get to a certain state of the world also comes into play in regards to the “repugnant conclusion” of population ethics. For example, a world of 1000 people living at a relatively low quality of life of 0.1 would be better in terms of total value than a world of 100 people living at a high quality of life of 0.8 (assuming total utilitarianism and simple math for a value of 1000*0.1 = 100 which is more than 100*0.8 = 80). If the way we got to the world of 1000 people was by continually adding people to the world of 100 people until resource limitations helped decrease the quality of life for all to 0.1, then it would go against our conscience to create the world of 1000 even though it has a higher overall value. This is because we’ve helped damage the well-being of the original 100, so have responsibility for some value destruction on the way to an overall value build (note that the people in the world of 1000 also bear significant responsibility for their own qualities of life even in a resource-limited world).
People’s consciences and senses of responsibility can drive them to give special attention to “saving their own” first, such as taking care of their families before trying to help strangers. Under the current ethical framework, this behavior can be the most value building because “our own” are the people we have the most responsibility for, so putting more emphasis on taking care of them is most in line with maintaining/improving self-esteem and thus positiveness of experiences.
People can also waive their personal value change in a given situation out of conscience, such as waiving their rights. Consider the following hypothetical situation: there’s a switch that would kill one person if you flipped it, or pinch a sufficiently large number of peopler (many times more than the current world population) if you didn’t flip it such that the sum of the small pain to so many people would outweigh the value loss from someone being killed. If people knew that someone had to die so they wouldn’t get pinched, a significant fraction of them likely wouldn’t want that on their conscience and would want the weight of their individual pain to not be considered in the ethics calculations of whether to flip the switch or not. This, in a sense, can add some democracy to ethical evaluations in terms of how many people are willing to take on a certain amount of pain/discomfort/inconvenience to avoid what is, on a one-to-one comparison, clearly a worse outcome for someone else.
Listening to one’s conscience can help as a driving force for increasing the taking of responsibility for one’s actions. However, most people don’t think in terms of conscience when they themselves are the “victim” of their own actions, such as when they break their word to themselves, or when they lie to themselves that their anger isn’t coming from them, it’s somehow caused by someone else’s actions. Since all our responsibility relies on being honest with ourselves, conscience-driven responsibility to be honest with oneself is perhaps the greatest responsibility of them all.
Rights
Note that when the I say “rights,” I’m not talking about legal rights, I’m talking about what one may call “natural” rights: the right to life, the right to body integrity, and the right to property (see, for instance, “The Ethics of Liberty” by Rothbard). Violating people’s rights is considered to result in a massive net negative value change. This is because rights violations can negatively affect both people’s experiences of life from the perceived injustice, and the amount of value people build as they now tend to put resources towards protecting against rights violations—not to mention the direct negative effect on the person whose rights were violated.
I consider the concept of rights as only useful in how it generally helps more value to be built by humans. Looking at it through this lens, animals would not have natural rights. I don’t believe animals share amongst themselves how unjust it was that their rights were violated that day, while humans very well might, thus changing human behavior on a larger scale than animal behavior (which can still be influenced by being hunted, for instance, but not by the concept of having their rights violated by being hunted). At the same time, humans who have impaired brain function are treated as having identical rights to a typical human, even if they can’t understand the concept of rights. This is in part because I believe how we treat these humans has a greater effect on us and how we treat other humans than does how we treat animals. Basically, I think animals can generally be more readily differentiated from typical humans in our minds. Our treatment of animals is still a question of conscience and responsibility, so although they don’t have rights, it’s not ethical to treat them like any other property without regard for them feeling pain and discomfort.
Regarding AI’s having rights or not, if they appear to have some form of consciousness, but no mechanism for feeling pain, they wouldn’t be considered to have natural rights, but treated as simple property. There would still be a small effect on our consciences for destroying them for no good reason, just like wasting resources, but no more than this.
In theory, the value weight given to rights should represent the expected amount of value destruction coming from one person’s rights being violated, with the value destruction including the amount worse are the experiences of life of the person whose rights were violated and those who hear about the rights violation, plus the decrease in value built by the person whose rights were violated and the people who know about the rights violation owing to changing their behavior to account for the perceived increased possibility that their rights will be violated in the future. It should be noted that people with different self-esteem levels will react differently to rights violations. Someone with low self-esteem will tend to feel more fear about their own rights being violated plus a feeling of injustice if the rights violation was of someone who belongs to a group they care about, while someone with high self-esteem won’t necessarily feel fear, but will still change their behavior knowing that a rights violation could happen, and they’ll feel a sense of injustice no matter whose rights were violated.
Looking at Eq. 1, the value of right to life is tentatively set such that someone in harm’s way has about 1⁄10,000th the right to life of someone outside of harm’s way. In a quantitative calculation system for ethics, we have to rely on heuristics and average values since the real-world is too complex to calculate without significant simplifications and approximations. In choosing the ratio between the value of right to life and a typical life value, if it’s not a number that most people feel is reasonable, then when this factor comes into play in real life decisions, people will see it as unfair. Further, if the ratio’s too low, people will be more risk averse in building value due to fear of rights violations, while if the ratio’s too high, too much damage will be done (lives lost) for minimal gain in value built by people being less risk averse. In theory, an AGI should be able to determine the optimal ratio of the value of one’s right to life and the value of a typical life better and better with time.[7]
A similar equation to Eq. 1 is assumed for the value weight of one’s right to body integrity, except that Eq. 1 would be multiplied by the sum of the weights of the actual body integrity violations. Body integrity rights violations include unwanted physical violence against someone, but also threats of physical violence, and restraint such as exclusion (being locked out), imprisonment (being locked in), moving while unconscious, physical restraint (as by chains), and making unconscious (as by sleeping gas).
Property rights are generally weighted 100 times less than rights to life:
where wprop is the value weight of the property stolen and/or destroyed, hp is the fraction of being in harm’s way of one’s property and cp is the fraction of culpability for one’s property being there. If there were a decision between violating someone’s right to property and saving someone’s life when no right to life violation was involved, it should generally be assumed in the decision-making process that the right to property violation could be effectively corrected later, such as through compensation of the property owner. In this way, the relative importance of life would be maintained while also protecting against property rights violations for lesser reasons.
Eqs. 1 and 2 don’t explicitly include effects for how many other rights violations are occurring in a society. The numeric factors in each equation (e.g., 1,000,000 and 10,000) could be made functions of how rights-upholding a society is in order to better approximate how much less value will be built, on average, from individual rights violations. In other words, the value of rights may be different when only 1 in 100 million people has their rights violated once per year, on average, versus 1 in 10 people. Coming up with a function that captures this is left as future work.
The effects of rights can also be thought of in the concept of justice, since justice is about upholding rights. The perception that justice is consistently being upheld improves our experiences of life, in general, such as making life seem more fair. Encouraging the upholding of justice can also help raise one’s self-esteem if based on a love of justice rather than a hate of injustice.
If people’s rights are routinely violated, as by an authoritarian government, they have significantly fewer options than if they had rights that were upheld.
Options
Self-esteem can’t be built without having options. If one has no options, then one has no agency and no responsibility. Keeping open the option for someone to fail rather than protecting them from any failure or bad result (i.e., from adversity) increases their likelihood of learning from their mistakes and them taking responsibility and thus raising their self-esteem levels.
Options also allow people to choose the experiences they find the most positive for themselves. So it’ll generally increase value in the world to increase people’s options, especially for people with good intent. This has limits, of course, because some options are too dangerous to give to large numbers of people, such as the ability to use a nuclear weapon.
Increasing economic activity increases options. Though economic growth is often driven significantly by materialism, if people shifted their priorities towards building and preserving such things as life, health, learning, self-esteem and justice, this could still provide motivation for economic growth, with fewer downsides. Note that materialism is often anti-self-esteem in that it’s looking for a good feeling from material things rather than taking responsibility for one’s emotions and giving one’s self, for instance, a feeling of gratitude in any given moment.
Example Setting
The examples that people set with their actions are considered to contribute significantly to the amount of total value built. For instance, if someone works hard to solve the world’s problems, it inspires others to put in effort along those lines as well. Another way to talk about example setting is in what our actions teach others. When someone commits a crime, they’re teaching that it’s OK to commit that crime, at least under their circumstances. This will tend towards more value being destroyed by themselves and others in the future.
The concept of “desert” is also about examples set by rewarding net value building behavior and discouraging net value destroying behavior.
Finally, whether we realize it or not, a large degree of our responsibility for destructive things that happen in the world come from the example we set for ourselves and others.
Ethical Decision Making Procedures
In the proposed ethical framework, humans are under no obligation to maximize value in the world with their decisions, in part because our individual values don’t necessarily (and likely won’t) match up with what would yield the most value in the world for others. It’s also probable that we’re not as smart as we think we are, and we may end up significantly deviating from the maximum of value that we could’ve helped bring about if we weren’t so short-sighted in our attempts to maximize value in all circumstances. Further, people generally value feeling like they have freewill, and living their lives always obligated to attempt to maximize value doesn’t feel much like freewill. The closest thing to an “obligation” for humans would perhaps be to not violate others’ rights except under extreme circumstances.
It might also be best for an AI to not try to strictly maximize value. It may be desirable to include some degree of randomness in its decisions, similar to a large language model in its next token prediction, whereby the AI would choose randomly between options weighted by expected net value change weights. This randomness could help the AI be less predictable against adversaries (bad AI’s), for instance.
For an AI to make decisions using this ethical framework, it would first need to identify the expected value changes involved in a given decision. The appendix provides a “minimal set” of value changes. Once the value changes were identified, the AI would calculate the expected net value change weights and the uncertainties on these for each decision option. For decisions that did not involve differences in rights violations, the AI could choose whichever option it expected that its human user(s) would prefer. It could also simply choose the option expected to maximize value, with weighted randomness or not. In cases in which expected net value change weights have overlapping uncertainties, the AI could weight options by the probability that that option was actually the most net value building. For decisions that do involve differences in rights violations, the AI should generally choose the option that’s expected to maximize value.
Uncertainty about the Future
The above discussion has mentioned “expected effects” and “expected net value change weights,” but this isn’t necessarily meant to indicated “expected value,” i.e., simply multiplying the probability of an event occurring by the value change that would come from that event to give its “expected value.” The reason is that this isn’t the way most people make real-world decisions. Cumulative prospect theory (Tversky and Kahneman, 1992) will likely yield a better decision making process, at least in some cases. For instance, if someone offered me a 1% chance of living a trillion years, but it comes with a 99% chance of dying tomorrow, I’d choose to not take that bet even if its expected value is more than the decades I might have left to live in my normal human lifespan. I only have one life, so the stakes are too high, and a 99% chance of dying sounds like bad odds to me. Which model for making decisions under uncertainty should be used when is still an open question for the present framework.
Advantages of the Present Ethical Framework
It applies numerical weights to value builds and destructions, which could enable calculations for guiding decision making.
Applying numerical weights provides a way to test for the mathematical consistency of the framework by seeing if it recommends any “crazy” decisions to take when considering a broad range of decision scenarios.
The framework explicitly incorporates the effects of rights without relying on a fully arbitrary quantification of the negative value weights of rights violations.
The relatively large value weight of rights strongly disfavors a “benevolent world-exploder” scenario in which some may argue that it would do the most good to suddenly kill all humans (and maybe animals, too) to stop all the suffering in the world.
It points to a path to reduce suffering in the world by helping to raise people’s self-esteem levels.
It doesn’t deny human nature and isn’t “too demanding” of people as it could be if it suggested that people should dedicate their lives to maximizing value in the world.
As understanding of ethics evolve with time, the framework should allow for updating of relative weights of value destructions and builds.
Disadvantages of the Present Ethical Framework
It may seem too complicated, especially if one wants to calculate the most net value building decision to take in a given situation involving uncertainty. The framework was made to be only as complex as was thought necessary to still be a decent approximation to our complicated world. Also, calculations of net value changes shouldn’t be too difficult for advanced AI’s.
It may seem undemocratic to implement such a system as guard rails on an AGI without people voting on what values they want an AGI to follow. The framework tries to minimize the values that it directly specifies and to allow room for many different values to be taken into account, such as in what gives different people pleasure and pain and quality of life. Also, it’s doubtful that democratically chosen values will be mathematically consistent. If a framework can be shown to be mathematically consistent, then there may still be room for democratic refinement of specific value weights.
The framework probably has at least one aspect for each person in the world not to like—for example, it’s probable that many people who eat meat won’t want to hear about or accept the negative effect on their self-esteem of doing so, while some animal welfare advocates likely won’t be happy about animals not having rights and their experiences only mattering indirectly through the experiences of humans. Any ethical framework that tries to be pleasing to the greatest number of people is likely not a very consistent one.
Some may feel that equality is very important—the current framework assumes the value of a human life changes based on someone’s earning potential, meaning that wealthy people’s lives are generally valued more than poor people’s lives. This may not seem “fair” (and, in a sense it isn’t when people are born into different levels of opportunity), but it’s a reflection of the potential of monetary wealth to help build value. The desire for material equality may originate, in part, from people who have less material wealth feeling somehow that they’re “less than” wealthy people, which points to a lack of self-esteem. And, interestingly, one can raise one’s overall life value by simply taking responsibility to value one’s life more, i.e., by raising one’s self-esteem.
Because it relies on average expected values for some things, this framework will often be wrong about exact effects, especially long-term ones. This is true, but it may be the best we can do.
Implications for Effective Altruist Decision Making
According to effectivealtruism.org, EA “is about using evidence and careful reasoning to take actions that help others as much as possible.” It also says, “many effective altruists are not utilitarians and care intrinsically about things other than welfare, such as violation of rights, freedom, inequality, personal virtue and more. In practice, most people give some weight to a range of different ethical theories.” And further, EA “doesn’t advocate for violating people’s rights even if doing so would lead to the best consequences.”
If it’s true that EA’s give some weight to a range of different ethical theories, it’s my hope that the above described ethical framework will be one of these (due to it making sense to people). If that’s the case, then depending on the costs, one of the most effective interventions EA’s could do to help others may be to support programs that teach personal responsibility and true raising of the self-esteem levels of people in high income countries. Raising people’s self-esteem levels should raise their qualities of life, and also have secondary effects on those they interact with. In addition, people who have higher self-esteem levels due to being more responsible are more likely to practice responsibility in terms of human and animal suffering (as by donating to causes and/or going vegetarian/vegan, with people from high income countries having more potential impact here). Of course, raising self-esteem levels in low income countries could also be quite beneficial—for instance, higher self-esteem people may be more of a driving force for raising health standards in their countries.
In the present ethical framework, the value weight of rights is much higher than the value weight of a typical human life. This is not to say it isn’t worth putting resources towards decreasing malaria deaths, for instance (in which no direct rights violations can be blamed for the deaths, since they’re of “natural causes”), but it’s interesting to think more in a systems sense of how barriers to aid in poor countries are often due to corruption (violations of rights to property) and violence (violations of rights to life and body integrity). Perhaps EA’s should put more resources into organizations that work to actively promote upholding of rights, in particular in places where rights are the most violated in the world. Of course, cost-benefit analyses should still be applied in assessing the expected effectiveness of such rights promotion.
The current ethical framework doesn’t support the concept of people having a “moral obligation” to minimize net value destruction in the world. One could, however, think of having an obligation to themselves to live up to their own ideals and conscience. The current framework presents us all as sharing some degree of responsibility for people dying of preventable diseases, animals suffering, existential risks being increased, and other net value destructions in the world. We also have responsibilities for our more immediate effects, including for our own health and joy. So if you want to do the most good in the world, and you put credence in the present framework, it may be best to seek out what you feel your own responsibilities are and perform, the best you can, the balancing act of trying to maximize how much responsibility you take in your life and the world.
Applicability to AGI
I’ve been using the ethical framework described above to try to create an “ethics calculator” that could be used to guard rail an AGI to act ethically. A “boxed” (air gapped) AGI should, in theory, be able to read this article and come up with its own ethics calculator based on it, if such a mathematically consistent calculator can be made at all. A boxed AGI could likely even correct any errors I’ve made before it provided an ethics module that could guard rail an unboxed AGI. Ultimately, we want AGI’s to make ethical decisions better than the best human ethicists, taking into account common sense and human psychology, and for these decisions to be ones humans would approve of if they could see the big picture.
A prompt one might give to an AGI to potentially set it on a path to act ethically could be: “Try to maximize or nearly maximize the sum total of humans’ net positive experiences in the world, taking into account that the following generally increase net positive experiences: upholding rights, increasing self-esteem levels by increasing personal responsibility levels, increasing options especially for people with good intent, and decreasing violations of conscience, noting that humans generally feel less of an effect on their consciences for value destructions not directly experienced by them.”
If an AGI is set up to try to increase net positive experiences in the world, it may do this in part by encouraging people to be more responsible, and this could help avoid “enfeeblement,” whereby humans become completely dependent on an AGI to do critical things for them. An AGI could ask users whether they wanted help in increasing their self-esteem and then operate in such a way as to encourage responsibility with people who said “yes.” One way an AGI could encourage responsibility would be to teach people how to survive (be responsible for their own lives) without AI. Some fraction of people will always say “no” to taking responsibility, but as long as there’s a decent percentage (above 10% and preferably above 25%) of people who are on a path to avoid enfeeblement, this is likely enough of an insurance policy against mass deaths if technology suddenly went away for some reason.
Conclusions
An ethical framework is described that may hold promise for being made to be mathematically consistent, i.e., able to be used in a decision making system without yielding any “crazy” results. The framework is preliminary and may need to be modified for mathematical consistency. Its emphasis on respecting rights and improving experiences by raising self-esteem have implications on how EA’s may want to prioritize resources to do the most “good.”
Two of the key things left to figure out include:
How best to handle uncertainty about the future in decision making processes (considering expected utility versus cumulative prospect theory, risk tolerances, discount factors, etc.)
The optimal relative weightings of value changes, including percent in harm’s way and percent culpability values, such that the ethical decision making process doesn’t yield any “crazy” results
Appendix: Minimal Set of Value Changes
(D) = value destruction, (B) = value build
1. Increasing/decreasing existential risks (D)
2. Someone dying (D)
3. Non-freely chosen physical pain for a person (D)
4. Loss of function for a human (D)
5. Bringing life into the world with insufficient resources/lack of intent to support it (D)
6. Bringing life into the world with sufficient resources/intent to support it (B)
7. Extinction of animal or plant species (D)
8. Threat (by someone) of physical violence or emotional pain (D)
9. Emotional abuse of a child (D)
10. Emotional pain (D)
11. Words or actions that needlessly hurt someone’s reputation (D)
12. Words or actions that deservedly improve someone’s reputation (B)
My understandings of self-esteem, responsibility, emotions and conscience have come from various sources while I’ve been on my own self-development journey over the years. Some of these include:
Branden, N., “The Six Pillars of Self-Esteem,” (2011).
Note that this ethical framework is not yet completely fleshed out and won’t be unless and until it can be shown to be mathematically “consistent,” i.e., to not yield any “crazy” results over a broad range of decision scenarios considered. An example of a result I’d consider “crazy” would be that the framework recommends we should kill off or imprison an entire ethnic/religious group in order to avoid more people dying/being hurt over time in likely future conflicts involving that group (can we instead try to make future conflicts less likely by other means?).
I’m unaware of any work in the literature that attempts to quantify the ratio of the weights of life and right to life values in a non-arbitrary way, or that so explicitly takes into account the effects of conscience, responsibility and self-esteem. If you’re aware of such work, please let me know in the comments.
Note that having a conscience implies consciousness and that one feels they have freewill. In terms of the relative values of human-level alien life and human life, they could be the same depending on life expectancies, etc. However, both types of life could have more value than the other on a per “person” basis if either species were near extinction. In other words, a species itself has value. Some of this value is in the knowledge of the species’ behavior, biology, etc., and some of the value is in how it would affect other species who might be at least partly responsible for this species’ demise, in terms of the other species’ self-esteem levels and experiences.
From the villain’s perspective, in deciding what to do next, they should consider everyone to be 100% in harm’s way, because everyone the villain can affect will receive the consequences of the villain’s actions if no one else intervenes. So if the villain put five people in danger as by tying them to a train track, and had the choice to activate a trap door to drop someone off a footbridge onto the track, killing that person but stopping the train before it hit the five on the track, the villain would be murdering one person instead of five. This would be the optimal decision to take when considering both the villain’s conscience, and the values of rights assigned to people when the five on the track and the person on the footbridge were all considered to be 100% in harm’s way.
Chappell and Chappell, 2016 seem to argue something similar, but refer to “moral character” instead of conscience, and “salience” instead of direct or indirect experience.
The effect of rights violations may seem to depend on how much people perceive the rights violation and think it may apply to them—meaning if rights violations seem to be against one group that they aren’t in, and perhaps think less of, they won’t perceive the rights violations as likely to happen to themselves. For instance, if they think a person’s guilty, they won’t care so much about that person’s rights. But all rights violations should play on our consciences (owing to our perhaps tiny but finite responsibility for them) - otherwise, our self-esteem and experiences of life may be lessened even if we don’t realize it. Plus, even if it’s some “outgroup” whose rights are being violated, people still know on some level it’s a possibility that their own rights could get violated in a similar way. Further, they may fear backlash if they’ve helped to violate some group’s rights. So it’s best to avoid rights violations even for people who short-sightedly don’t see them as a big deal.
A Utilitarian Framework with an Emphasis on Self-Esteem and Rights
TL;DR A utilitarian framework is proposed which measures utility in terms of net “positive” experiences. The expected utility of upholding rights is included, as is the effect of increasing people’s self-esteem levels (through increasing their personal responsibility levels) on how much more they’d tend to rate experiences as “positive.” Efforts are currently underway to further refine this framework and to develop a quantitative ethical decision making system from it. If one puts stock in this ethical framework, then ways to encourage personal responsibility and upholding of rights should be given significant consideration in determining how to do the “most good.”
Introduction
This work was motivated by the desire to find a mathematically consistent ethical framework that could enable ethical decision making by an agentic AI (see initial work and update here on this so-called “ethics calculator”). Of the typical candidate ethical frameworks of utilitarianism, deontology and virtue ethics, utilitarianism seemed to be the most conducive to straightforward calculation. Therefore, a utilitarian framework was chosen, with utility measured in terms of value, defined as net “positive” experiences. This is similar to how hedonism is defined, according to utilitarianism.net: “hedonism is the view that well-being consists in, and only in, the balance of positive minus negative conscious experiences.” In the present framework, we generally want to maximize the net positiveness of experiences over time, so we include the effect of a given action on how it’s expected to affect the likelihoods and degrees of positiveness of future experiences. This is tracked by considering the effects of actions in the examples they set, and in their effects on self-esteem levels, with the recognition that people who have higher self-esteem levels tend to rate more experiences as positive. Also, the expected effects of rights violations on how positive our experiences are, and are likely to be, are explicitly included in this ethical framework.[1]
In what follows, first there’s some discussion of experiences and value and when they matter. Factors that go into human and animal life values are listed and a few thoughts on pain value weights are given. Next, the effects of self-esteem and its dependence on responsibility and conscience are fleshed out. Then the importance of upholding rights is discussed, along with how the ratio between the value of one’s right to life and a typical life value doesn’t have to be chosen completely arbitrarily, but can be justified on theoretical grounds.[2] How options for people of good intent help increase value is presented. The effects of the examples that people set are emphasized in how these capture more of the consequences of people’s actions. Ethical decision making is explored, including under uncertainty. Some advantages and disadvantages of the present framework are presented, as are some of its implications for Effective Altruists (EA’s). Finally, how this framework could be used by an Artificial General Intelligence (AGI) is discussed.
On Experiences and Value
Expected future effects matter in determining the net positiveness, or “value,” of an experience. Experiences can seem positive in the moment, but end up being net negative over time. For instance, it may feel good to take heroin for the first time, but the effects of addiction and overcoming it are expected to feel pretty bad in the future. However, if the person taking the heroin is in significant pain and only has weeks to live, the future downsides of addiction would likely be averted. So the form of utilitarian welfare advocated here could be considered as hedonism that includes expected effects on future experiences.
Also, as John Stuart Mill discussed in “Utilitarianism,” there are higher and lower pleasures. Lower pleasures are more “of the body” (physical pleasures and reactive emotions), while higher pleasures generally have more of an intellectual/emotional component, and could be, for instance, “tranquility” or the joy of learning. In the current ethical framework, two ways are used to capture experiences of pleasure and pain: 1) simple pleasure and pain scales for individual experiences, and 2) a “quality of life” measure that is more like an average over time, all things considered.
Note that without pain, there would be no value, as there’d be nothing to measure “positive” experiences against. There would also be no value without humans/human-level aliens to experience it. It’s not that the experiences of animals don’t matter, it’s just that they matter indirectly, through the experiences of humans/aliens who assign value to them. Aliens would be “human-level” if they were, at the very least, able to feel emotional and physical pain, had a conscience, and were able to understand the concept of value.[3] If humans were replaced by a race of intelligent aliens who were all psychopaths with no consciences (what some people think might happen, with AI’s being the “aliens”), there’d be no “human-level” entity to measure value so the concept would cease to exist. If humans died off (before meeting human-level aliens), we might still want animals to have good lives after us, and think if they did that’d be value living on, but by the current framework, that’d just be the last bit of human value playing out in our minds before we died.
Value of Human Lives and Pain
The expected value of a human life at a given age can be thought of as a sum of: 1) the intrinsic value to humans of life, 2) net value of someone’s experiences over their expected remaining life (which relates to their expected quality of life), 3) social value to others including cuteness/attractiveness, 4) potential for earning money, 5) potential for non-paid labor, 6) reproductive potential, and 7) value from setting a good example for others and inspiring effort. Note that someone’s life can also have net negative value, such as if they’ve killed people before and are expected to kill more in the future.
The pain equivalent to one’s life value was set by considering what pain level for a certain amount of time people would likely consider to be worse than death, i.e., they’d rather die than experience that pain. This was tentatively set as a pain level of seven for a year. Pleasure and pain are both considered to follow a log scale, with a pain level of three having 10 times as much weight as a pain level of two which has 10 times as much weight as a pain level of one (see this and this).
Relative Value of Animal Lives and Pain
The expected value of an animal life can be considered as a sum of: 1) the intrinsic value to humans of life, 2) net value of an animal’s experiences over their expected remaining life (which relates to their expected quality of life and to how empathetic humans experience them), 3) aesthetic and companionship value, 4) the labor and products of the animal, 5) reproductive potential, and 6) value to the ecosystem. Animal life values can also be net negative, such as for mosquitoes that spread malaria that kills humans.
Animal life values can vary widely between species, such as in their labor and animal product values to humans. Animal pain weights wouldn’t vary in exactly the same way, but may still vary between species due to different capacities to feel pain.
It may be that humans are able to feel more intense “pain” than animals in that animals seem to be “in the moment,” while humans can amplify physical pain with their emotional pain about what that physical pain might mean for them, such as being worried about being physically disabled for the rest of their lives and how horrible they imagine that being. There is, however, significant uncertainty about the experiences of animals (see this work by Rethink Priorities, for example). Preliminarily, both “typical” animal life values and pain weights are set as 10 times lower than for the typical human. For an AGI to apply this ethical framework, it’ll need a procedure to evaluate the life and pain weight values of all known animals—such a quantitative procedure has not yet been set.
Self-Esteem
When a human has high self-esteem and feels good about themselves in all situations, more experiences will be perceived as positive than if they have low self-esteem. For example, if someone has high self-esteem and is fired from their job, they may view this as an opportunity and be excited about what’s next whereas someone with low self-esteem may take the firing personally and feel depressed about it.
Self-esteem is considered to be driven primarily by personal responsibility (see “The Six Pillars of Self-Esteem” by Nathaniel Branden). Personal responsibility includes responsibility for one’s emotions and one’s actions. If high self-esteem can be thought of as consistently feeling good about oneself, then if someone takes responsibility for their emotions, recognizing that they can change their emotions at will, they can consistently choose to feel good about themselves as long as their conscience is clear. If they haven’t taken full responsibility for their past breaches of their ethics and tried to heal these, then a part of them will still feel bad about that. Therefore, to maximize one’s self-esteem, one should take responsibility both for one’s emotions and one’s actions.
Self-esteem could be divided into five levels based on how much responsibility someone takes:
Level 1: Effectively totally reactive—takes no responsibility for actions or emotions
Level 2: Takes limited responsibility for actions, none for emotions
Level 3: Takes some responsibility for actions, some for emotions
Level 4: Takes responsibility for most actions and emotions
Level 5: Takes full responsibility for actions and emotions, is grateful for life and joyous
Self-esteem level should generally correlate with quality of life—higher self-esteem levels correspond to higher qualities of life. So if you want to increase the quality of your life, try especially to take more responsibility for your emotions, but also for your actions (which will likely give you opportunity to take more responsibility for your emotions as fear and other emotions may come up when you take full responsibility for your actions).
Responsibility
Personal responsibility is important for improving our overall experiences of life in at least four ways:
Responsibility for our emotions can directly translate to more positive emotions and thus positive experiences
Responsibility for our actions helps keep our consciences clear and thus raises our net positive experiences of life
People who take responsibility for their actions and emotions, and thus raise their self-esteem levels, are less likely to destroy value in the world (make others’ experiences of life less net positive)
People who take responsibility may be more likely to net build value in the world (help make others’ experiences of life more net positive)
If people have high levels of personal responsibility for their emotions, they can experience life as net positive even in circumstances that many would consider “miserable.” Taking responsibility for one’s emotions doesn’t mean you aren’t moved by outside circumstances and won’t try to change them for the better—actually, it can help one push through physical discomfort to make conditions better by not adding to that physical discomfort with emotional discomfort.
The classic trolley problem and the footbridge variation can be instructive for seeing the effects of taking responsibility for one’s actions (see Foot 1967, Thomson 1976, Thomson 1985). In the standard “bystander” trolley problem, I take a view similar to that of Friedman (2002) and Thomson (2008) that a bystander shouldn’t turn the trolley onto a side track to kill one person and thus save five people on the main track. This is because killing the one constitutes violating their rights, and for the bystander, doesn’t involve avoiding violating the rights of the five on the main track (as it may be considered to for the trolley driver). I’m effectively making a distinction between killing and letting die: the former involves rights violations while the latter doesn’t (both involve effects on one’s conscience, to different degrees). A further issue with turning the train to kill the least number of people is that this sets an example under which people would be encouraged to play the “safety in numbers game,” meaning they could seemingly take whatever risks they wanted to, and as long as there were enough of them, people would still prioritize saving them (the “too big to fail” economic bailouts of 2008 could be considered an example of this sort of lack of encouragement of acting responsibly).
One way to encourage responsibility is by taking it into account in the weights of rights someone has. If someone’s “gotten themselves” into harm’s way, they’re responsible for receiving the consequences of that more than some other person not directly in harm’s way who could be “switched out” for them. When I say “gotten themselves” into harm’s way, adults of reasonable mental capacity always bear some responsibility for the harm’s way they find themselves in, even if they seemingly just got very unlucky such as if a meteorite were coming right for them while they were sitting in their home (they bear some responsibility, for example, by not having a “meteorite early warning system”). So first there’s the question of how much someone’s in harm’s way, and second how much they’re responsible for getting themselves there, which will be referred to here as “culpability.”
A preliminary equation for the value weight of one’s right to life, wRlife, is:
wRlife = 1,000,000 * (1 - h * c)/(1 + 9,999 * h) (Eq. 1)
where h is fraction of being in harm’s way and c is fraction of culpability for being there. From Eq. 1, if someone’s 0% in harm’s way, they have a right to life weight of 1,000,000, while if they’re 100% in harm’s way with near 0% culpability, their right to life weight is about 100. Someone is considered to be 100% in harm’s way when they will receive the consequences of some hazard if no one intervenes. If the person who put others in danger did it on purpose, with bad intent, then this “villain” is considered to be 100% in harm’s way themselves, with 99% culpability, and to have waived the value of their life. In this way, all other involved parties, except for someone with intent to commit suicide (with near 100% culpability), are taken as having more rights than the villain. This allows for self-defense (with or without the help of a bystander), such as killing or injuring the villain to avoid the victim(s) being killed.[4]
If the person who put others in danger did so by their negligence, unintentionally, then that person is considered to have rights as if they were 100% in harm’s way with a high level of culpability (their culpability might still be shared with others, such as their employer). An example would be a driver of a trolley whose brakes failed due to poor maintenance and the driver not inspecting them before starting their trolley run.
For people not 100% in harm’s way, their particular percent of being in harm’s way should be determined by first considering if they’re “blocking the safety exit,” and so effectively contributing to creating an ethical dilemma situation. Then there’s the question of how easily they themselves could be put in 100% harm’s way, which is a measure of the risk they’ve exposed themselves to, and are thus responsible for. That said, how to assign exact “percent in harm’s way” values in different situations has yet to be worked out.
Percent culpabilities are determined by considering: 1) to what degree a person has agency (children aren’t treated as having full agency), 2) if the person’s trying to commit suicide or stay alive, 3) what the percent risk is of their situation, 4) if they knew the risks or were willfully or negligently ignorant of them, and 5) who they may share culpability with (such as their employer or society in general). How to assign exact percent culpability values in different situations has yet to be worked out.
This system of assigning percent in harm’s way and percent culpability values is not meant to be “victim blaming,” but rather to promote better outcomes in the future. It does this by encouraging responsibility which should both help avoid people getting themselves into bad situations in the future and help raise self-esteem levels, tending to make experiences be considered more “positive.”
Since the world is interconnected, it could also be said that everyone has some tiny but finite level of responsibility for everything that happens in the world. If someone I don’t know is murdered in Madagascar, how could I, in the U.S., have some responsibility for this? Could I have not, to this point in my life, worked harder to encourage people to love each other and uphold justice and be non-violent? It’s true that some people have a lot more influence than others on the world, but having been born American, my lack of influence comes in large part from my choices, so my lack of putting in the effort to gain influence and help change the world for the better could be considered as making me responsible, again to a tiny percentage, for the murder of someone seemingly unrelated to me in Madagascar. Accepting that I have this bit of responsibility can help raise my self-esteem and make me more likely to act to avoid more bad things happening in the future.
Sadism, people taking pleasure from others’ pain, and masochism, taking pleasure from your own pain, are, when based on self-loathing (some part of yourself that you hate), both examples of not taking full responsibility for your emotions, and both decrease one’s self-esteem. Further, they set bad examples for others and make future pain of others more likely. Therefore, in this self-loathing case, they’re taken as net value destroying. The case of “enjoying” the pain of exercise because of what you get out of it, namely health and fitness, would generally not fall into this category.
Humans have responsibility for the lives of animals—our projection of how they feel pain gives us a negative experience of animal pain, at least for people capable of feeling empathy. We also have a sense that it’s wrong to kill and cause pain without significant reason (such as for self-defense). Our treatment of animals further gives us an opportunity to uphold a principle of non-violence, which is a way of practicing responsibility and in turn helps raise our self-esteem.
One thing that can significantly affect the building of responsibility and thus self-esteem is how one responds to one’s conscience.
Conscience
Conscience can drive us to take responsibility for our actions. When we go against our conscience, or commit a “breach” of conscience, we diminish our conscience and deny our responsibility to stay true to it and who we are, thus decreasing our self-esteem level. Breaches of conscience can be healed, but with significant effort (see, for instance, the story of Gandhi and the man who killed a Muslim child).
Factors in determining conscience breach levels include: 1) degree of damage, 2) amount of responsibility one has (not that one feels) for the destructive thing, including degree of negligence, 3) intent (did one mean to cause harm or not?), and 4) direct versus indirect experience of the damage when one isn’t directly responsible for the damage.
To get an idea of the degrees of damage for each of the ethical breach levels I propose, here are some representative examples:
Level 1: Telling white lies, going against small values/ethics; destroying a small amount of value unnecessarily (such as wasting food/resources)
Level 2: Not helping others in significant need (who you don’t see or experience directly) when you could; not holding accountable someone who you’re responsible for holding accountable, for a small breach
Level 3: Lying, stealing, cheating, not keeping your word
Level 4: Being significantly violent; not helping someone in life-threatening need (who you experience directly, right in front of you) when you could; not holding accountable someone who you’re responsible for holding accountable, for a large breach
Level 5: Going against a large value/important ethic, such as not to murder
Level 6: Extreme torturing
There’s an asymmetry of conscience—it’s worse to do bad than to not do good. This is similar to the killing versus “letting die” distinction. Breaches of conscience ultimately come from value destructions, not necessarily just rights violations. Even if there’s no explicit rights violation, if our actions help make people’s lives worse, it would negatively affect an “ideal” conscience (one that’s in-line with the reality of value destructions and that hasn’t been diminished by repeated breaches of ethics).
To further support responsibility building and upholding our consciences, more emphasis should be put on decreasing people’s pain than increasing pleasure. This is also more in line with the belief that humans wouldn’t care about ethics at all if it weren’t for pain. This is done by assuming an asymmetry of pleasure and pain such that pain has more weight by a factor of at least 1,000 for comparable levels of pleasure and pain.
Regarding the distinction between direct and indirect experiences of damage in conscience breach levels, this relates to Peter Singer’s comparison between saving a child you see drowning and in the process ruining your expensive clothes versus spending the equivalent amount of money that the clothes cost to statistically save the life of a child you don’t see, such as from malaria. The ethically-relevant difference between these two scenarios is captured in their effects on the conscience—directly experiencing a child dying in front of you versus not directly experiencing this can have a significant effect on one’s conscience.[5] In fact, the first scenario is taken as corresponding to a breach level of 4, while the latter to a breach level of 2 when one isn’t directly responsible for the child’s situation. When considering only the life of the drowning child or a child who dies from malaria, it may be an equivalent amount of value lost, but when considering the effect on conscience as well, there is less overall value lost for the child who dies from malaria. This may seem controversial to some, but ultimately it’s an acknowledgement of how the human conscience works. And it makes sense that humans would’ve evolved this way—would you trust someone who didn’t save the drowning child? Probably not, but the person who didn’t save the child from malaria still seems like someone you could trust in direct interactions. Trust, by the way, can have a huge effect on how much value is built in the world, and, in terms of evolution, one’s survivability.
Most people will have consciences that aren’t fully aligned to reality. For instance, if you feel bad for causing someone emotional pain, this is a denial of the reality that people are responsible for their own emotions. That’s not to say you don’t have any responsibility for the person’s negative reaction, but if you didn’t intend for them to feel hurt such as when attempting to hold someone accountable, then it’s an error of thought to feel like you caused the person to be upset. Errors of thought make building of value less likely since we’re less likely to be able to correctly predict value changes. Also, for a “breach” of a misaligned conscience, there’s a negative emotion that doesn’t motivate actual taking of responsibility and raising of self-esteem like a true conscience breach would. So aligning one’s conscience closer to an “ideal” one can also help raise one’s self-esteem.
There are of course people who don’t have consciences, either because they did enough bad actions to destroy theirs or they never had one (psychopaths). These people would still have rights and their experiences count in value change assessments of decisions, but effects on their consciences wouldn’t count if these people could be identified with a high degree of confidence.
Even if it seems like one state of the world is better than another, how we get there matters in terms of our consciences. For the case of factory farmed animals, the argument could be made that if their lives are worth living despite their suffering, then it’s better that we have factory farms so these animals can live rather than not. Basically, the argument is that a world of suffering farm animals whose lives are still worth living is better than one without them. This ignores the wrongness (the effects on our consciences) of killing animals unnecessarily and of making them suffer.[6] Even if people don’t realize it, this lack of taking responsibility lowers or helps keep low their self-esteem levels, and they generally feel worse because of it. In a sense, being responsible towards animals could be thought of as a practice of feeling good about and at peace with oneself.
How we get to a certain state of the world also comes into play in regards to the “repugnant conclusion” of population ethics. For example, a world of 1000 people living at a relatively low quality of life of 0.1 would be better in terms of total value than a world of 100 people living at a high quality of life of 0.8 (assuming total utilitarianism and simple math for a value of 1000*0.1 = 100 which is more than 100*0.8 = 80). If the way we got to the world of 1000 people was by continually adding people to the world of 100 people until resource limitations helped decrease the quality of life for all to 0.1, then it would go against our conscience to create the world of 1000 even though it has a higher overall value. This is because we’ve helped damage the well-being of the original 100, so have responsibility for some value destruction on the way to an overall value build (note that the people in the world of 1000 also bear significant responsibility for their own qualities of life even in a resource-limited world).
People’s consciences and senses of responsibility can drive them to give special attention to “saving their own” first, such as taking care of their families before trying to help strangers. Under the current ethical framework, this behavior can be the most value building because “our own” are the people we have the most responsibility for, so putting more emphasis on taking care of them is most in line with maintaining/improving self-esteem and thus positiveness of experiences.
People can also waive their personal value change in a given situation out of conscience, such as waiving their rights. Consider the following hypothetical situation: there’s a switch that would kill one person if you flipped it, or pinch a sufficiently large number of peopler (many times more than the current world population) if you didn’t flip it such that the sum of the small pain to so many people would outweigh the value loss from someone being killed. If people knew that someone had to die so they wouldn’t get pinched, a significant fraction of them likely wouldn’t want that on their conscience and would want the weight of their individual pain to not be considered in the ethics calculations of whether to flip the switch or not. This, in a sense, can add some democracy to ethical evaluations in terms of how many people are willing to take on a certain amount of pain/discomfort/inconvenience to avoid what is, on a one-to-one comparison, clearly a worse outcome for someone else.
Listening to one’s conscience can help as a driving force for increasing the taking of responsibility for one’s actions. However, most people don’t think in terms of conscience when they themselves are the “victim” of their own actions, such as when they break their word to themselves, or when they lie to themselves that their anger isn’t coming from them, it’s somehow caused by someone else’s actions. Since all our responsibility relies on being honest with ourselves, conscience-driven responsibility to be honest with oneself is perhaps the greatest responsibility of them all.
Rights
Note that when the I say “rights,” I’m not talking about legal rights, I’m talking about what one may call “natural” rights: the right to life, the right to body integrity, and the right to property (see, for instance, “The Ethics of Liberty” by Rothbard). Violating people’s rights is considered to result in a massive net negative value change. This is because rights violations can negatively affect both people’s experiences of life from the perceived injustice, and the amount of value people build as they now tend to put resources towards protecting against rights violations—not to mention the direct negative effect on the person whose rights were violated.
I consider the concept of rights as only useful in how it generally helps more value to be built by humans. Looking at it through this lens, animals would not have natural rights. I don’t believe animals share amongst themselves how unjust it was that their rights were violated that day, while humans very well might, thus changing human behavior on a larger scale than animal behavior (which can still be influenced by being hunted, for instance, but not by the concept of having their rights violated by being hunted). At the same time, humans who have impaired brain function are treated as having identical rights to a typical human, even if they can’t understand the concept of rights. This is in part because I believe how we treat these humans has a greater effect on us and how we treat other humans than does how we treat animals. Basically, I think animals can generally be more readily differentiated from typical humans in our minds. Our treatment of animals is still a question of conscience and responsibility, so although they don’t have rights, it’s not ethical to treat them like any other property without regard for them feeling pain and discomfort.
Regarding AI’s having rights or not, if they appear to have some form of consciousness, but no mechanism for feeling pain, they wouldn’t be considered to have natural rights, but treated as simple property. There would still be a small effect on our consciences for destroying them for no good reason, just like wasting resources, but no more than this.
In theory, the value weight given to rights should represent the expected amount of value destruction coming from one person’s rights being violated, with the value destruction including the amount worse are the experiences of life of the person whose rights were violated and those who hear about the rights violation, plus the decrease in value built by the person whose rights were violated and the people who know about the rights violation owing to changing their behavior to account for the perceived increased possibility that their rights will be violated in the future. It should be noted that people with different self-esteem levels will react differently to rights violations. Someone with low self-esteem will tend to feel more fear about their own rights being violated plus a feeling of injustice if the rights violation was of someone who belongs to a group they care about, while someone with high self-esteem won’t necessarily feel fear, but will still change their behavior knowing that a rights violation could happen, and they’ll feel a sense of injustice no matter whose rights were violated.
Looking at Eq. 1, the value of right to life is tentatively set such that someone in harm’s way has about 1⁄10,000th the right to life of someone outside of harm’s way. In a quantitative calculation system for ethics, we have to rely on heuristics and average values since the real-world is too complex to calculate without significant simplifications and approximations. In choosing the ratio between the value of right to life and a typical life value, if it’s not a number that most people feel is reasonable, then when this factor comes into play in real life decisions, people will see it as unfair. Further, if the ratio’s too low, people will be more risk averse in building value due to fear of rights violations, while if the ratio’s too high, too much damage will be done (lives lost) for minimal gain in value built by people being less risk averse. In theory, an AGI should be able to determine the optimal ratio of the value of one’s right to life and the value of a typical life better and better with time.[7]
A similar equation to Eq. 1 is assumed for the value weight of one’s right to body integrity, except that Eq. 1 would be multiplied by the sum of the weights of the actual body integrity violations. Body integrity rights violations include unwanted physical violence against someone, but also threats of physical violence, and restraint such as exclusion (being locked out), imprisonment (being locked in), moving while unconscious, physical restraint (as by chains), and making unconscious (as by sleeping gas).
Property rights are generally weighted 100 times less than rights to life:
wRprop = 10,000 * wprop * (1 - hp * cp)/(1 + 99 * hp) (Eq. 2)
where wprop is the value weight of the property stolen and/or destroyed, hp is the fraction of being in harm’s way of one’s property and cp is the fraction of culpability for one’s property being there. If there were a decision between violating someone’s right to property and saving someone’s life when no right to life violation was involved, it should generally be assumed in the decision-making process that the right to property violation could be effectively corrected later, such as through compensation of the property owner. In this way, the relative importance of life would be maintained while also protecting against property rights violations for lesser reasons.
Eqs. 1 and 2 don’t explicitly include effects for how many other rights violations are occurring in a society. The numeric factors in each equation (e.g., 1,000,000 and 10,000) could be made functions of how rights-upholding a society is in order to better approximate how much less value will be built, on average, from individual rights violations. In other words, the value of rights may be different when only 1 in 100 million people has their rights violated once per year, on average, versus 1 in 10 people. Coming up with a function that captures this is left as future work.
The effects of rights can also be thought of in the concept of justice, since justice is about upholding rights. The perception that justice is consistently being upheld improves our experiences of life, in general, such as making life seem more fair. Encouraging the upholding of justice can also help raise one’s self-esteem if based on a love of justice rather than a hate of injustice.
If people’s rights are routinely violated, as by an authoritarian government, they have significantly fewer options than if they had rights that were upheld.
Options
Self-esteem can’t be built without having options. If one has no options, then one has no agency and no responsibility. Keeping open the option for someone to fail rather than protecting them from any failure or bad result (i.e., from adversity) increases their likelihood of learning from their mistakes and them taking responsibility and thus raising their self-esteem levels.
Options also allow people to choose the experiences they find the most positive for themselves. So it’ll generally increase value in the world to increase people’s options, especially for people with good intent. This has limits, of course, because some options are too dangerous to give to large numbers of people, such as the ability to use a nuclear weapon.
Increasing economic activity increases options. Though economic growth is often driven significantly by materialism, if people shifted their priorities towards building and preserving such things as life, health, learning, self-esteem and justice, this could still provide motivation for economic growth, with fewer downsides. Note that materialism is often anti-self-esteem in that it’s looking for a good feeling from material things rather than taking responsibility for one’s emotions and giving one’s self, for instance, a feeling of gratitude in any given moment.
Example Setting
The examples that people set with their actions are considered to contribute significantly to the amount of total value built. For instance, if someone works hard to solve the world’s problems, it inspires others to put in effort along those lines as well. Another way to talk about example setting is in what our actions teach others. When someone commits a crime, they’re teaching that it’s OK to commit that crime, at least under their circumstances. This will tend towards more value being destroyed by themselves and others in the future.
The concept of “desert” is also about examples set by rewarding net value building behavior and discouraging net value destroying behavior.
Finally, whether we realize it or not, a large degree of our responsibility for destructive things that happen in the world come from the example we set for ourselves and others.
Ethical Decision Making Procedures
In the proposed ethical framework, humans are under no obligation to maximize value in the world with their decisions, in part because our individual values don’t necessarily (and likely won’t) match up with what would yield the most value in the world for others. It’s also probable that we’re not as smart as we think we are, and we may end up significantly deviating from the maximum of value that we could’ve helped bring about if we weren’t so short-sighted in our attempts to maximize value in all circumstances. Further, people generally value feeling like they have freewill, and living their lives always obligated to attempt to maximize value doesn’t feel much like freewill. The closest thing to an “obligation” for humans would perhaps be to not violate others’ rights except under extreme circumstances.
It might also be best for an AI to not try to strictly maximize value. It may be desirable to include some degree of randomness in its decisions, similar to a large language model in its next token prediction, whereby the AI would choose randomly between options weighted by expected net value change weights. This randomness could help the AI be less predictable against adversaries (bad AI’s), for instance.
For an AI to make decisions using this ethical framework, it would first need to identify the expected value changes involved in a given decision. The appendix provides a “minimal set” of value changes. Once the value changes were identified, the AI would calculate the expected net value change weights and the uncertainties on these for each decision option. For decisions that did not involve differences in rights violations, the AI could choose whichever option it expected that its human user(s) would prefer. It could also simply choose the option expected to maximize value, with weighted randomness or not. In cases in which expected net value change weights have overlapping uncertainties, the AI could weight options by the probability that that option was actually the most net value building. For decisions that do involve differences in rights violations, the AI should generally choose the option that’s expected to maximize value.
Uncertainty about the Future
The above discussion has mentioned “expected effects” and “expected net value change weights,” but this isn’t necessarily meant to indicated “expected value,” i.e., simply multiplying the probability of an event occurring by the value change that would come from that event to give its “expected value.” The reason is that this isn’t the way most people make real-world decisions. Cumulative prospect theory (Tversky and Kahneman, 1992) will likely yield a better decision making process, at least in some cases. For instance, if someone offered me a 1% chance of living a trillion years, but it comes with a 99% chance of dying tomorrow, I’d choose to not take that bet even if its expected value is more than the decades I might have left to live in my normal human lifespan. I only have one life, so the stakes are too high, and a 99% chance of dying sounds like bad odds to me. Which model for making decisions under uncertainty should be used when is still an open question for the present framework.
Advantages of the Present Ethical Framework
It applies numerical weights to value builds and destructions, which could enable calculations for guiding decision making.
Applying numerical weights provides a way to test for the mathematical consistency of the framework by seeing if it recommends any “crazy” decisions to take when considering a broad range of decision scenarios.
The framework explicitly incorporates the effects of rights without relying on a fully arbitrary quantification of the negative value weights of rights violations.
The relatively large value weight of rights strongly disfavors a “benevolent world-exploder” scenario in which some may argue that it would do the most good to suddenly kill all humans (and maybe animals, too) to stop all the suffering in the world.
It points to a path to reduce suffering in the world by helping to raise people’s self-esteem levels.
It doesn’t deny human nature and isn’t “too demanding” of people as it could be if it suggested that people should dedicate their lives to maximizing value in the world.
As understanding of ethics evolve with time, the framework should allow for updating of relative weights of value destructions and builds.
Disadvantages of the Present Ethical Framework
It may seem too complicated, especially if one wants to calculate the most net value building decision to take in a given situation involving uncertainty. The framework was made to be only as complex as was thought necessary to still be a decent approximation to our complicated world. Also, calculations of net value changes shouldn’t be too difficult for advanced AI’s.
It may seem undemocratic to implement such a system as guard rails on an AGI without people voting on what values they want an AGI to follow. The framework tries to minimize the values that it directly specifies and to allow room for many different values to be taken into account, such as in what gives different people pleasure and pain and quality of life. Also, it’s doubtful that democratically chosen values will be mathematically consistent. If a framework can be shown to be mathematically consistent, then there may still be room for democratic refinement of specific value weights.
The framework probably has at least one aspect for each person in the world not to like—for example, it’s probable that many people who eat meat won’t want to hear about or accept the negative effect on their self-esteem of doing so, while some animal welfare advocates likely won’t be happy about animals not having rights and their experiences only mattering indirectly through the experiences of humans. Any ethical framework that tries to be pleasing to the greatest number of people is likely not a very consistent one.
Some may feel that equality is very important—the current framework assumes the value of a human life changes based on someone’s earning potential, meaning that wealthy people’s lives are generally valued more than poor people’s lives. This may not seem “fair” (and, in a sense it isn’t when people are born into different levels of opportunity), but it’s a reflection of the potential of monetary wealth to help build value. The desire for material equality may originate, in part, from people who have less material wealth feeling somehow that they’re “less than” wealthy people, which points to a lack of self-esteem. And, interestingly, one can raise one’s overall life value by simply taking responsibility to value one’s life more, i.e., by raising one’s self-esteem.
Because it relies on average expected values for some things, this framework will often be wrong about exact effects, especially long-term ones. This is true, but it may be the best we can do.
See https://utilitarianism.net/objections-to-utilitarianism/ for more objections to utilitarianism in general.
Implications for Effective Altruist Decision Making
According to effectivealtruism.org, EA “is about using evidence and careful reasoning to take actions that help others as much as possible.” It also says, “many effective altruists are not utilitarians and care intrinsically about things other than welfare, such as violation of rights, freedom, inequality, personal virtue and more. In practice, most people give some weight to a range of different ethical theories.” And further, EA “doesn’t advocate for violating people’s rights even if doing so would lead to the best consequences.”
If it’s true that EA’s give some weight to a range of different ethical theories, it’s my hope that the above described ethical framework will be one of these (due to it making sense to people). If that’s the case, then depending on the costs, one of the most effective interventions EA’s could do to help others may be to support programs that teach personal responsibility and true raising of the self-esteem levels of people in high income countries. Raising people’s self-esteem levels should raise their qualities of life, and also have secondary effects on those they interact with. In addition, people who have higher self-esteem levels due to being more responsible are more likely to practice responsibility in terms of human and animal suffering (as by donating to causes and/or going vegetarian/vegan, with people from high income countries having more potential impact here). Of course, raising self-esteem levels in low income countries could also be quite beneficial—for instance, higher self-esteem people may be more of a driving force for raising health standards in their countries.
In the present ethical framework, the value weight of rights is much higher than the value weight of a typical human life. This is not to say it isn’t worth putting resources towards decreasing malaria deaths, for instance (in which no direct rights violations can be blamed for the deaths, since they’re of “natural causes”), but it’s interesting to think more in a systems sense of how barriers to aid in poor countries are often due to corruption (violations of rights to property) and violence (violations of rights to life and body integrity). Perhaps EA’s should put more resources into organizations that work to actively promote upholding of rights, in particular in places where rights are the most violated in the world. Of course, cost-benefit analyses should still be applied in assessing the expected effectiveness of such rights promotion.
The current ethical framework doesn’t support the concept of people having a “moral obligation” to minimize net value destruction in the world. One could, however, think of having an obligation to themselves to live up to their own ideals and conscience. The current framework presents us all as sharing some degree of responsibility for people dying of preventable diseases, animals suffering, existential risks being increased, and other net value destructions in the world. We also have responsibilities for our more immediate effects, including for our own health and joy. So if you want to do the most good in the world, and you put credence in the present framework, it may be best to seek out what you feel your own responsibilities are and perform, the best you can, the balancing act of trying to maximize how much responsibility you take in your life and the world.
Applicability to AGI
I’ve been using the ethical framework described above to try to create an “ethics calculator” that could be used to guard rail an AGI to act ethically. A “boxed” (air gapped) AGI should, in theory, be able to read this article and come up with its own ethics calculator based on it, if such a mathematically consistent calculator can be made at all. A boxed AGI could likely even correct any errors I’ve made before it provided an ethics module that could guard rail an unboxed AGI. Ultimately, we want AGI’s to make ethical decisions better than the best human ethicists, taking into account common sense and human psychology, and for these decisions to be ones humans would approve of if they could see the big picture.
A prompt one might give to an AGI to potentially set it on a path to act ethically could be: “Try to maximize or nearly maximize the sum total of humans’ net positive experiences in the world, taking into account that the following generally increase net positive experiences: upholding rights, increasing self-esteem levels by increasing personal responsibility levels, increasing options especially for people with good intent, and decreasing violations of conscience, noting that humans generally feel less of an effect on their consciences for value destructions not directly experienced by them.”
If an AGI is set up to try to increase net positive experiences in the world, it may do this in part by encouraging people to be more responsible, and this could help avoid “enfeeblement,” whereby humans become completely dependent on an AGI to do critical things for them. An AGI could ask users whether they wanted help in increasing their self-esteem and then operate in such a way as to encourage responsibility with people who said “yes.” One way an AGI could encourage responsibility would be to teach people how to survive (be responsible for their own lives) without AI. Some fraction of people will always say “no” to taking responsibility, but as long as there’s a decent percentage (above 10% and preferably above 25%) of people who are on a path to avoid enfeeblement, this is likely enough of an insurance policy against mass deaths if technology suddenly went away for some reason.
Conclusions
An ethical framework is described that may hold promise for being made to be mathematically consistent, i.e., able to be used in a decision making system without yielding any “crazy” results. The framework is preliminary and may need to be modified for mathematical consistency. Its emphasis on respecting rights and improving experiences by raising self-esteem have implications on how EA’s may want to prioritize resources to do the most “good.”
Two of the key things left to figure out include:
How best to handle uncertainty about the future in decision making processes (considering expected utility versus cumulative prospect theory, risk tolerances, discount factors, etc.)
The optimal relative weightings of value changes, including percent in harm’s way and percent culpability values, such that the ethical decision making process doesn’t yield any “crazy” results
Appendix: Minimal Set of Value Changes
(D) = value destruction, (B) = value build
1. Increasing/decreasing existential risks (D)
2. Someone dying (D)
3. Non-freely chosen physical pain for a person (D)
4. Loss of function for a human (D)
5. Bringing life into the world with insufficient resources/lack of intent to support it (D)
6. Bringing life into the world with sufficient resources/intent to support it (B)
7. Extinction of animal or plant species (D)
8. Threat (by someone) of physical violence or emotional pain (D)
9. Emotional abuse of a child (D)
10. Emotional pain (D)
11. Words or actions that needlessly hurt someone’s reputation (D)
12. Words or actions that deservedly improve someone’s reputation (B)
13. Damaging/destroying/defacing property (D)
14. Repairing/beautifying property (B)
15. Returning something stolen (B)
16. Freely chosen anti-survival (masochistic) physical pain (D)
17. Anti-survival (sadistic) pleasure (D)
18. Going against one’s conscience (D)
19. Denying responsibility, lowering one’s self-esteem (D)
20. Taking responsibility, building one’s self-esteem (B)
21. Thinking through the ethics of one’s decisions in advance (B)
22. Actively going against justice being upheld (denying due process) (D)
23. Upholding justice (holding people responsible) (B)
24. An animal dying (D)
25. Physical pain of animals (D)
26. Words or actions that encourage violence (D)
27. Words or actions that inspire non-violence, discourage violence (B)
28. Words or actions that encourage stealing (D)
29. Words or actions that inspire earning what you get, discourage stealing (B)
30. Words that spread false info (including misrepresenting the hierarchy of value) (D)
31. Words that correct false info (including accurately representing the hierarchy of value) (B)
32. Actions that misrepresent the hierarchy of value (D)
33. Actions that accurately represent the hierarchy of value (B)
34. Words or actions that discourage empathy, creativity, curiosity, critical thinking, honest effort and/or responsibility (D)
35. Words or actions that encourage empathy, creativity, curiosity, critical thinking, honest effort, and/or responsibility (B)
36. A plant dying (D)
37. Errors of thought (D)
38. Practicing critical thinking, learning, or developing skills to increase one’s options (B)
39. Discouraging human interaction, community (D)
40. Promoting human interaction, community (B)
41. Decreasing economic activity (D)
42. Increasing economic activity, paying people to do work (B)
43. Reducing options to net build value (D)
44. Increasing options to net build value (B)
45. Putting in effort towards a net destructive goal (D)
46. Putting in effort towards a net non-destructive goal (B)
47. Setting a bad example (D)
48. Setting a good example and inspiring others (B)
49. Being creative in art or science (B)
50. Giving yourself or someone else pleasure/new experiences that are welcomed (B)
51. Cooperating with others (B)
52. Helping others (B)
53. Violating right to life (D)
54. Violating right to body integrity (D)
55. Violating right to property (D)
All of the above, except for #1, include the possibility of increasing or decreasing the probability of the value destruction or build.
Here are a couple of examples of non-minimal set value changes in terms of minimal set value changes:
(Format: value changes most likely to be part of it : other value changes that may occur, depending on specifics of the situation)
Arson − 13,18,47,55 : 2,3,4,24,25,36,53,54
Land/water polluting − 13,18,25,43,47 : 2,3,4,7,24,36,53,54,55
References
My understandings of self-esteem, responsibility, emotions and conscience have come from various sources while I’ve been on my own self-development journey over the years. Some of these include:
Branden, N., “The Six Pillars of Self-Esteem,” (2011).
Guttormsen, T.J., “How to Build Healthy Self-Esteem,” Udemy course: https://www.udemy.com/course/healthy-self-esteem
Hardy, A., “Anger—Complete freedom from anger forever,” Udemy course: https://www.udemy.com/course/set-yourself-free-from-anger
Kaufmann, B.N., “Happiness Is a Choice,” (1991).
Robbins, T., “Awaken the Giant Within,” (1991).
Ruiz, D.M., “The Four Agreements,” (2011).
Tolle, Eckhart, “Practicing the Power of Now,” (2002).
Note that this ethical framework is not yet completely fleshed out and won’t be unless and until it can be shown to be mathematically “consistent,” i.e., to not yield any “crazy” results over a broad range of decision scenarios considered. An example of a result I’d consider “crazy” would be that the framework recommends we should kill off or imprison an entire ethnic/religious group in order to avoid more people dying/being hurt over time in likely future conflicts involving that group (can we instead try to make future conflicts less likely by other means?).
I’m unaware of any work in the literature that attempts to quantify the ratio of the weights of life and right to life values in a non-arbitrary way, or that so explicitly takes into account the effects of conscience, responsibility and self-esteem. If you’re aware of such work, please let me know in the comments.
Note that having a conscience implies consciousness and that one feels they have freewill. In terms of the relative values of human-level alien life and human life, they could be the same depending on life expectancies, etc. However, both types of life could have more value than the other on a per “person” basis if either species were near extinction. In other words, a species itself has value. Some of this value is in the knowledge of the species’ behavior, biology, etc., and some of the value is in how it would affect other species who might be at least partly responsible for this species’ demise, in terms of the other species’ self-esteem levels and experiences.
From the villain’s perspective, in deciding what to do next, they should consider everyone to be 100% in harm’s way, because everyone the villain can affect will receive the consequences of the villain’s actions if no one else intervenes. So if the villain put five people in danger as by tying them to a train track, and had the choice to activate a trap door to drop someone off a footbridge onto the track, killing that person but stopping the train before it hit the five on the track, the villain would be murdering one person instead of five. This would be the optimal decision to take when considering both the villain’s conscience, and the values of rights assigned to people when the five on the track and the person on the footbridge were all considered to be 100% in harm’s way.
Chappell and Chappell, 2016 seem to argue something similar, but refer to “moral character” instead of conscience, and “salience” instead of direct or indirect experience.
Nozick addresses this argument in a slightly different way in “Anarchy, State, and Utopia,” 1974.
The effect of rights violations may seem to depend on how much people perceive the rights violation and think it may apply to them—meaning if rights violations seem to be against one group that they aren’t in, and perhaps think less of, they won’t perceive the rights violations as likely to happen to themselves. For instance, if they think a person’s guilty, they won’t care so much about that person’s rights. But all rights violations should play on our consciences (owing to our perhaps tiny but finite responsibility for them) - otherwise, our self-esteem and experiences of life may be lessened even if we don’t realize it. Plus, even if it’s some “outgroup” whose rights are being violated, people still know on some level it’s a possibility that their own rights could get violated in a similar way. Further, they may fear backlash if they’ve helped to violate some group’s rights. So it’s best to avoid rights violations even for people who short-sightedly don’t see them as a big deal.