RSS

Mo­ral uncertainty

TagLast edit: 2 Aug 2023 9:04 UTC by Leo

Moral uncertainty is uncertainty about how to act given lack of certainty in any one moral theory, as well as the study of how we ought to act given this uncertainty.

We are sometimes uncertain about empirical facts, such as whether it will rain tomorrow. But we can also be uncertain about moral facts, such as whether it’s wrong to steal, or how we should value the well-being of animals. Uncertainty about whether it’s wrong to steal is uncertainty about moral or normative facts, while uncertainty about how we should value the well-being of animals is uncertainty about axiological or value facts.

Can moral uncertainty be rational, and what should we do in response to it? We might think that it can never be rational to be uncertain about normative or axiological facts, because such facts are, like mathematical facts, knowable a priori. Nevertheless it seems that agents like ourselves are uncertain about non-trivial mathematical facts, and that we are also uncertain about normative and axiological facts. Given this, it seems necessary to develop some account of how we ought to act under moral uncertainty.

Several such accounts have been formulated in recent years:

Two important problems facing the view that moral uncertainty can affect how we ought to act are the regress problem—the problem that we will be uncertain not only about typical moral questions, but also about which approach to moral uncertainty is correct, and so on ad infinitum[5] - and the problem of intertheoretic comparisons—the problem that there seems to be no principled ways to make comparisons between different moral theories.[6]

Further reading

MacAskill, William, Krister Bykvist, & Toby Ord (2020) Moral Uncertainty, Oxford: Oxford University Press.

Related entries

altruistic wager | decision-theoretic uncertainty | ethics of existential risk | expected value | fanaticism | moral trade | normative uncertainty

  1. ^

    Weatherson, Brian (2014) Running risks morally, Philosophical Studies, 167: 141–63.

  2. ^

    Gustafsson, Johan E. & Olle Torpman (2014) In defence of my favourite theory, Pacific Philosophical Quarterly, 95(2): 159-174.

  3. ^

    MacAskill, William (2014) Normative uncertainty, Doctoral dissertation, University of Oxford.

  4. ^

    Bostrom, Nick (2009) Moral uncertainty—towards a solution?, Overcoming Bias, January 1.

  5. ^
  6. ^

    Nissan-Rozen, Ittay (2015) Against moral hedging, Economics and Philosophy, 31(3): 349-369.

New book: Mo­ral Uncer­tainty by MacAskill, Ord & Bykvist

frankieaw10 Sep 2020 16:22 UTC
91 points
2 comments1 min readEA link

A dilemma for Max­i­mize Ex­pected Choice­wor­thi­ness (MEC)

Calvin_Baker1 Sep 2022 14:46 UTC
33 points
7 comments10 min readEA link

Wheel­ing and deal­ing: An in­ter­nal bar­gain­ing ap­proach to moral uncertainty

MichaelPlant10 Jul 2022 12:04 UTC
56 points
18 comments23 min readEA link

Mo­ral Uncer­tainty and Mo­ral Real­ism Are in Tension

Lukas_Gloor25 Jan 2022 11:07 UTC
39 points
4 comments12 min readEA link

Re­solv­ing moral un­cer­tainty with randomization

Bob Jacobs 🔸29 Mar 2023 10:10 UTC
29 points
3 comments10 min readEA link

Mak­ing de­ci­sions un­der moral uncertainty

MichaelA🔸1 Jan 2020 13:02 UTC
44 points
8 comments17 min readEA link

The prop­erty rights ap­proach to moral uncertainty

Harry R. Lloyd14 Oct 2022 16:49 UTC
31 points
14 comments2 min readEA link
(www.happierlivesinstitute.org)

A philo­soph­i­cal re­view of Open Philan­thropy’s Cause Pri­ori­ti­sa­tion Framework

MichaelPlant15 Jul 2022 8:21 UTC
160 points
11 comments29 min readEA link
(www.happierlivesinstitute.org)

Mo­ral Anti-Real­ism: In­tro­duc­tion & Summary

Lukas_Gloor2 Apr 2022 14:32 UTC
50 points
1 comment7 min readEA link

The “Mo­ral Uncer­tainty” Rab­bit Hole, Fully Excavated

Lukas_Gloor1 Apr 2022 13:14 UTC
23 points
2 comments34 min readEA link

The moral am­bi­guity of fish­ing on wild aquatic an­i­mal populations

MichaelStJules6 May 2024 23:03 UTC
75 points
2 comments4 min readEA link

Hedg­ing against deep and moral uncertainty

MichaelStJules12 Sep 2020 23:44 UTC
83 points
13 comments9 min readEA link

A longter­mist cri­tique of “The ex­pected value of ex­tinc­tion risk re­duc­tion is pos­i­tive”

Anthony DiGiovanni1 Jul 2021 21:01 UTC
131 points
10 comments32 min readEA link

If Con­trac­tu­al­ism, Then AMF

Bob Fischer13 Oct 2023 18:03 UTC
62 points
54 comments24 min readEA link

The op­tion value ar­gu­ment doesn’t work when it’s most needed

Winston24 Oct 2023 19:40 UTC
125 points
6 comments6 min readEA link

A Case for Vol­un­tary Abor­tion Reduction

Ariel Simnegar 🔸20 Dec 2022 13:23 UTC
39 points
150 comments13 min readEA link

Cause pri­ori­ti­za­tion for down­side-fo­cused value systems

Lukas_Gloor31 Jan 2018 14:47 UTC
75 points
10 comments48 min readEA link

Wor­ld­view uncertainty

Ronja_Lutz16 May 2018 14:02 UTC
15 points
5 comments8 min readEA link

Ar­gu­ments for moral indefinability

richard_ngo8 Feb 2019 11:09 UTC
34 points
8 comments7 min readEA link
(thinkingcomplete.blogspot.com)

The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

Linch2 Aug 2019 21:55 UTC
144 points
17 comments4 min readEA link
(docs.google.com)

Light­ning talks from EA Global: San Fran­cisco (2018)

EA Global8 Jun 2018 20:51 UTC
7 points
0 comments11 min readEA link
(www.youtube.com)

An In­tro­duc­tion to the CRAFT Sequence

Bob Fischer8 Jul 2024 10:27 UTC
103 points
1 comment4 min readEA link

Se­quence overview: Welfare and moral weights

MichaelStJules1 Aug 2024 16:51 UTC
34 points
0 comments1 min readEA link

Re­think Pri­ori­ties’ Mo­ral Par­li­a­ment Tool

Derek Shiller17 Jul 2024 12:34 UTC
179 points
12 comments12 min readEA link

Me­taeth­i­cal Fa­nat­i­cism (Dialogue)

Lukas_Gloor17 Jun 2020 12:33 UTC
35 points
10 comments15 min readEA link

Ex­pected value un­der nor­ma­tive uncertainty

Gordon Seidoh Worley8 Jun 2020 15:45 UTC
16 points
1 comment1 min readEA link
(philpapers.org)

Wrong by Induction

Ales_Flidr6 Sep 2018 22:00 UTC
20 points
2 comments6 min readEA link

Might stop­ping mis­car­riages be a very im­por­tant cause area?

SaraAzubuike5 Dec 2021 17:18 UTC
16 points
13 comments2 min readEA link

[Pod­cast] Ajeya Co­tra on wor­ld­view di­ver­sifi­ca­tion and how big the fu­ture could be

Eevee🔹22 Jan 2021 23:57 UTC
57 points
20 comments1 min readEA link
(80000hours.org)

Quotes about the long reflection

MichaelA🔸5 Mar 2020 7:48 UTC
55 points
14 comments13 min readEA link

Mo­ral plu­ral­ism and longter­mism | Sunyshore

Eevee🔹17 Apr 2021 0:14 UTC
26 points
0 comments5 min readEA link
(sunyshore.substack.com)

Peter Singer on the 80K podcast

Jon_Behar18 Dec 2019 17:17 UTC
31 points
0 comments1 min readEA link
(80000hours.org)

New re­search on moral weights

GiveWell4 Dec 2019 11:57 UTC
42 points
4 comments2 min readEA link
(blog.givewell.org)

EA read­ing list: clue­less­ness and epistemic modesty

richard_ngo3 Aug 2020 9:23 UTC
27 points
3 comments1 min readEA link

Stan­ley the Uncer­tain [Creative Writ­ing Con­test]

Akash28 Oct 2021 20:12 UTC
33 points
1 comment3 min readEA link

Even Allo­ca­tion Strat­egy un­der High Model Ambiguity

MichaelStJules31 Dec 2020 9:10 UTC
15 points
3 comments2 min readEA link

The ex­pected value of ex­tinc­tion risk re­duc­tion is positive

JanB9 Dec 2018 8:00 UTC
66 points
22 comments61 min readEA link

Deon­tol­ogy and virtue ethics as “effec­tive the­o­ries” of con­se­quen­tial­ist ethics

Jan_Kulveit17 Nov 2022 9:20 UTC
55 points
12 comments10 min readEA link

SoGive’s moral weights—please take part!

Sanjay5 Apr 2021 22:47 UTC
26 points
2 comments2 min readEA link

Brian To­masik – Rea­sons to Be Nice to Other Value Systems

Tessa A 🔸17 Jan 2014 0:28 UTC
24 points
0 comments1 min readEA link
(longtermrisk.org)

Why the Irre­ducible Nor­ma­tivity Wager (Mostly) Fails

Lukas_Gloor14 Jun 2020 13:33 UTC
25 points
13 comments10 min readEA link

Chris­tian Tarsney on fu­ture bias and a pos­si­ble solu­tion to moral fanaticism

80000_Hours5 May 2021 19:38 UTC
7 points
0 comments113 min readEA link

Seek­ing feed­back on new EA-al­igned eco­nomics paper

jh21 Oct 2021 21:19 UTC
35 points
13 comments1 min readEA link

Against the nor­ma­tive re­al­ist’s wager

Joe_Carlsmith13 Oct 2022 16:36 UTC
27 points
5 comments23 min readEA link

Nor­ma­tive Uncer­tainty and the Depen­dence Problem

Gordon Seidoh Worley23 Mar 2020 17:29 UTC
15 points
4 comments2 min readEA link
(philpapers.org)

Vive la Différence? Struc­tural Diver­sity as a Challenge for Me­tanor­ma­tive Theories

Gordon Seidoh Worley26 May 2020 0:45 UTC
17 points
0 comments1 min readEA link
(philpapers.org)

Nor­ma­tive Uncer­tainty and Prob­a­bil­is­tic Mo­ral Knowledge

Gordon Seidoh Worley11 Nov 2019 20:26 UTC
13 points
0 comments1 min readEA link
(philpapers.org)

Amanda Askell: The moral value of information

EA Global2 Jun 2017 8:48 UTC
19 points
0 comments13 min readEA link
(www.youtube.com)

Chris­tian Tarsney on fu­ture bias and a pos­si­ble solu­tion to moral fanaticism

Pablo6 May 2021 10:39 UTC
26 points
6 comments1 min readEA link
(80000hours.org)

Prac­ti­cal ethics given moral uncertainty

William_MacAskill31 Jan 2012 5:00 UTC
23 points
5 comments3 min readEA link

Mo­ral­ity vs re­lated concepts

MichaelA🔸10 Feb 2020 8:02 UTC
19 points
2 comments8 min readEA link

Han­dling Mo­ral Uncer­tainty with Aver­age vs. To­tal Utili­tar­i­anism: One Method That Ap­par­ently *Doesn’t* Work (But Seemed Like it Should)

Marcel D5 Jan 2023 22:18 UTC
10 points
0 comments8 min readEA link

How might we be wildly wrong?

William_MacAskill4 Sep 2013 19:19 UTC
3 points
0 comments2 min readEA link

On “fringe” ideas

Kelsey Piper19 Aug 2019 7:00 UTC
250 points
6 comments5 min readEA link
(theunitofcaring.tumblr.com)

Shar­ing the World with Digi­tal Minds

Aaron Gertler 🔸1 Dec 2020 8:00 UTC
12 points
1 comment1 min readEA link
(www.nickbostrom.com)

“Aligned with who?” Re­sults of sur­vey­ing 1,000 US par­ti­ci­pants on AI values

Holly Morgan21 Mar 2023 22:07 UTC
40 points
0 comments2 min readEA link
(www.lesswrong.com)

Tak­ing Uncer­tainty Se­ri­ously (or, Why Tools Mat­ter)

Bob Fischer19 Jul 2024 10:30 UTC
115 points
4 comments8 min readEA link

Up­date on the Global Pri­ori­ties In­sti­tute’s (GPI) activities

Global Priorities Institute24 Dec 2019 13:51 UTC
23 points
0 comments7 min readEA link

Will MacAskill on nor­ma­tive uncertainty

lukeprog9 Apr 2014 15:27 UTC
2 points
0 comments8 min readEA link

Causes and Uncer­tainty: Re­think­ing Value in Expectation

Bob Fischer11 Oct 2023 9:15 UTC
220 points
29 comments15 min readEA link

[Linkpost] My moral view: Re­duc­ing suffer­ing, ‘how to be’ as fun­da­men­tal to moral­ity, no pos­i­tive value, cons of grand the­ory, and more—By Si­mon Knutsson

Alistair Webster25 Aug 2023 12:53 UTC
35 points
2 comments2 min readEA link
(centerforreducingsuffering.org)

EA read­ing list: moral un­cer­tainty, moral co­op­er­a­tion, and val­ues spreading

richard_ngo3 Aug 2020 9:21 UTC
14 points
0 comments1 min readEA link

Ideal gov­er­nance (for com­pa­nies, coun­tries and more)

Holden Karnofsky7 Apr 2022 16:54 UTC
80 points
19 comments14 min readEA link

Some thoughts on mod­er­a­tion in do­ing good

Vasco Grilo🔸20 Jan 2024 9:28 UTC
34 points
2 comments10 min readEA link
(80000hours.org)

Why I Don’t Ac­count for Mo­ral Uncertainty

[deleted]11 Jan 2015 3:52 UTC
25 points
16 comments2 min readEA link

Mo­ral Trade, Im­pact Distri­bu­tions and Large Worlds

Larks20 Sep 2024 3:45 UTC
25 points
1 comment4 min readEA link

Solu­tion to the two en­velopes prob­lem for moral weights

MichaelStJules19 Feb 2024 0:15 UTC
66 points
26 comments27 min readEA link

Gra­da­tions of moral weight

MichaelStJules29 Feb 2024 23:08 UTC
13 points
0 comments10 min readEA link

Mak­ing de­ci­sions when both morally and em­piri­cally uncertain

MichaelA🔸2 Jan 2020 7:08 UTC
22 points
4 comments20 min readEA link

“Ex­is­ten­tial Risk” is badly named and leads to nar­row fo­cus on as­tro­nom­i­cal waste

freedomandutility22 Aug 2022 20:25 UTC
38 points
2 comments2 min readEA link

Re­duc­ing x-risk might be ac­tively harmful

MountainPath18 Nov 2024 14:18 UTC
22 points
7 comments1 min readEA link

Uncer­tainty and Wild An­i­mal Welfare

abrahamrowe19 Jul 2019 13:33 UTC
24 points
0 comments8 min readEA link

Is Our Uni­verse A New­comb’s Para­dox Si­mu­la­tion?

Jordan Arel15 May 2022 7:28 UTC
16 points
8 comments2 min readEA link

[Question] De­ci­sion mak­ing un­der model am­bi­guity, moral un­cer­tainty, and other agents with free will?

Jobst Heitzig (vodle.it)2 Nov 2022 12:52 UTC
16 points
1 comment8 min readEA link

Sor­ti­tion Model of Mo­ral Uncertainty

Bob Jacobs 🔸4 Oct 2020 20:58 UTC
33 points
5 comments2 min readEA link

Stu­art Rus­sell Hu­man Com­pat­i­ble AI Roundtable with Allan Dafoe, Rob Re­ich, & Ma­ri­etje Schaake

Mahendra Prasad11 Feb 2021 7:43 UTC
16 points
0 comments1 min readEA link

The Wages of North-At­lantic Bias

Sach Wry19 Aug 2022 12:34 UTC
8 points
2 comments17 min readEA link

[Question] What should we ac­tu­ally do in re­sponse to moral un­cer­tainty?

John Bridge 🔸30 May 2022 9:48 UTC
4 points
5 comments1 min readEA link

Pod­cast: Krister Bykvist on moral un­cer­tainty, ra­tio­nal­ity, metaethics, AI and fu­ture pop­u­la­tions

Gus Docker21 Oct 2021 15:17 UTC
8 points
0 comments1 min readEA link
(www.utilitarianpodcast.com)

[Question] Does any thor­ough dis­cus­sion of moral par­li­a­ments ex­ist?

richard_ngo6 Sep 2019 15:33 UTC
36 points
10 comments1 min readEA link

Cri­tique of MacAskill’s “Is It Good to Make Happy Peo­ple?”

Magnus Vinding23 Aug 2022 9:21 UTC
222 points
115 comments8 min readEA link

Meth­ods for im­prov­ing un­cer­tainty anal­y­sis in EA cost-effec­tive­ness models

Froolow29 Aug 2022 18:07 UTC
127 points
8 comments37 min readEA link

Th­e­sis Sum­mary and Open Ques­tions: Nor­ma­tive Uncer­tainty and In­for­ma­tion Value

rileyharris27 Apr 2022 8:46 UTC
32 points
0 comments3 min readEA link

Refin­ing My Mo­ral Values Us­ing Steel­man Solitaire

Evan LaForge2 Feb 2024 10:07 UTC
19 points
3 comments9 min readEA link

Con­struc­tive Crit­i­cism of Mo­ral Uncer­tainty (book)

mariushobbhahn4 Jun 2021 6:04 UTC
28 points
3 comments5 min readEA link

Fair Col­lec­tive Effec­tive Altruism

Jobst Heitzig (vodle.it)28 Nov 2022 13:35 UTC
6 points
1 comment5 min readEA link
(www.lesswrong.com)

Ajeya Co­tra: Im­ple­ment­ing cause pri­ori­ti­sa­tion at Open Philanthropy

EA Global3 Nov 2017 7:43 UTC
14 points
0 comments1 min readEA link
(www.youtube.com)

Re­view and Sum­mary of ‘Mo­ral Uncer­tainty’

finm2 Oct 2020 15:41 UTC
37 points
3 comments1 min readEA link
(www.finmoorhouse.com)

A bar­gain­ing-the­o­retic ap­proach to moral uncertainty

Global Priorities Institute8 Aug 2019 13:41 UTC
3 points
0 comments3 min readEA link
(globalprioritiesinstitute.org)

Non-Con­se­quen­tial­ist Con­sid­er­a­tions For Cause-Pri­oritza­tion Part 1

Parker_Whitfill28 Nov 2018 22:22 UTC
14 points
2 comments15 min readEA link

What you pri­ori­tise is mostly moral intuition

JamesÖz24 Dec 2022 12:06 UTC
74 points
8 comments12 min readEA link

Con­se­quen­tial­ism as an ex­per­i­men­tal discipline

vandermude29 Nov 2022 6:07 UTC
9 points
2 comments3 min readEA link

Three tricky bi­ases: hu­man bias, ex­is­tence bias, and happy bias

Steven Rouk2 Feb 2024 14:20 UTC
52 points
0 comments11 min readEA link

Reflec­tive Equil­ibria and the Hunt for a For­mal­ized Pragmatism

BenjaminCaulfield7 Jun 2023 22:55 UTC
4 points
1 comment12 min readEA link

Mo­ral un­cer­tainty and pub­lic jus­tifi­ca­tion (Bar­ret and Sch­midt, 2021)

Global Priorities Institute22 Oct 2021 9:57 UTC
4 points
3 comments3 min readEA link
(globalprioritiesinstitute.org)

Non-util­i­tar­ian effec­tive altruism

keir bradwell29 Jan 2023 6:07 UTC
42 points
10 comments17 min readEA link
(keirbradwell.substack.com)

Thought ex­per­i­ment: Trad­ing off risk, in­tra­gen­er­a­tional and in­ter­gen­er­a­tional in­equal­ity, and fairness

Jobst Heitzig (vodle.it)2 Sep 2023 23:32 UTC
9 points
6 comments3 min readEA link

Le idee “es­treme”

EA Italy31 Dec 2022 4:15 UTC
1 point
0 comments5 min readEA link

[Opz­ionale] La pos­si­bil­ità di una Catas­trofe Mo­rale in atto (Ri­as­sunto)

EA Italy31 Dec 2022 4:17 UTC
1 point
0 comments5 min readEA link

EA thoughts from Is­rael-Ha­mas war

ezrah6 Dec 2023 8:53 UTC
271 points
64 comments9 min readEA link

Large episte­molog­i­cal con­cerns I should maybe have about EA a priori

Luise7 Jun 2023 14:11 UTC
115 points
16 comments8 min readEA link

Com­plex­ity of value but not dis­value im­plies more fo­cus on s-risk. Mo­ral un­cer­tainty and prefer­ence util­i­tar­i­anism also do.

Chi13 Feb 2024 22:24 UTC
95 points
7 comments2 min readEA link

Balanc­ing safety and waste

Daniel_Friedrich17 Mar 2024 10:57 UTC
6 points
0 comments7 min readEA link

The Mo­ral Two En­velopes Prob­lem and the Mo­ral Weights Project

Derek Shiller15 Oct 2024 18:12 UTC
91 points
20 comments14 min readEA link

[Linkpost] Leif We­nar’s The Deaths of Effec­tive Altruism

Earthling27 Mar 2024 15:35 UTC
46 points
58 comments1 min readEA link
(www.wired.com)

Mo­ral Con­sid­er­a­tions In De­sign­ing AI Systems

Hans Gundlach5 Jul 2024 18:13 UTC
8 points
1 comment7 min readEA link

Are we as rigor­ous in ad­dress­ing Utili­tar­i­anism’s the­o­ret­i­cal challenges as we are with its prac­ti­cal im­pli­ca­tions?

Silvan10 Jul 2024 13:08 UTC
15 points
5 comments5 min readEA link

Why is it so hard to know if you’re helping?

Vasco Grilo🔸16 Oct 2024 16:08 UTC
59 points
6 comments12 min readEA link
(www.theintrinsicperspective.com)

A brief the­ory of why we think things are good or bad

David Johnston20 Oct 2024 12:15 UTC
5 points
1 comment4 min readEA link

Bar­gain­ing among worldviews

Hayley Clatterbuck18 Oct 2024 18:32 UTC
57 points
5 comments12 min readEA link
No comments.