RSS

Mo­ral uncertainty

TagLast edit: 2 Aug 2023 9:04 UTC by Leo

Moral uncertainty is uncertainty about how to act given lack of certainty in any one moral theory, as well as the study of how we ought to act given this uncertainty.

We are sometimes uncertain about empirical facts, such as whether it will rain tomorrow. But we can also be uncertain about moral facts, such as whether it’s wrong to steal, or how we should value the well-being of animals. Uncertainty about whether it’s wrong to steal is uncertainty about moral or normative facts, while uncertainty about how we should value the well-being of animals is uncertainty about axiological or value facts.

Can moral uncertainty be rational, and what should we do in response to it? We might think that it can never be rational to be uncertain about normative or axiological facts, because such facts are, like mathematical facts, knowable a priori. Nevertheless it seems that agents like ourselves are uncertain about non-trivial mathematical facts, and that we are also uncertain about normative and axiological facts. Given this, it seems necessary to develop some account of how we ought to act under moral uncertainty.

Several such accounts have been formulated in recent years:

Two important problems facing the view that moral uncertainty can affect how we ought to act are the regress problem—the problem that we will be uncertain not only about typical moral questions, but also about which approach to moral uncertainty is correct, and so on ad infinitum[5] - and the problem of intertheoretic comparisons—the problem that there seems to be no principled ways to make comparisons between different moral theories.[6]

Further reading

MacAskill, William, Krister Bykvist, & Toby Ord (2020) Moral Uncertainty, Oxford: Oxford University Press.

Related entries

altruistic wager | decision-theoretic uncertainty | ethics of existential risk | expected value | fanaticism | moral trade | normative uncertainty

  1. ^

    Weatherson, Brian (2014) Running risks morally, Philosophical Studies, 167: 141–63.

  2. ^

    Gustafsson, Johan E. & Olle Torpman (2014) In defence of my favourite theory, Pacific Philosophical Quarterly, 95(2): 159-174.

  3. ^

    MacAskill, William (2014) Normative uncertainty, Doctoral dissertation, University of Oxford.

  4. ^

    Bostrom, Nick (2009) Moral uncertainty—towards a solution?, Overcoming Bias, January 1.

  5. ^
  6. ^

    Nissan-Rozen, Ittay (2015) Against moral hedging, Economics and Philosophy, 31(3): 349-369.

[Linkpost] Leif We­nar’s The Deaths of Effec­tive Altruism

Arden Wiese27 Mar 2024 15:35 UTC
38 points
58 comments1 min readEA link
(www.wired.com)

Balanc­ing safety and waste

Daniel_Friedrich17 Mar 2024 10:57 UTC
6 points
0 comments7 min readEA link

Gra­da­tions of moral weight

MichaelStJules29 Feb 2024 23:08 UTC
13 points
0 comments10 min readEA link

Solu­tion to the two en­velopes prob­lem for moral weights

MichaelStJules19 Feb 2024 0:15 UTC
61 points
26 comments27 min readEA link

Com­plex­ity of value but not dis­value im­plies more fo­cus on s-risk. Mo­ral un­cer­tainty and prefer­ence util­i­tar­i­anism also do.

Chi13 Feb 2024 22:24 UTC
95 points
7 comments2 min readEA link

Three tricky bi­ases: hu­man bias, ex­is­tence bias, and happy bias

Steven Rouk2 Feb 2024 14:20 UTC
41 points
0 comments11 min readEA link

Refin­ing My Mo­ral Values Us­ing Steel­man Solitaire

Evan LaForge2 Feb 2024 10:07 UTC
16 points
3 comments9 min readEA link

Some thoughts on mod­er­a­tion in do­ing good

Vasco Grilo20 Jan 2024 9:28 UTC
34 points
2 comments10 min readEA link
(80000hours.org)

EA thoughts from Is­rael-Ha­mas war

ezrah6 Dec 2023 8:53 UTC
271 points
64 comments9 min readEA link

The op­tion value ar­gu­ment doesn’t work when it’s most needed

Winston24 Oct 2023 19:40 UTC
122 points
6 comments6 min readEA link

If Con­trac­tu­al­ism, Then AMF

Bob Fischer13 Oct 2023 18:03 UTC
62 points
54 comments24 min readEA link

Causes and Uncer­tainty: Re­think­ing Value in Expectation

Bob Fischer11 Oct 2023 9:15 UTC
218 points
29 comments15 min readEA link

Thought ex­per­i­ment: Trad­ing off risk, in­tra­gen­er­a­tional and in­ter­gen­er­a­tional in­equal­ity, and fairness

Jobst Heitzig (vodle.it)2 Sep 2023 23:32 UTC
9 points
6 comments3 min readEA link

[Linkpost] My moral view: Re­duc­ing suffer­ing, ‘how to be’ as fun­da­men­tal to moral­ity, no pos­i­tive value, cons of grand the­ory, and more—By Si­mon Knutsson

Alistair Webster25 Aug 2023 12:53 UTC
35 points
2 comments2 min readEA link
(centerforreducingsuffering.org)

Reflec­tive Equil­ibria and the Hunt for a For­mal­ized Pragmatism

BenjaminCaulfield7 Jun 2023 22:55 UTC
4 points
1 comment13 min readEA link

Large episte­molog­i­cal con­cerns I should maybe have about EA a priori

Luise7 Jun 2023 14:11 UTC
107 points
16 comments8 min readEA link

Re­solv­ing moral un­cer­tainty with randomization

Bob Jacobs29 Mar 2023 10:10 UTC
29 points
3 comments10 min readEA link

“Aligned with who?” Re­sults of sur­vey­ing 1,000 US par­ti­ci­pants on AI values

Holly Morgan21 Mar 2023 22:07 UTC
40 points
0 comments2 min readEA link
(www.lesswrong.com)

Non-util­i­tar­ian effec­tive altruism

keir bradwell29 Jan 2023 6:07 UTC
41 points
10 comments17 min readEA link
(keirbradwell.substack.com)

Han­dling Mo­ral Uncer­tainty with Aver­age vs. To­tal Utili­tar­i­anism: One Method That Ap­par­ently *Doesn’t* Work (But Seemed Like it Should)

Harrison Durland5 Jan 2023 22:18 UTC
10 points
0 comments8 min readEA link

[Opz­ionale] La pos­si­bil­ità di una Catas­trofe Mo­rale in atto (Ri­as­sunto)

EA Italy31 Dec 2022 4:17 UTC
1 point
0 comments5 min readEA link

Le idee “es­treme”

EA Italy31 Dec 2022 4:15 UTC
1 point
0 comments5 min readEA link

What you pri­ori­tise is mostly moral intuition

James Özden24 Dec 2022 12:06 UTC
73 points
8 comments12 min readEA link

A Case for Vol­un­tary Abor­tion Reduction

Ariel Simnegar20 Dec 2022 13:23 UTC
28 points
149 comments13 min readEA link

Con­se­quen­tial­ism as an ex­per­i­men­tal discipline

vandermude29 Nov 2022 6:07 UTC
9 points
2 comments3 min readEA link

Fair Col­lec­tive Effec­tive Altruism

Jobst Heitzig (vodle.it)28 Nov 2022 13:35 UTC
5 points
1 comment5 min readEA link
(www.lesswrong.com)

Deon­tol­ogy and virtue ethics as “effec­tive the­o­ries” of con­se­quen­tial­ist ethics

Jan_Kulveit17 Nov 2022 9:20 UTC
55 points
11 comments10 min readEA link

[Question] De­ci­sion mak­ing un­der model am­bi­guity, moral un­cer­tainty, and other agents with free will?

Jobst Heitzig (vodle.it)2 Nov 2022 12:52 UTC
16 points
1 comment8 min readEA link

The prop­erty rights ap­proach to moral uncertainty

Harry R. Lloyd14 Oct 2022 16:49 UTC
31 points
14 comments2 min readEA link
(www.happierlivesinstitute.org)

Against the nor­ma­tive re­al­ist’s wager

Joe_Carlsmith13 Oct 2022 16:36 UTC
27 points
5 comments23 min readEA link

A dilemma for Max­i­mize Ex­pected Choice­wor­thi­ness (MEC)

Calvin_Baker1 Sep 2022 14:46 UTC
33 points
7 comments9 min readEA link

Meth­ods for im­prov­ing un­cer­tainty anal­y­sis in EA cost-effec­tive­ness models

Froolow29 Aug 2022 18:07 UTC
127 points
8 comments36 min readEA link

Cri­tique of MacAskill’s “Is It Good to Make Happy Peo­ple?”

Magnus Vinding23 Aug 2022 9:21 UTC
219 points
115 comments8 min readEA link

“Ex­is­ten­tial Risk” is badly named and leads to nar­row fo­cus on as­tro­nom­i­cal waste

freedomandutility22 Aug 2022 20:25 UTC
38 points
2 comments2 min readEA link

The Wages of North-At­lantic Bias

Sach Wry19 Aug 2022 12:34 UTC
8 points
2 comments17 min readEA link

A philo­soph­i­cal re­view of Open Philan­thropy’s Cause Pri­ori­ti­sa­tion Framework

MichaelPlant15 Jul 2022 8:21 UTC
155 points
11 comments25 min readEA link
(www.happierlivesinstitute.org)

Wheel­ing and deal­ing: An in­ter­nal bar­gain­ing ap­proach to moral uncertainty

MichaelPlant10 Jul 2022 12:04 UTC
56 points
18 comments22 min readEA link

[Question] What should we ac­tu­ally do in re­sponse to moral un­cer­tainty?

John Bridge30 May 2022 9:48 UTC
4 points
5 comments1 min readEA link

Is Our Uni­verse A New­comb’s Para­dox Si­mu­la­tion?

Jordan Arel15 May 2022 7:28 UTC
16 points
8 comments2 min readEA link

Th­e­sis Sum­mary and Open Ques­tions: Nor­ma­tive Uncer­tainty and In­for­ma­tion Value

rileyharris27 Apr 2022 8:46 UTC
31 points
0 comments3 min readEA link

Ideal gov­er­nance (for com­pa­nies, coun­tries and more)

Holden Karnofsky7 Apr 2022 16:54 UTC
80 points
19 comments14 min readEA link

Mo­ral Anti-Real­ism: In­tro­duc­tion & Summary

Lukas_Gloor2 Apr 2022 14:32 UTC
49 points
1 comment7 min readEA link

The “Mo­ral Uncer­tainty” Rab­bit Hole, Fully Excavated

Lukas_Gloor1 Apr 2022 13:14 UTC
23 points
2 comments35 min readEA link

Mo­ral Uncer­tainty and Mo­ral Real­ism Are in Tension

Lukas_Gloor25 Jan 2022 11:07 UTC
39 points
4 comments12 min readEA link

Might stop­ping mis­car­riages be a very im­por­tant cause area?

SaraAzubuike5 Dec 2021 17:18 UTC
12 points
13 comments2 min readEA link

Stan­ley the Uncer­tain [Creative Writ­ing Con­test]

Akash28 Oct 2021 20:12 UTC
33 points
1 comment3 min readEA link

Mo­ral un­cer­tainty and pub­lic jus­tifi­ca­tion (Bar­ret and Sch­midt, 2021)

Global Priorities Institute22 Oct 2021 9:57 UTC
4 points
3 comments3 min readEA link
(globalprioritiesinstitute.org)

Seek­ing feed­back on new EA-al­igned eco­nomics paper

jh21 Oct 2021 21:19 UTC
35 points
13 comments1 min readEA link

Pod­cast: Krister Bykvist on moral un­cer­tainty, ra­tio­nal­ity, metaethics, AI and fu­ture pop­u­la­tions

Gus Docker21 Oct 2021 15:17 UTC
8 points
0 comments1 min readEA link
(www.utilitarianpodcast.com)

A longter­mist cri­tique of “The ex­pected value of ex­tinc­tion risk re­duc­tion is pos­i­tive”

Anthony DiGiovanni1 Jul 2021 21:01 UTC
125 points
10 comments32 min readEA link

Con­struc­tive Crit­i­cism of Mo­ral Uncer­tainty (book)

mariushobbhahn4 Jun 2021 6:04 UTC
28 points
3 comments5 min readEA link

Chris­tian Tarsney on fu­ture bias and a pos­si­ble solu­tion to moral fanaticism

Pablo6 May 2021 10:39 UTC
26 points
6 comments1 min readEA link
(80000hours.org)

Chris­tian Tarsney on fu­ture bias and a pos­si­ble solu­tion to moral fanaticism

80000_Hours5 May 2021 19:38 UTC
7 points
0 comments114 min readEA link

Mo­ral plu­ral­ism and longter­mism | Sunyshore

BrownHairedEevee17 Apr 2021 0:14 UTC
26 points
0 comments6 min readEA link
(sunyshore.substack.com)

SoGive’s moral weights—please take part!

Sanjay5 Apr 2021 22:47 UTC
26 points
2 comments2 min readEA link

Stu­art Rus­sell Hu­man Com­pat­i­ble AI Roundtable with Allan Dafoe, Rob Re­ich, & Ma­ri­etje Schaake

Mahendra Prasad11 Feb 2021 7:43 UTC
16 points
0 comments1 min readEA link

[Pod­cast] Ajeya Co­tra on wor­ld­view di­ver­sifi­ca­tion and how big the fu­ture could be

BrownHairedEevee22 Jan 2021 23:57 UTC
57 points
20 comments1 min readEA link
(80000hours.org)

Even Allo­ca­tion Strat­egy un­der High Model Ambiguity

MichaelStJules31 Dec 2020 9:10 UTC
15 points
3 comments3 min readEA link

Shar­ing the World with Digi­tal Minds

Aaron Gertler1 Dec 2020 8:00 UTC
12 points
1 comment1 min readEA link
(www.nickbostrom.com)

Sor­ti­tion Model of Mo­ral Uncertainty

Bob Jacobs4 Oct 2020 20:58 UTC
33 points
5 comments2 min readEA link

Re­view and Sum­mary of ‘Mo­ral Uncer­tainty’

finm2 Oct 2020 15:41 UTC
37 points
3 comments1 min readEA link
(www.finmoorhouse.com)

Hedg­ing against deep and moral uncertainty

MichaelStJules12 Sep 2020 23:44 UTC
83 points
13 comments9 min readEA link

New book: Mo­ral Uncer­tainty by MacAskill, Ord & Bykvist

frankieaw10 Sep 2020 16:22 UTC
85 points
2 comments1 min readEA link

EA read­ing list: clue­less­ness and epistemic modesty

richard_ngo3 Aug 2020 9:23 UTC
23 points
3 comments1 min readEA link

EA read­ing list: moral un­cer­tainty, moral co­op­er­a­tion, and val­ues spreading

richard_ngo3 Aug 2020 9:21 UTC
14 points
0 comments1 min readEA link

Me­taeth­i­cal Fa­nat­i­cism (Dialogue)

Lukas_Gloor17 Jun 2020 12:33 UTC
29 points
10 comments15 min readEA link

Why the Irre­ducible Nor­ma­tivity Wager (Mostly) Fails

Lukas_Gloor14 Jun 2020 13:33 UTC
25 points
13 comments10 min readEA link

Ex­pected value un­der nor­ma­tive uncertainty

Gordon Seidoh Worley8 Jun 2020 15:45 UTC
16 points
1 comment1 min readEA link
(philpapers.org)

Vive la Différence? Struc­tural Diver­sity as a Challenge for Me­tanor­ma­tive Theories

Gordon Seidoh Worley26 May 2020 0:45 UTC
17 points
0 comments1 min readEA link
(philpapers.org)

Nor­ma­tive Uncer­tainty and the Depen­dence Problem

Gordon Seidoh Worley23 Mar 2020 17:29 UTC
15 points
4 comments2 min readEA link
(philpapers.org)

Quotes about the long reflection

MichaelA5 Mar 2020 7:48 UTC
55 points
14 comments13 min readEA link

Mo­ral­ity vs re­lated concepts

MichaelA10 Feb 2020 8:02 UTC
19 points
2 comments8 min readEA link

Mak­ing de­ci­sions when both morally and em­piri­cally uncertain

MichaelA2 Jan 2020 7:08 UTC
22 points
4 comments20 min readEA link

Mak­ing de­ci­sions un­der moral uncertainty

MichaelA1 Jan 2020 13:02 UTC
44 points
8 comments17 min readEA link

Up­date on the Global Pri­ori­ties In­sti­tute’s (GPI) activities

Global Priorities Institute24 Dec 2019 13:51 UTC
23 points
0 comments7 min readEA link

Peter Singer on the 80K podcast

Jon_Behar18 Dec 2019 17:17 UTC
31 points
0 comments1 min readEA link
(80000hours.org)

New re­search on moral weights

GiveWell4 Dec 2019 11:57 UTC
42 points
4 comments2 min readEA link
(blog.givewell.org)

Nor­ma­tive Uncer­tainty and Prob­a­bil­is­tic Mo­ral Knowledge

Gordon Seidoh Worley11 Nov 2019 20:26 UTC
13 points
0 comments1 min readEA link
(philpapers.org)

[Question] Does any thor­ough dis­cus­sion of moral par­li­a­ments ex­ist?

richard_ngo6 Sep 2019 15:33 UTC
36 points
10 comments1 min readEA link

On “fringe” ideas

Kelsey Piper19 Aug 2019 7:00 UTC
230 points
5 comments5 min readEA link
(theunitofcaring.tumblr.com)

A bar­gain­ing-the­o­retic ap­proach to moral uncertainty

Global Priorities Institute8 Aug 2019 13:41 UTC
3 points
0 comments3 min readEA link
(globalprioritiesinstitute.org)

The Pos­si­bil­ity of an On­go­ing Mo­ral Catas­tro­phe (Sum­mary)

Linch2 Aug 2019 21:55 UTC
129 points
15 comments4 min readEA link
(docs.google.com)

Uncer­tainty and Wild An­i­mal Welfare

abrahamrowe19 Jul 2019 13:33 UTC
24 points
0 comments8 min readEA link

Ar­gu­ments for moral indefinability

richard_ngo8 Feb 2019 11:09 UTC
33 points
8 comments7 min readEA link
(thinkingcomplete.blogspot.com)

The ex­pected value of ex­tinc­tion risk re­duc­tion is positive

JanB9 Dec 2018 8:00 UTC
69 points
22 comments39 min readEA link

Non-Con­se­quen­tial­ist Con­sid­er­a­tions For Cause-Pri­oritza­tion Part 1

Parker_Whitfill28 Nov 2018 22:22 UTC
14 points
2 comments15 min readEA link

Wrong by Induction

Ales_Flidr6 Sep 2018 22:00 UTC
20 points
2 comments6 min readEA link

Light­ning talks from EA Global: San Fran­cisco (2018)

EA Global8 Jun 2018 20:51 UTC
7 points
0 comments11 min readEA link
(www.youtube.com)

Wor­ld­view uncertainty

Ronja_Lutz16 May 2018 14:02 UTC
15 points
5 comments8 min readEA link

Cause pri­ori­ti­za­tion for down­side-fo­cused value systems

Lukas_Gloor31 Jan 2018 14:47 UTC
75 points
10 comments49 min readEA link

Ajeya Co­tra: Im­ple­ment­ing cause pri­ori­ti­sa­tion at Open Philanthropy

EA Global3 Nov 2017 7:43 UTC
14 points
0 comments1 min readEA link
(www.youtube.com)

Amanda Askell: The moral value of information

EA Global2 Jun 2017 8:48 UTC
19 points
0 comments14 min readEA link
(www.youtube.com)

Why I Don’t Ac­count for Mo­ral Uncertainty

[deleted]11 Jan 2015 3:52 UTC
25 points
16 comments2 min readEA link

Will MacAskill on nor­ma­tive uncertainty

lukeprog9 Apr 2014 15:27 UTC
2 points
0 comments8 min readEA link

Brian To­masik – Rea­sons to Be Nice to Other Value Systems

Tessa17 Jan 2014 0:28 UTC
18 points
0 comments1 min readEA link
(longtermrisk.org)

How might we be wildly wrong?

William_MacAskill4 Sep 2013 19:19 UTC
3 points
0 comments2 min readEA link

Prac­ti­cal ethics given moral uncertainty

William_MacAskill31 Jan 2012 5:00 UTC
23 points
5 comments3 min readEA link
No comments.