RSS

Risks from malev­olent actors

TagLast edit: Sep 8, 2022, 9:55 AM by Pablo

Risks from malevolent actors are risks posed by individuals with narcissistic, psychopathic, sadistic, or other socially harmful personality traits.[1]

Malevolence as a risk

Malevolence may be operationalized as the general factor that accounts for the observed correlations between negative personality traits—the so-called “dark core” of personality.[2] People who score unusually highly on this general factor could pose serious risks for the long-term future of humanity, including existential risks and risks of astronomical suffering. Such people appear to be more likely than average to attain positions of power in government and industry. And conditional on gaining this influence, they are also much more likely to cause major harm.

The types of risks malevolent actors in power might pose vary greatly. They include the spread of extremist and dangerous ideologies, the escalation of great power conflicts, the formation of totalitarian regimes, and the undermining of public institutions and global coordination. Since malevolence has a broadly negative influence, it is probably best regarded as a risk factor rather than as a distinct kind of risk.

Interventions against malevolent actors

In a pioneering essay on the topic, David Althaus and Tobias Baumann have proposed a number of interventions aimed at reducing risks from malevolent actors.[1] Their proposals include advancing the science of malevolence—by aligning constructs with the morally relevant forms of malevolence and by developing manipulation-proof measures of it—, and promoting political reforms to make the rise of malevolent actors less probable. Because malevolent actors likely pose the gravest risks in scenarios where they gain access to transformative technology, the authors also propose shaping the development of those technologies so as to mitigate the harms posed by these individuals, such as by reducing the presence of malevolent traits in AI training environments and by ensuring that candidates for whole brain emulation score low on measures of malevolence.

Evaluation

80,000 Hours rates risks from malevolent actors a “potential highest priority area”: an issue that, if more thoroughly examined, could rank as a top global challenge.[3]

Further reading

Althaus, David & Tobias Baumann (2020) Reducing long-term risks from malevolent actors, Effective Altruism Forum, April 29.

Related entries

existential risk | existential risk factor | s-risk | transformative development

  1. ^

    Althaus, David & Tobias Baumann (2020) Reducing long-term risks from malevolent actors, Effective Altruism Forum, April 29.

  2. ^

    Moshagen, Morten, Benjamin E. Hilbig & Ingo Zettler (2018) The dark core of personality, Psychological Review, vol. 125, pp. 656–688.

  3. ^

    80,000 Hours (2022) Our current list of pressing world problems, 80,000 Hours.

Re­duc­ing long-term risks from malev­olent actors

David_AlthausApr 29, 2020, 8:55 AM
344 points
93 comments37 min readEA link

Rea­sons for op­ti­mism about mea­sur­ing malev­olence to tackle x- and s-risks

Jamie_HarrisApr 2, 2024, 10:26 AM
85 points
12 comments8 min readEA link

Can a ter­ror­ist at­tack cause hu­man ex­tinc­tion? Not on priors

Vasco Grilo🔸Dec 2, 2023, 8:20 AM
43 points
9 comments15 min readEA link

Some gov­er­nance re­search ideas to pre­vent malev­olent con­trol over AGI and why this might mat­ter a hell of a lot

Jim BuhlerMay 23, 2023, 1:07 PM
63 points
5 comments16 min readEA link

[Question] Most harm­ful peo­ple in his­tory?

SiebeRozendalSep 11, 2022, 3:04 AM
17 points
9 comments1 min readEA link

Nar­ra­tion: Re­duc­ing long-term risks from malev­olent actors

D0TheMathJul 15, 2021, 4:26 PM
23 points
0 comments1 min readEA link
(anchor.fm)

New book on s-risks

Tobias_BaumannOct 26, 2022, 12:04 PM
292 points
27 comments1 min readEA link

Assess­ing the Danger­ous­ness of Malev­olent Ac­tors in AGI Gover­nance: A Pre­limi­nary Exploration

Callum HinchcliffeOct 14, 2023, 9:18 PM
28 points
4 comments9 min readEA link

New Book: “Rea­soned Poli­tics” + Why I have writ­ten a book about politics

Magnus VindingMar 3, 2022, 11:31 AM
95 points
9 comments5 min readEA link

What is malev­olence? On the na­ture, mea­sure­ment, and dis­tri­bu­tion of dark traits

David_AlthausOct 23, 2024, 8:41 AM
107 points
6 comments52 min readEA link

In­for­ma­tion se­cu­rity con­sid­er­a­tions for AI and the long term future

Jeffrey LadishMay 2, 2022, 8:53 PM
134 points
8 comments11 min readEA link

Prob­lem ar­eas be­yond 80,000 Hours’ cur­rent pri­ori­ties

Arden KoehlerJun 22, 2020, 12:49 PM
281 points
62 comments15 min readEA link

How big are risks from non-state ac­tors? Base rates for ter­ror­ist attacks

rosehadsharFeb 16, 2022, 10:20 AM
54 points
3 comments19 min readEA link

Will re­leas­ing the weights of large lan­guage mod­els grant wide­spread ac­cess to pan­demic agents?

Jeff Kaufman 🔸Oct 30, 2023, 5:42 PM
56 points
18 comments1 min readEA link
(arxiv.org)

So­nia Ben Oua­grham-Gorm­ley on Bar­ri­ers to Bioweapons

Vasco Grilo🔸Feb 15, 2024, 5:58 PM
21 points
0 comments1 min readEA link
(hearthisidea.com)

Why we may ex­pect our suc­ces­sors not to care about suffering

Jim BuhlerJul 10, 2023, 1:54 PM
63 points
31 comments8 min readEA link

2024 S-risk In­tro Fellowship

Center on Long-Term RiskOct 12, 2023, 7:14 PM
90 points
2 comments1 min readEA link

New s-risks au­dio­book available now

Alistair WebsterMay 24, 2023, 8:27 PM
87 points
3 comments1 min readEA link
(centerforreducingsuffering.org)

Against Anony­mous Hit Pieces

Anti-OmegaJun 18, 2023, 7:36 PM
−25 points
3 comments1 min readEA link

CLR Sum­mer Re­search Fel­low­ship 2024

Center on Long-Term RiskFeb 15, 2024, 6:26 PM
89 points
2 comments8 min readEA link

[Question] Are there highly lev­er­aged dona­tion op­por­tu­ni­ties to pre­vent wars and dic­ta­tor­ships?

Dawn DrescherFeb 26, 2022, 3:31 AM
58 points
8 comments1 min readEA link

In­sti­tu­tions Can­not Res­train Dark-Triad AI Exploitation

RemmeltDec 27, 2022, 10:34 AM
8 points
0 comments1 min readEA link

Bulk­ing in­for­ma­tion ad­di­tion­al­ities in global de­vel­op­ment for medium-term lo­cal prosperity

brb243Apr 11, 2022, 5:52 PM
4 points
0 comments4 min readEA link

Reimag­in­ing Malev­olence: A Primer on Malev­olence and Im­pli­ca­tions for EA

Kenneth_DiaoApr 11, 2024, 12:50 PM
28 points
3 comments44 min readEA link

De­lay, De­tect, Defend: Prepar­ing for a Fu­ture in which Thou­sands Can Re­lease New Pan­demics by Kevin Esvelt

JeremyNov 15, 2022, 4:23 PM
177 points
7 comments1 min readEA link
(dam.gcsp.ch)
No comments.