AI alignment

TagLast edit: 22 Jul 2022 20:58 UTC by Leo

AI alignment is research on how to align AI systems with human or moral goals.

Evaluation

80,000 Hours rates AI alignment a “highest priority area”: a problem at the top of their ranking of global issues assessed by importance, tractability and neglectedness.^[1]

External links

AI Alignment Forum.

^
80,000 Hours (2021) Our current list of the most important world problems, 80,000 Hours.

2019 AI Alignment Literature Review and Charity Comparison

Larks19 Dec 2019 2:58 UTC

147 points

28 comments62 min readEA link

2018 AI Alignment Literature Review and Charity Comparison

Larks18 Dec 2018 4:48 UTC

118 points

28 comments63 min readEA link

Ben Garfinkel: How sure are we about this AI stuff?

bgarfinkel9 Feb 2019 19:17 UTC

128 points

20 comments18 min readEA link

AI Research Considerations for Human Existential Safety (ARCHES)

Andrew Critch21 May 2020 6:55 UTC

29 points

0 comments3 min readEA link

(acritch.com)

AGI Safety Fundamentals curriculum and application

richard_ngo20 Oct 2021 21:45 UTC

123 points

20 comments8 min readEA link

(docs.google.com)

Why AI alignment could be hard with modern deep learning

Ajeya21 Sep 2021 15:35 UTC

153 points

17 comments14 min readEA link

(www.cold-takes.com)

Disentangling arguments for the importance of AI safety

richard_ngo23 Jan 2019 14:58 UTC

63 points

14 comments8 min readEA link

Delegated agents in practice: How companies might end up selling AI services that act on behalf of consumers and coalitions, and what this implies for safety research

Remmelt26 Nov 2020 16:39 UTC

11 points

0 comments4 min readEA link

Why I prioritize moral circle expansion over reducing extinction risk through artificial intelligence alignment

Jacy20 Feb 2018 18:29 UTC

107 points

72 comments35 min readEA link

(www.sentienceinstitute.org)

DeepMind is hiring for the Scalable Alignment and Alignment Teams

Rohin Shah13 May 2022 12:19 UTC

102 points

0 comments9 min readEA link

My current thoughts on MIRI’s “highly reliable agent design” work

Daniel_Dewey7 Jul 2017 1:17 UTC

60 points

59 comments19 min readEA link

2017 AI Safety Literature Review and Charity Comparison

Larks20 Dec 2017 21:54 UTC

43 points

17 comments23 min readEA link

Preventing an AI-related catastrophe—Problem profile

Benjamin Hilton29 Aug 2022 18:49 UTC

138 points

18 comments4 min readEA link

(80000hours.org)

Crazy ideas sometimes do work

Aryeh Englander4 Sep 2021 3:27 UTC

71 points

8 comments1 min readEA link

AI alignment researchers may have a comparative advantage in reducing s-risks

Lukas_Gloor15 Feb 2023 13:01 UTC

79 points

5 comments13 min readEA link

Launching applications for AI Safety Careers Course India 2024

varun_agr1 May 2024 5:30 UTC

23 points

1 comment1 min readEA link

The academic contribution to AI safety seems large

Gavin30 Jul 2020 10:30 UTC

117 points

28 comments9 min readEA link

Hiring engineers and researchers to help align GPT-3

Paul_Christiano1 Oct 2020 18:52 UTC

107 points

19 comments3 min readEA link

2016 AI Risk Literature Review and Charity Comparison

Larks13 Dec 2016 4:36 UTC

57 points

12 comments28 min readEA link

[Link post] Coordination challenges for preventing AI conflict

stefan.torges9 Mar 2021 9:39 UTC

58 points

0 comments1 min readEA link

(longtermrisk.org)

What is it like doing AI safety work?

Kat Woods21 Feb 2023 19:24 UTC

99 points

2 comments10 min readEA link

Announcing AI Safety Support

Linda Linsefors19 Nov 2020 20:19 UTC

55 points

0 comments4 min readEA link

A tale of 2.5 orthogonality theses

Arepo1 May 2022 13:53 UTC

140 points

31 comments11 min readEA link

There are no coherence theorems

EJT20 Feb 2023 21:52 UTC

107 points

49 comments19 min readEA link

A central AI alignment problem: capabilities generalization, and the sharp left turn

So8res15 Jun 2022 14:19 UTC

51 points

2 comments10 min readEA link

[Question] What is most confusing to you about AI stuff?

Sam Clarke23 Nov 2021 16:00 UTC

25 points

15 comments1 min readEA link

From language to ethics by automated reasoning

Michele Campolo21 Nov 2021 15:16 UTC

8 points

0 comments6 min readEA link

Large Language Models as Fiduciaries to Humans

johnjnay24 Jan 2023 19:53 UTC

25 points

0 comments34 min readEA link

(papers.ssrn.com)

Draft report on existential risk from power-seeking AI

Joe_Carlsmith28 Apr 2021 21:41 UTC

88 points

34 comments1 min readEA link

Counterarguments to the basic AI risk case

Katja_Grace14 Oct 2022 20:30 UTC

284 points

23 comments34 min readEA link

Deceptive Alignment is <1% Likely by Default

DavidW21 Feb 2023 15:07 UTC

54 points

26 comments14 min readEA link

Cognitive Science/Psychology As a Neglected Approach to AI Safety

Kaj_Sotala5 Jun 2017 13:46 UTC

40 points

37 comments4 min readEA link

Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training

evhub12 Jan 2024 19:51 UTC

65 points

0 comments1 min readEA link

(arxiv.org)

Ngo and Yudkowsky on alignment difficulty

richard_ngo15 Nov 2021 22:47 UTC

71 points

13 comments94 min readEA link

TAI Safety Bibliographic Database

Jess_Riedel22 Dec 2020 16:03 UTC

61 points

9 comments17 min readEA link

AI alignment shouldn’t be conflated with AI moral achievement

Matthew_Barnett30 Dec 2023 3:08 UTC

111 points

15 comments5 min readEA link

[Linkpost] AI Alignment, Explained in 5 Points (updated)

Daniel_Eth18 Apr 2023 8:09 UTC

31 points

2 comments1 min readEA link

(medium.com)

My personal cruxes for working on AI safety

Buck13 Feb 2020 7:11 UTC

136 points

35 comments44 min readEA link

How do takeoff speeds affect the probability of bad outcomes from AGI?

KR7 Jul 2020 17:53 UTC

18 points

0 comments8 min readEA link

“Aligned with who?” Results of surveying 1,000 US participants on AI values

Holly Morgan21 Mar 2023 22:07 UTC

40 points

0 comments2 min readEA link

(www.lesswrong.com)

Train for incorrigibility, then reverse it (Shutdown Problem Contest Submission)

Daniel_Eth18 Jul 2023 8:26 UTC

16 points

0 comments2 min readEA link

Scrutinizing AI Risk (80K, #81) - v. quick summary

Ben23 Jul 2020 19:02 UTC

10 points

1 comment3 min readEA link

AGI safety career advice

richard_ngo2 May 2023 7:36 UTC

211 points

20 comments1 min readEA link

AMA: Ajeya Cotra, researcher at Open Phil

Ajeya28 Jan 2021 17:38 UTC

84 points

105 comments1 min readEA link

[Question] What are the coolest topics in AI safety, to a hopelessly pure mathematician?

Jenny K E7 May 2022 7:18 UTC

89 points

29 comments1 min readEA link

Introducing The Nonlinear Fund: AI Safety research, incubation, and funding

Kat Woods18 Mar 2021 14:07 UTC

71 points

32 comments5 min readEA link

What is it to solve the alignment problem?

Joe_Carlsmith24 Aug 2024 21:19 UTC

32 points

1 comment1 min readEA link

AGI safety from first principles

richard_ngo21 Oct 2020 17:42 UTC

77 points

10 comments3 min readEA link

(www.alignmentforum.org)

Paul Christiano: Current work in AI alignment

EA Global3 Apr 2020 7:06 UTC

80 points

3 comments24 min readEA link

(www.youtube.com)

Connor Leahy on Conjecture and Dying with Dignity

Michaël Trazzi22 Jul 2022 19:30 UTC

34 points

0 comments10 min readEA link

(theinsideview.ai)

Deep Deceptiveness

So8res21 Mar 2023 2:51 UTC

40 points

1 comment1 min readEA link

My Understanding of Paul Christiano’s Iterated Amplification AI Safety Research Agenda

Chi15 Aug 2020 19:59 UTC

38 points

3 comments39 min readEA link

My Objections to “We’re All Gonna Die with Eliezer Yudkowsky”

Quintin Pope21 Mar 2023 1:23 UTC

166 points

21 comments39 min readEA link

2020 AI Alignment Literature Review and Charity Comparison

Larks21 Dec 2020 15:25 UTC

155 points

16 comments68 min readEA link

Introduction to Pragmatic AI Safety [Pragmatic AI Safety #1]

TW1239 May 2022 17:02 UTC

68 points

0 comments6 min readEA link

Apply to the second ML for Alignment Bootcamp (MLAB 2) in Berkeley [Aug 15 - Fri Sept 2]

Buck6 May 2022 0:19 UTC

111 points

7 comments6 min readEA link

There should be an AI safety project board

mariushobbhahn14 Mar 2022 16:08 UTC

24 points

3 comments1 min readEA link

Chaining the evil genie: why “outer” AI safety is probably easy

titotal30 Aug 2022 13:55 UTC

40 points

12 comments10 min readEA link

Why Would AI “Aim” To Defeat Humanity?

Holden Karnofsky29 Nov 2022 18:59 UTC

24 points

0 comments32 min readEA link

(www.cold-takes.com)

How might we align transformative AI if it’s developed very soon?

Holden Karnofsky29 Aug 2022 15:48 UTC

163 points

17 comments44 min readEA link

High-level hopes for AI alignment

Holden Karnofsky20 Dec 2022 2:11 UTC

123 points

14 comments19 min readEA link

(www.cold-takes.com)

Parallels Between AI Safety by Debate and Evidence Law

Cullen 🔸20 Jul 2020 22:52 UTC

30 points

2 comments2 min readEA link

(cullenokeefe.com)

[Question] How strong is the evidence of unaligned AI systems causing harm?

Eevee🔹21 Jul 2020 4:08 UTC

31 points

1 comment1 min readEA link

“The Race to the End of Humanity” – Structural Uncertainty Analysis in AI Risk Models

Froolow19 May 2023 12:03 UTC

48 points

4 comments21 min readEA link

The current alignment plan, and how we might improve it | EAG Bay Area 23

Buck7 Jun 2023 21:03 UTC

66 points

0 comments33 min readEA link

Announcing AXRP, the AI X-risk Research Podcast

DanielFilan23 Dec 2020 20:10 UTC

32 points

1 comment1 min readEA link

AGI misalignment x-risk may be lower due to an overlooked goal specification technology

johnjnay21 Oct 2022 2:03 UTC

20 points

1 comment1 min readEA link

AI alignment with humans… but with which humans?

Geoffrey Miller8 Sep 2022 23:43 UTC

51 points

20 comments3 min readEA link

Possible OpenAI’s Q* breakthrough and DeepMind’s AlphaGo-type systems plus LLMs

Burnydelic23 Nov 2023 7:02 UTC

13 points

4 comments2 min readEA link

Interpreting Neural Networks through the Polytope Lens

Sid Black23 Sep 2022 18:03 UTC

35 points

0 comments1 min readEA link

Alignment 201 curriculum

richard_ngo12 Oct 2022 19:17 UTC

94 points

9 comments1 min readEA link

On how various plans miss the hard bits of the alignment challenge

So8res12 Jul 2022 5:35 UTC

125 points

13 comments29 min readEA link

Rohin Shah: What’s been happening in AI alignment?

EA Global29 Jul 2020 20:15 UTC

18 points

0 comments14 min readEA link

(www.youtube.com)

On Deference and Yudkowsky’s AI Risk Estimates

bgarfinkel19 Jun 2022 14:35 UTC

285 points

194 comments17 min readEA link

[Question] How much EA analysis of AI safety as a cause area exists?

richard_ngo6 Sep 2019 11:15 UTC

94 points

20 comments2 min readEA link

We Are Conjecture, A New Alignment Research Startup

Connor Leahy9 Apr 2022 15:07 UTC

31 points

0 comments1 min readEA link

Conjecture: Internal Infohazard Policy

Connor Leahy29 Jul 2022 19:35 UTC

34 points

3 comments19 min readEA link

Apply to the ML for Alignment Bootcamp (MLAB) in Berkeley [Jan 3 - Jan 22]

Habryka3 Nov 2021 18:20 UTC

140 points

6 comments1 min readEA link

Why the Orthogonality Thesis’s veracity is not the point:

Antoine de Scorraille23 Jul 2020 15:40 UTC

3 points

0 comments3 min readEA link

[Link] How understanding valence could help make future AIs safer

Milan_Griffes8 Oct 2020 18:53 UTC

22 points

2 comments3 min readEA link

(Even) More Early-Career EAs Should Try AI Safety Technical Research

tlevin30 Jun 2022 21:14 UTC

86 points

40 comments11 min readEA link

Relevant pre-AGI possibilities

kokotajlod20 Jun 2020 13:15 UTC

22 points

0 comments1 min readEA link

(aiimpacts.org)

Buck Shlegeris: How I think students should orient to AI safety

EA Global25 Oct 2020 5:48 UTC

11 points

0 comments1 min readEA link

(www.youtube.com)

Intellectual Diversity in AI Safety

KR22 Jul 2020 19:07 UTC

21 points

8 comments3 min readEA link

AI Risk: Increasing Persuasion Power

kewlcats3 Aug 2020 20:25 UTC

4 points

0 comments1 min readEA link

EA, Psychology & AI Safety Research

Sam Ellis26 May 2022 23:46 UTC

28 points

3 comments6 min readEA link

Safe AI and moral AI

William D'Alessandro1 Jun 2023 21:18 UTC

3 points

0 comments11 min readEA link

New report on how much computational power it takes to match the human brain (Open Philanthropy)

Aaron Gertler 🔸15 Sep 2020 1:06 UTC

45 points

1 comment18 min readEA link

(www.openphilanthropy.org)

Speedrun: AI Alignment Prizes

joe9 Feb 2023 11:55 UTC

27 points

0 comments18 min readEA link

Aligning the Aligners: Ensuring Aligned AI acts for the common good of all mankind

timunderwood16 Jan 2023 11:13 UTC

40 points

2 comments4 min readEA link

[linkpost] “What Are Reasonable AI Fears?” by Robin Hanson, 2023-04-23

Arjun Panickssery14 Apr 2023 23:26 UTC

41 points

3 comments4 min readEA link

(quillette.com)

The basic reasons I expect AGI ruin

RobBensinger18 Apr 2023 3:37 UTC

58 points

13 comments1 min readEA link

What Should the Average EA Do About AI Alignment?

Raemon25 Feb 2017 20:07 UTC

42 points

39 comments7 min readEA link

Qualities that alignment mentors value in junior researchers

Akash14 Feb 2023 23:27 UTC

31 points

1 comment1 min readEA link

The Importance of AI Alignment, explained in 5 points

Daniel_Eth11 Feb 2023 2:56 UTC

50 points

4 comments13 min readEA link

Important, actionable research questions for the most important century

Holden Karnofsky24 Feb 2022 16:34 UTC

298 points

13 comments19 min readEA link

Order Matters for Deceptive Alignment

DavidW15 Feb 2023 20:12 UTC

20 points

1 comment1 min readEA link

(www.lesswrong.com)

Don’t Call It AI Alignment

Gil20 Feb 2023 5:27 UTC

16 points

7 comments2 min readEA link

Disagreements about Alignment: Why, and how, we should try to solve them

ojorgensen8 Aug 2022 22:32 UTC

16 points

6 comments16 min readEA link

Community Building for Graduate Students: A Targeted Approach

Neil Crawford29 Mar 2022 19:47 UTC

13 points

0 comments3 min readEA link

Opportunities for individual donors in AI safety

alexflint12 Mar 2018 2:10 UTC

13 points

11 comments10 min readEA link

Who Aligns the Alignment Researchers?

ben.smith5 Mar 2023 23:22 UTC

23 points

4 comments1 min readEA link

List #3: Why not to assume on prior that AGI-alignment workarounds are available

Remmelt24 Dec 2022 9:54 UTC

6 points

0 comments1 min readEA link

[Question] Alignment & Capabilities: What’s the difference?

John G. Halstead31 Aug 2023 22:13 UTC

50 points

10 comments1 min readEA link

List #1: Why stopping the development of AGI is hard but doable

Remmelt24 Dec 2022 9:52 UTC

24 points

2 comments1 min readEA link

De Dicto and De Se Reference Matters for Alignment

philgoetz3 Oct 2023 21:57 UTC

5 points

2 comments9 min readEA link

Mahendra Prasad: Rational group decision-making

EA Global8 Jul 2020 15:06 UTC

15 points

0 comments16 min readEA link

(www.youtube.com)

We should expect to worry more about speculative risks

bgarfinkel29 May 2022 21:08 UTC

120 points

14 comments3 min readEA link

What does it mean to become an expert in AI Hardware?

Toph9 Jan 2021 4:15 UTC

87 points

10 comments11 min readEA link

SERI ML Alignment Theory Scholars Program 2022

Ryan Kidd27 Apr 2022 16:33 UTC

57 points

2 comments3 min readEA link

Meditations on careers in AI Safety

PabloAMC 🔸23 Mar 2022 22:00 UTC

88 points

30 comments2 min readEA link

Conversation on AI risk with Adam Gleave

AI Impacts27 Dec 2019 21:43 UTC

18 points

3 comments4 min readEA link

(aiimpacts.org)

There are two factions working to prevent AI dangers. Here’s why they’re deeply divided.

Sharmake10 Aug 2022 19:52 UTC

10 points

0 comments4 min readEA link

(www.vox.com)

Ngo and Yudkowsky on AI capability gains

richard_ngo19 Nov 2021 1:54 UTC

23 points

4 comments39 min readEA link

Atari early

AI Impacts2 Apr 2020 23:28 UTC

34 points

2 comments5 min readEA link

(aiimpacts.org)

I’m Buck Shlegeris, I do research and outreach at MIRI, AMA

Buck15 Nov 2019 22:44 UTC

123 points

228 comments2 min readEA link

Truthful AI

Owen Cotton-Barratt20 Oct 2021 15:11 UTC

55 points

14 comments10 min readEA link

[Question] Why aren’t you freaking out about OpenAI? At what point would you start?

AppliedDivinityStudies10 Oct 2021 13:06 UTC

80 points

22 comments2 min readEA link

[Question] Is it crunch time yet? If so, who can help?

Nicholas / Heather Kross13 Oct 2021 4:11 UTC

29 points

9 comments1 min readEA link

[Question] What harm could AI safety do?

SeanEngelhart15 May 2021 1:11 UTC

12 points

7 comments1 min readEA link

Situational awareness (Section 2.1 of “Scheming AIs”)

Joe_Carlsmith26 Nov 2023 23:00 UTC

12 points

1 comment1 min readEA link

AI Safety: Applying to Graduate Studies

frances_lorenz15 Dec 2021 22:56 UTC

23 points

0 comments12 min readEA link

[Question] If AIs had subcortical brain simulation, would that solve the alignment problem?

Rainbow Affect31 Jul 2023 15:48 UTC

1 point

0 comments2 min readEA link

Symbiosis, not alignment, as the goal for liberal democracies in the transition to artificial general intelligence

simonfriederich17 Mar 2023 13:04 UTC

18 points

2 comments24 min readEA link

(rdcu.be)

Shallow review of live agendas in alignment & safety

Gavin27 Nov 2023 11:33 UTC

76 points

8 comments29 min readEA link

FLI AI Alignment podcast: Evan Hubinger on Inner Alignment, Outer Alignment, and Proposals for Building Safe Advanced AI

evhub1 Jul 2020 20:59 UTC

13 points

2 comments1 min readEA link

(futureoflife.org)

How to build a safe advanced AI (Evan Hubinger) | What’s up in AI safety? (Asya Bergal)

EA Global25 Oct 2020 5:48 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Long-Term Future Fund: April 2019 grant recommendations

Habryka23 Apr 2019 7:00 UTC

142 points

242 comments46 min readEA link

Sydney AI Safety Fellowship

Chris Leong2 Dec 2021 7:35 UTC

16 points

0 comments2 min readEA link

New blog: Planned Obsolescence

Ajeya27 Mar 2023 19:46 UTC

198 points

9 comments1 min readEA link

(www.planned-obsolescence.org)

AI alignment prize winners and next round [link]

RyanCarey20 Jan 2018 12:07 UTC

7 points

1 comment1 min readEA link

Announcing the Vitalik Buterin Fellowships in AI Existential Safety!

DanielFilan21 Sep 2021 0:41 UTC

62 points

0 comments1 min readEA link

(grants.futureoflife.org)

AI safety university groups: a promising opportunity to reduce existential risk

mic30 Jun 2022 18:37 UTC

53 points

1 comment11 min readEA link

A Simple Model of AGI Deployment Risk

djbinder9 Jul 2021 9:44 UTC

30 points

0 comments5 min readEA link

[Question] Is transformative AI the biggest existential risk? Why or why not?

Eevee🔹5 Mar 2022 3:54 UTC

9 points

10 comments1 min readEA link

Ngo and Yudkowsky on scientific reasoning and pivotal acts

EliezerYudkowsky21 Feb 2022 17:00 UTC

33 points

1 comment35 min readEA link

Jan Leike, Helen Toner, Malo Bourgon, and Miles Brundage: Working in AI

EA Global11 Aug 2017 8:19 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Law-Following AI 1: Sequence Introduction and Structure

Cullen 🔸27 Apr 2022 17:16 UTC

35 points

2 comments9 min readEA link

The Parable of the Boy Who Cried 5% Chance of Wolf

Kat Woods15 Aug 2022 14:22 UTC

77 points

8 comments2 min readEA link

Misgeneralization as a misnomer

So8res6 Apr 2023 20:43 UTC

48 points

0 comments1 min readEA link

Consider trying the ELK contest (I am)

Holden Karnofsky5 Jan 2022 19:42 UTC

110 points

17 comments16 min readEA link

[Question] What are the top priorities in a slow-takeoff, multipolar world?

JP Addison🔸25 Aug 2021 8:47 UTC

26 points

9 comments1 min readEA link

Thoughts on short timelines

Tobias_Baumann23 Oct 2018 15:59 UTC

22 points

14 comments5 min readEA link

Four questions I ask AI safety researchers

Akash17 Jul 2022 17:25 UTC

30 points

3 comments1 min readEA link

Ought: why it matters and ways to help

Paul_Christiano26 Jul 2019 1:56 UTC

52 points

5 comments5 min readEA link

[Linkpost] How To Get Into Independent Research On Alignment/Agency

Jackson Wagner14 Feb 2022 21:40 UTC

10 points

0 comments1 min readEA link

Law-Following AI 3: Lawless AI Agents Undermine Stabilizing Agreements

Cullen 🔸27 Apr 2022 17:20 UTC

28 points

3 comments3 min readEA link

Why AI is Harder Than We Think—Melanie Mitchell

Eevee🔹28 Apr 2021 8:19 UTC

45 points

7 comments2 min readEA link

(arxiv.org)

Announcing the Harvard AI Safety Team

Xander12330 Jun 2022 18:34 UTC

128 points

4 comments5 min readEA link

Brain-computer interfaces and brain organoids in AI alignment?

freedomandutility15 Apr 2023 22:28 UTC

8 points

2 comments1 min readEA link

[Question] What considerations influence whether I have more influence over short or long timelines?

kokotajlod5 Nov 2020 19:57 UTC

18 points

0 comments1 min readEA link

Nobody’s on the ball on AGI alignment

leopold29 Mar 2023 14:26 UTC

327 points

65 comments9 min readEA link

(www.forourposterity.com)

Racing through a minefield: the AI deployment problem

Holden Karnofsky31 Dec 2022 21:44 UTC

79 points

1 comment13 min readEA link

(www.cold-takes.com)

AI Safety field-building projects I’d like to see

Akash11 Sep 2022 23:45 UTC

31 points

4 comments6 min readEA link

(www.lesswrong.com)

[Question] What are the challenges and problems with programming law-breaking constraints into AGI?

MichaelStJules2 Feb 2020 20:53 UTC

20 points

34 comments1 min readEA link

How Do AI Timelines Affect Giving Now vs. Later?

MichaelDickens3 Aug 2021 3:36 UTC

36 points

8 comments8 min readEA link

Michael Page, Dario Amodei, Helen Toner, Tasha McCauley, Jan Leike, & Owen Cotton-Barratt: Musings on AI

EA Global11 Aug 2017 8:19 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

[AN #80]: Why AI risk might be solved without additional intervention from longtermists

Rohin Shah3 Jan 2020 7:52 UTC

58 points

12 comments10 min readEA link

(www.alignmentforum.org)

Long-Term Future Fund: May 2021 grant recommendations

abergal27 May 2021 6:44 UTC

110 points

17 comments57 min readEA link

Consider paying me to do AI safety research work

Rupert5 Nov 2020 8:09 UTC

11 points

3 comments2 min readEA link

Projects I would like to see (possibly at AI Safety Camp)

Linda Linsefors27 Sep 2023 21:27 UTC

9 points

0 comments1 min readEA link

Two reasons we might be closer to solving alignment than it seems

Kat Woods24 Sep 2022 17:38 UTC

44 points

17 comments4 min readEA link

Information security careers for GCR reduction

ClaireZabel20 Jun 2019 23:56 UTC

187 points

35 comments8 min readEA link

[Question] I’m interviewing Max Tegmark about AI safety and more. What shouId I ask him?

Robert_Wiblin13 May 2022 15:32 UTC

18 points

2 comments1 min readEA link

Are alignment researchers devoting enough time to improving their research capacity?

Carson Jones4 Nov 2022 0:58 UTC

11 points

1 comment1 min readEA link

On presenting the case for AI risk

Aryeh Englander8 Mar 2022 21:37 UTC

114 points

12 comments4 min readEA link

AGI Predictions

Pablo21 Nov 2020 12:02 UTC

36 points

0 comments1 min readEA link

(www.lesswrong.com)

Getting started independently in AI Safety

JJ Hepburn6 Jul 2021 15:20 UTC

41 points

10 comments2 min readEA link

Language Agents Reduce the Risk of Existential Catastrophe

cdkg29 May 2023 9:59 UTC

29 points

6 comments26 min readEA link

Sharing the World with Digital Minds

Aaron Gertler 🔸1 Dec 2020 8:00 UTC

12 points

1 comment1 min readEA link

(www.nickbostrom.com)

How to pursue a career in technical AI alignment

Charlie Rogers-Smith4 Jun 2022 21:36 UTC

265 points

9 comments39 min readEA link

Intent alignment should not be the goal for AGI x-risk reduction

johnjnay26 Oct 2022 1:24 UTC

7 points

1 comment1 min readEA link

Jesse Clifton: Open-source learning — a bargaining approach

EA Global18 Oct 2019 18:05 UTC

10 points

0 comments1 min readEA link

(www.youtube.com)

Tan Zhi Xuan: AI alignment, philosophical pluralism, and the relevance of non-Western philosophy

EA Global21 Nov 2020 8:12 UTC

19 points

1 comment1 min readEA link

(www.youtube.com)

Informatica: Special Issue on Superintelligence

RyanCarey3 May 2017 5:05 UTC

7 points

0 comments2 min readEA link

[Question] Are we confident that superintelligent artificial intelligence disempowering humans would be bad?

Vasco Grilo🔸10 Jun 2023 9:24 UTC

24 points

27 comments1 min readEA link

[Question] Brief summary of key disagreements in AI Risk

Aryeh Englander26 Dec 2019 19:40 UTC

31 points

3 comments1 min readEA link

New report: “Scheming AIs: Will AIs fake alignment during training in order to get power?”

Joe_Carlsmith15 Nov 2023 17:16 UTC

71 points

4 comments1 min readEA link

Survey on AI existential risk scenarios

Sam Clarke8 Jun 2021 17:12 UTC

154 points

11 comments6 min readEA link

Katja Grace: AI safety

EA Global11 Aug 2017 8:19 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Some global catastrophic risk estimates

Tamay10 Feb 2021 19:32 UTC

106 points

15 comments1 min readEA link

The Multidisciplinary Approach to Alignment (MATA) and Archetypal Transfer Learning (ATL)

Miguel19 Jun 2023 3:23 UTC

4 points

0 comments7 min readEA link

[Question] What is an example of recent, tangible progress in AI safety research?

Aaron Gertler 🔸14 Jun 2021 5:29 UTC

35 points

4 comments1 min readEA link

Key Papers in Language Model Safety

aogara20 Jun 2022 14:59 UTC

20 points

0 comments22 min readEA link

Christiano and Yudkowsky on AI predictions and human intelligence

EliezerYudkowsky23 Feb 2022 16:51 UTC

31 points

0 comments42 min readEA link

AGI in a vulnerable world

AI Impacts2 Apr 2020 3:43 UTC

17 points

0 comments1 min readEA link

(aiimpacts.org)

[linkpost] Sharing powerful AI models: the emerging paradigm of structured access

ts20 Jan 2022 21:10 UTC

11 points

3 comments1 min readEA link

What success looks like

mariushobbhahn28 Jun 2022 14:30 UTC

112 points

20 comments19 min readEA link

The case for more Alignment Target Analysis (ATA)

Chi20 Sep 2024 1:14 UTC

21 points

0 comments1 min readEA link

Quantifying the Far Future Effects of Interventions

MichaelDickens18 May 2016 2:15 UTC

8 points

0 comments11 min readEA link

Artificial intelligence career stories

EA Global25 Oct 2020 6:56 UTC

12 points

0 comments1 min readEA link

(www.youtube.com)

AI things that are perhaps as important as human-controlled AI

Chi3 Mar 2024 18:07 UTC

113 points

9 comments21 min readEA link

[Question] Career Advice: Philosophy + Programming → AI Safety

tcelferact18 Mar 2022 15:09 UTC

30 points

11 comments2 min readEA link

Coherence arguments imply a force for goal-directed behavior

Katja_Grace6 Apr 2021 21:44 UTC

19 points

1 comment11 min readEA link

(worldspiritsockpuppet.com)

Soares, Tallinn, and Yudkowsky discuss AGI cognition

EliezerYudkowsky29 Nov 2021 17:28 UTC

15 points

0 comments40 min readEA link

Guardrails vs Goal-directedness in AI Alignment

freedomandutility30 Dec 2023 12:58 UTC

13 points

2 comments1 min readEA link

Some AI Governance Research Ideas

MarkusAnderljung3 Jun 2021 10:51 UTC

101 points

5 comments2 min readEA link

Timelines are short, p(doom) is high: a global stop to frontier AI development until x-safety consensus is our only reasonable hope

Greg_Colbourn12 Oct 2023 11:24 UTC

73 points

85 comments9 min readEA link

Defining alignment research

richard_ngo19 Aug 2024 22:49 UTC

48 points

1 comment1 min readEA link

Gentleness and the artificial Other

Joe_Carlsmith2 Jan 2024 18:21 UTC

89 points

2 comments1 min readEA link

Imitation Learning is Probably Existentially Safe

Vasco Grilo🔸30 Apr 2024 17:06 UTC

19 points

7 comments3 min readEA link

(www.openphilanthropy.org)

Otherness and control in the age of AGI

Joe_Carlsmith2 Jan 2024 18:15 UTC

37 points

1 comment1 min readEA link

When “yang” goes wrong

Joe_Carlsmith8 Jan 2024 16:35 UTC

56 points

1 comment1 min readEA link

Does AI risk “other” the AIs?

Joe_Carlsmith9 Jan 2024 17:51 UTC

22 points

3 comments1 min readEA link

An even deeper atheism

Joe_Carlsmith11 Jan 2024 17:28 UTC

25 points

2 comments1 min readEA link

[Question] What would you do if you had a lot of money/power/influence and you thought that AI timelines were very short?

Greg_Colbourn12 Nov 2021 21:59 UTC

29 points

8 comments1 min readEA link

Bryan Johnson seems more EA aligned than I expected

PeterSlattery22 Apr 2024 9:38 UTC

13 points

27 comments2 min readEA link

(www.youtube.com)

Aspiration-based, non-maximizing AI agent designs

Bob Jacobs 🔸7 May 2024 16:13 UTC

12 points

1 comment38 min readEA link

[Cause Exploration Prizes] Expanding communication about AGI risks

Ines22 Sep 2022 5:30 UTC

13 points

0 comments11 min readEA link

Introducing the Principles of Intelligent Behaviour in Biological and Social Systems (PIBBSS) Fellowship

adamShimi18 Dec 2021 15:25 UTC

37 points

5 comments10 min readEA link

AGI risk: analogies & arguments

Gavin23 Mar 2021 13:18 UTC

31 points

3 comments8 min readEA link

(www.gleech.org)

The Problem With the Word ‘Alignment’

Peli Grietzer21 May 2024 21:37 UTC

13 points

1 comment6 min readEA link

A mesa-optimization perspective on AI valence and moral patienthood

jacobpfau9 Sep 2021 22:23 UTC

10 points

18 comments17 min readEA link

Potential Risks from Advanced AI

EA Global13 Aug 2017 7:00 UTC

9 points

0 comments18 min readEA link

On the abolition of man

Joe_Carlsmith18 Jan 2024 18:17 UTC

71 points

4 comments1 min readEA link

EA megaprojects continued

mariushobbhahn3 Dec 2021 10:33 UTC

183 points

48 comments7 min readEA link

What does it mean for an AGI to be ‘safe’?

So8res7 Oct 2022 4:43 UTC

53 points

21 comments1 min readEA link

Twitter-length responses to 24 AI alignment arguments

RobBensinger14 Mar 2022 19:34 UTC

67 points

17 comments8 min readEA link

What does (and doesn’t) AI mean for effective altruism?

EA Global12 Aug 2017 7:00 UTC

9 points

0 comments12 min readEA link

Alignment Newsletter One Year Retrospective

Rohin Shah10 Apr 2019 7:00 UTC

62 points

22 comments21 min readEA link

AI safety tax dynamics

Owen Cotton-Barratt23 Oct 2024 12:21 UTC

21 points

9 comments6 min readEA link

(strangecities.substack.com)

[Question] Should the EA community have a DL engineering fellowship?

PabloAMC 🔸24 Dec 2021 13:43 UTC

26 points

6 comments1 min readEA link

[Question] Is this a good way to bet on short timelines?

kokotajlod28 Nov 2020 14:31 UTC

17 points

16 comments1 min readEA link

Public-facing Censorship Is Safety Theater, Causing Reputational Damage

Yitz23 Sep 2022 5:08 UTC

49 points

7 comments1 min readEA link

Open Philanthropy’s AI governance grantmaking (so far)

Aaron Gertler 🔸17 Dec 2020 12:00 UTC

63 points

0 comments6 min readEA link

(www.openphilanthropy.org)

Three kinds of competitiveness

AI Impacts2 Apr 2020 3:46 UTC

10 points

0 comments5 min readEA link

(aiimpacts.org)

Helen Toner: The Open Philanthropy Project’s work on AI risk

EA Global3 Nov 2017 7:43 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Paul Christiano on how OpenAI is developing real solutions to the ‘AI alignment problem’, and his vision of how humanity will progressively hand over decision-making to AI systems

80000_Hours2 Oct 2018 11:49 UTC

6 points

0 comments185 min readEA link

Cosmic AI safety

Magnus Vinding6 Dec 2024 22:32 UTC

22 points

3 comments6 min readEA link

Lessons learned from talking to >100 academics about AI safety

mariushobbhahn10 Oct 2022 13:16 UTC

138 points

21 comments1 min readEA link

[Question] What kind of event, targeted to undergraduate CS majors, would be most effective at getting people to work on AI safety?

CBiddulph19 Sep 2021 16:19 UTC

9 points

1 comment1 min readEA link

I’m Cullen O’Keefe, a Policy Researcher at OpenAI, AMA

Cullen 🔸11 Jan 2020 4:13 UTC

45 points

68 comments1 min readEA link

[Creative Writing Contest] An AI Safety Limerick

Ben_West🔸18 Oct 2021 19:11 UTC

21 points

5 comments1 min readEA link

A list of good heuristics that the case for AI X-risk fails

Aaron Gertler 🔸16 Jul 2020 9:56 UTC

25 points

9 comments2 min readEA link

(www.alignmentforum.org)

Owen Cotton-Barratt: What does (and doesn’t) AI mean for effective altruism?

EA Global11 Aug 2017 8:19 UTC

10 points

0 comments12 min readEA link

(www.youtube.com)

Is GPT-3 the death of the paperclip maximizer?

matthias_samwald3 Aug 2020 11:34 UTC

4 points

1 comment1 min readEA link

The Metaethics and Normative Ethics of AGI Value Alignment: Many Questions, Some Implications

Eleos Arete Citrini15 Sep 2021 19:05 UTC

25 points

0 comments8 min readEA link

[Question] The positive case for a focus on achieving safe AI?

vipulnaik25 Jun 2021 4:01 UTC

41 points

1 comment1 min readEA link

Action: Help expand funding for AI Safety by coordinating on NSF response

Evan R. Murphy20 Jan 2022 20:48 UTC

20 points

7 comments3 min readEA link

[Link] EAF Research agenda: “Cooperation, Conflict, and Transformative Artificial Intelligence”

stefan.torges17 Jan 2020 13:28 UTC

64 points

0 comments1 min readEA link

Shah and Yudkowsky on alignment failures

EliezerYudkowsky28 Feb 2022 19:25 UTC

38 points

7 comments92 min readEA link

[Question] How can I bet on short timelines?

kokotajlod7 Nov 2020 12:45 UTC

33 points

12 comments2 min readEA link

AI Impacts: Historic trends in technological progress

Aaron Gertler 🔸12 Feb 2020 0:08 UTC

55 points

5 comments3 min readEA link

AGI x-risk timelines: 10% chance (by year X) estimates should be the headline, not 50%.

Greg_Colbourn1 Mar 2022 12:02 UTC

69 points

22 comments2 min readEA link

Personal thoughts on careers in AI policy and strategy

carrickflynn27 Sep 2017 16:52 UTC

56 points

28 comments18 min readEA link

Discontinuous progress in history: an update

AI Impacts17 Apr 2020 16:28 UTC

69 points

3 comments24 min readEA link

[Question] Why should we not put effort into AI safety research?

Ben Thompson16 May 2021 5:11 UTC

15 points

5 comments1 min readEA link

[Question] Is there evidence that recommender systems are changing users’ preferences?

zdgroff12 Apr 2021 19:11 UTC

60 points

15 comments1 min readEA link

An overview of some promising work by junior alignment researchers

Akash26 Dec 2022 17:23 UTC

10 points

0 comments1 min readEA link

Max Tegmark: Risks and benefits of advanced artificial intelligence

EA Global5 Aug 2016 9:19 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Andrew Critch: Logical induction — progress in AI alignment

EA Global6 Aug 2016 0:40 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

Is AI forecasting a waste of effort on the margin?

Emrik5 Nov 2022 0:41 UTC

10 points

6 comments3 min readEA link

Christiano, Cotra, and Yudkowsky on AI progress

Ajeya25 Nov 2021 16:30 UTC

18 points

6 comments68 min readEA link

[Question] What are your recommendations for technical AI alignment podcasts?

Evan_Gaensbauer11 May 2022 21:52 UTC

13 points

4 comments1 min readEA link

Mauhn Releases AI Safety Documentation

Berg Severens2 Jul 2021 12:19 UTC

4 points

2 comments1 min readEA link

On AI and Compute

johncrox3 Apr 2019 21:26 UTC

39 points

12 comments8 min readEA link

Eric Drexler: Paretotopian goal alignment

EA Global15 Mar 2019 14:51 UTC

14 points

0 comments10 min readEA link

(www.youtube.com)

[Question] How should we invest in “long-term short-termism” given the likelihood of transformative AI?

James_Banks12 Jan 2021 23:54 UTC

8 points

0 comments1 min readEA link

Quick survey on AI alignment resources

frances_lorenz30 Jun 2022 19:08 UTC

15 points

0 comments1 min readEA link

Law-Following AI 2: Intent Alignment + Superintelligence → Lawless AI (By Default)

Cullen 🔸27 Apr 2022 17:18 UTC

19 points

0 comments6 min readEA link

AI Safety Needs Great Engineers

Andy Jones23 Nov 2021 21:03 UTC

98 points

13 comments4 min readEA link

Three Impacts of Machine Intelligence

Paul_Christiano23 Aug 2013 10:10 UTC

33 points

5 comments8 min readEA link

(rationalaltruist.com)

Critical Review of ‘The Precipice’: A Reassessment of the Risks of AI and Pandemics

Fods1211 May 2020 11:11 UTC

111 points

32 comments26 min readEA link

An ML safety insurance company—shower thoughts

EdoArad18 Oct 2021 7:45 UTC

15 points

4 comments1 min readEA link

Yudkowsky and Christiano discuss “Takeoff Speeds”

EliezerYudkowsky22 Nov 2021 19:42 UTC

42 points

0 comments60 min readEA link

On the correspondence between AI-misalignment and cognitive dissonance using a behavioral economics model

Stijn1 Nov 2022 9:15 UTC

11 points

0 comments6 min readEA link

Pile of Law and Law-Following AI

Cullen 🔸13 Jul 2022 0:29 UTC

28 points

2 comments3 min readEA link

How I Formed My Own Views About AI Safety

Neel Nanda27 Feb 2022 18:52 UTC

134 points

12 comments14 min readEA link

(www.neelnanda.io)

13 background claims about EA

Akash7 Sep 2022 3:54 UTC

70 points

16 comments3 min readEA link

How to get technological knowledge on AI/ML (for non-tech people)

FangFang30 Jun 2021 7:53 UTC

62 points

7 comments5 min readEA link

“Slower tech development” can be about ordering, gradualness, or distance from now

MichaelA🔸14 Nov 2021 20:58 UTC

47 points

3 comments4 min readEA link

Emergent Ventures AI

Gavin8 Apr 2022 22:08 UTC

22 points

0 comments1 min readEA link

(marginalrevolution.com)

Final Report of the National Security Commission on Artificial Intelligence (NSCAI, 2021)

MichaelA🔸1 Jun 2021 8:19 UTC

51 points

3 comments4 min readEA link

(www.nscai.gov)

Naturalism and AI alignment

Michele Campolo24 Apr 2021 16:20 UTC

17 points

3 comments7 min readEA link

$500 bounty for alignment contest ideas

Akash30 Jun 2022 1:55 UTC

18 points

1 comment2 min readEA link

Takeaways from safety by default interviews

AI Impacts7 Apr 2020 2:01 UTC

25 points

2 comments13 min readEA link

(aiimpacts.org)

AMA or discuss my 80K podcast episode: Ben Garfinkel, FHI researcher

bgarfinkel13 Jul 2020 16:17 UTC

87 points

140 comments1 min readEA link

Predict responses to the “existential risk from AI” survey

RobBensinger28 May 2021 1:38 UTC

36 points

8 comments2 min readEA link

Increased Availability and Willingness for Deployment of Resources for Effective Altruism and Long-Termism

Evan_Gaensbauer29 Dec 2021 20:20 UTC

46 points

1 comment2 min readEA link

“Existential risk from AI” survey results

RobBensinger1 Jun 2021 20:19 UTC

80 points

35 comments11 min readEA link

BERI is hiring an ML Software Engineer

sawyer🔸10 Nov 2021 19:36 UTC

17 points

2 comments1 min readEA link

Steering AI to care for animals, and soon

Andrew Critch14 Jun 2022 1:13 UTC

224 points

37 comments1 min readEA link

Owain Evans and Victoria Krakovna: Careers in technical AI safety

EA Global3 Nov 2017 7:43 UTC

7 points

0 comments1 min readEA link

(www.youtube.com)

12 career advising questions that may (or may not) be helpful for people interested in alignment research

Akash12 Dec 2022 22:36 UTC

14 points

0 comments1 min readEA link

7 essays on Building a Better Future

Jamie_Harris24 Jun 2022 14:28 UTC

21 points

0 comments2 min readEA link

CFP for Rebellion and Disobedience in AI workshop

Ram Rachum29 Dec 2022 16:09 UTC

4 points

0 comments1 min readEA link

[Question] Is a career in making AI systems more secure a meaningful way to mitigate the X-risk posed by AGI?

Kyle O’Brien13 Feb 2022 7:05 UTC

14 points

4 comments1 min readEA link

Ways to buy time

Akash12 Nov 2022 19:31 UTC

47 points

1 comment1 min readEA link

AI views and disagreements AMA: Christiano, Ngo, Shah, Soares, Yudkowsky

RobBensinger1 Mar 2022 1:13 UTC

30 points

4 comments1 min readEA link

(www.lesswrong.com)

AI alignment research links

Holden Karnofsky6 Jan 2022 5:52 UTC

16 points

0 comments6 min readEA link

(www.cold-takes.com)

Some promising career ideas beyond 80,000 Hours’ priority paths

Arden Koehler26 Jun 2020 10:34 UTC

142 points

28 comments15 min readEA link

Large Language Models as Corporate Lobbyists, and Implications for Societal-AI Alignment

johnjnay4 Jan 2023 22:22 UTC

10 points

6 comments8 min readEA link

[Linkpost] Jan Leike on three kinds of alignment taxes

Akash6 Jan 2023 23:57 UTC

29 points

0 comments1 min readEA link

Is this community over-emphasizing AI alignment?

Lixiang8 Jan 2023 6:23 UTC

1 point

5 comments1 min readEA link

Levelling Up in AI Safety Research Engineering

GabeM2 Sep 2022 4:59 UTC

165 points

21 comments17 min readEA link

Technical AGI safety research outside AI

richard_ngo18 Oct 2019 15:02 UTC

91 points

5 comments3 min readEA link

Why I expect successful (narrow) alignment

Tobias_Baumann29 Dec 2018 15:46 UTC

18 points

10 comments1 min readEA link

(s-risks.org)

Redwood Research is hiring for several roles

Jack R29 Nov 2021 0:18 UTC

75 points

0 comments1 min readEA link

VIRTUA: a novel about AI alignment

Karl von Wendt12 Jan 2023 9:37 UTC

23 points

0 comments1 min readEA link

Victoria Krakovna on AGI Ruin, The Sharp Left Turn and Paradigms of AI Alignment

Michaël Trazzi12 Jan 2023 17:09 UTC

16 points

0 comments1 min readEA link

Daniel Dewey: The Open Philanthropy Project’s work on potential risks from advanced AI

EA Global11 Aug 2017 8:19 UTC

7 points

0 comments18 min readEA link

(www.youtube.com)

SERI ML application deadline is extended until May 22.

Viktoria Malyasova22 May 2022 0:13 UTC

13 points

3 comments1 min readEA link

Collin Burns on Alignment Research And Discovering Latent Knowledge Without Supervision

Michaël Trazzi17 Jan 2023 17:21 UTC

21 points

3 comments1 min readEA link

13 Very Different Stances on AGI

Ozzie Gooen27 Dec 2021 23:30 UTC

84 points

23 comments3 min readEA link

The case for becoming a black-box investigator of language models

Buck6 May 2022 14:37 UTC

90 points

7 comments3 min readEA link

Discussion with Eliezer Yudkowsky on AGI interventions

RobBensinger11 Nov 2021 3:21 UTC

60 points

33 comments34 min readEA link

Compendium of problems with RLHF

Raphaël S30 Jan 2023 8:48 UTC

18 points

0 comments1 min readEA link

Alignment is mostly about making cognition aimable at all

So8res30 Jan 2023 15:22 UTC

57 points

3 comments1 min readEA link

Eli Lifland on Navigating the AI Alignment Landscape

Ozzie Gooen1 Feb 2023 0:07 UTC

48 points

9 comments31 min readEA link

(quri.substack.com)

Messy personal stuff that affected my cause prioritization (or: how I started to care about AI safety)

Julia_Wise🔸5 May 2022 17:59 UTC

265 points

14 comments2 min readEA link

Draft report on AI timelines

Ajeya15 Dec 2020 12:10 UTC

35 points

0 comments1 min readEA link

(alignmentforum.org)

Some AI research areas and their relevance to existential safety

Andrew Critch15 Dec 2020 12:15 UTC

12 points

1 comment56 min readEA link

(alignmentforum.org)

Aligning Recommender Systems as Cause Area

IvanVendrov8 May 2019 8:56 UTC

150 points

48 comments13 min readEA link

List #2: Why coordinating to align as humans to not develop AGI is a lot easier than, well… coordinating as humans with AGI coordinating to be aligned with humans

Remmelt24 Dec 2022 9:53 UTC

3 points

0 comments1 min readEA link

Forecasting Transformative AI: What Kind of AI?

Holden Karnofsky10 Aug 2021 21:38 UTC

62 points

3 comments10 min readEA link

[Crosspost] AI Regulation May Be More Important Than AI Alignment For Existential Safety

Otto24 Aug 2023 16:01 UTC

14 points

2 comments5 min readEA link

Effective Altruism Florida’s AI Expert Panel—Recording and Slides Available

Sam_E_2419 May 2023 19:15 UTC

2 points

0 comments1 min readEA link

Empirical work that might shed light on scheming (Section 6 of “Scheming AIs”)

Joe_Carlsmith11 Dec 2023 16:30 UTC

7 points

1 comment1 min readEA link

Animal Rights, The Singularity, and Astronomical Suffering

sapphire20 Aug 2020 20:23 UTC

51 points

0 comments3 min readEA link

Summary: Existential risk from power-seeking AI by Joseph Carlsmith

rileyharris28 Oct 2023 15:05 UTC

11 points

0 comments6 min readEA link

(www.millionyearview.com)

Status Quo Engines—AI essay

Ilana_Goldowitz_Jimenez28 May 2023 14:33 UTC

1 point

0 comments15 min readEA link

Advice for new alignment people: Info Max

Jonas Hallgren30 May 2023 15:42 UTC

10 points

0 comments1 min readEA link

Abstraction is Bigger than Natural Abstraction

Nicholas / Heather Kross31 May 2023 0:00 UTC

2 points

0 comments1 min readEA link

Intrinsic limitations of GPT-4 and other large language models, and why I’m not (very) worried about GPT-n

Fods123 Jun 2023 13:09 UTC

28 points

3 comments11 min readEA link

Decomposing alignment to take advantage of paradigms

Christopher King4 Jun 2023 14:26 UTC

2 points

0 comments4 min readEA link

From voluntary to mandatory, are the ESG disclosure frameworks still fertile ground for unrealised EA career pathways? – A 2023 update on ESG potential impact

Christopher Chan4 Jun 2023 12:00 UTC

21 points

5 comments11 min readEA link

Engaging with AI in a Personal Way

Spyder Rex4 Dec 2023 9:23 UTC

−9 points

0 comments1 min readEA link

Summing up “Scheming AIs” (Section 5)

Joe_Carlsmith9 Dec 2023 15:48 UTC

9 points

1 comment1 min readEA link

Speed arguments against scheming (Section 4.4-4.7 of “Scheming AIs”)

Joe_Carlsmith8 Dec 2023 21:10 UTC

6 points

0 comments1 min readEA link

There is only one goal or drive—only self-perpetuation counts

freest one13 Jun 2023 1:37 UTC

2 points

4 comments8 min readEA link

Report: Artificial Intelligence Risk Management in Spain

JorgeTorresC15 Jun 2023 16:08 UTC

22 points

0 comments3 min readEA link

(riesgoscatastroficosglobales.com)

New reference standard on LLM Application security started by OWASP

QuantumForest19 Jun 2023 19:56 UTC

5 points

0 comments1 min readEA link

The counting argument for scheming (Sections 4.1 and 4.2 of “Scheming AIs”)

Joe_Carlsmith6 Dec 2023 19:28 UTC

9 points

1 comment1 min readEA link

Join the Virtual AI Safety Unconference (VAISU)!

Nguyên21 Jun 2023 4:46 UTC

23 points

0 comments1 min readEA link

(vaisu.ai)

Yip Fai Tse on animal welfare & AI safety and long termism

Karthik Palakodeti22 Jun 2023 12:48 UTC

47 points

0 comments1 min readEA link

[Question] What do we know about Mustafa Suleyman’s position on AI Safety?

Chris Leong13 Aug 2023 19:41 UTC

14 points

3 comments1 min readEA link

Summary of “The Precipice” (2 of 4): We are a danger to ourselves

rileyharris13 Aug 2023 23:53 UTC

5 points

0 comments8 min readEA link

(www.millionyearview.com)

“AI Alignment” is a Dangerously Overloaded Term

Roko15 Dec 2023 15:06 UTC

20 points

2 comments3 min readEA link

Discovering alignment windfalls reduces AI risk

James Brady28 Feb 2024 21:14 UTC

22 points

3 comments8 min readEA link

(blog.elicit.com)

Clarifying two uses of “alignment”

Matthew_Barnett10 Mar 2024 17:41 UTC

36 points

28 comments4 min readEA link

Metaculus Launches Future of AI Series, Based on Research Questions by Arb

christian13 Mar 2024 21:14 UTC

34 points

0 comments1 min readEA link

(www.metaculus.com)

Against Explosive Growth

c.trout4 Sep 2024 21:45 UTC

24 points

9 comments1 min readEA link

What if doing the most good = benevolent AI takeover and human extinction?

Jordan Arel22 Mar 2024 19:56 UTC

2 points

4 comments3 min readEA link

Video and transcript of presentation on Scheming AIs

Joe_Carlsmith22 Mar 2024 15:56 UTC

23 points

1 comment1 min readEA link

SociaLLM: proposal for a language model design for personalised apps, social science, and AI safety research

Roman Leventov2 Jan 2024 8:11 UTC

4 points

2 comments1 min readEA link

Have your say on the future of AI regulation: Deadline approaching for your feedback on UN High-Level Advisory Body on AI Interim Report ‘Governing AI for Humanity’

Deborah W.A. Foulkes29 Mar 2024 6:37 UTC

17 points

1 comment1 min readEA link

Risk Alignment in Agentic AI Systems

Hayley Clatterbuck1 Oct 2024 22:51 UTC

31 points

1 comment3 min readEA link

(static1.squarespace.com)

Worrisome misunderstanding of the core issues with AI transition

Roman Leventov18 Jan 2024 10:05 UTC

4 points

3 comments1 min readEA link

Is RLHF cruel to AI?

Hzn16 Dec 2024 14:01 UTC

−2 points

2 comments3 min readEA link

Designing Artificial Wisdom: Decision Forecasting AI & Futarchy

Jordan Arel14 Jul 2024 5:10 UTC

5 points

1 comment6 min readEA link

“If we go extinct due to misaligned AI, at least nature will continue, right? … right?”

plex18 May 2024 15:06 UTC

13 points

10 comments1 min readEA link

(aisafety.info)

The necessity of “Guardian AI” and two conditions for its achievement

Proica28 May 2024 11:42 UTC

1 point

1 comment15 min readEA link

Geoffrey Hinton on the Past, Present, and Future of AI

Stephen McAleese12 Oct 2024 16:41 UTC

5 points

1 comment1 min readEA link

There Should Be More Alignment-Driven Startups

vaniver31 May 2024 2:05 UTC

27 points

3 comments1 min readEA link

5 ways to improve CoT faithfulness

CBiddulph8 Oct 2024 4:17 UTC

8 points

0 comments1 min readEA link

AI data gaps could lead to ongoing Animal Suffering

Darkness8i817 Oct 2024 10:52 UTC

7 points

3 comments5 min readEA link

A Rocket–Interpretability Analogy

plex21 Oct 2024 13:55 UTC

13 points

1 comment1 min readEA link

Miles Brundage resigned from OpenAI, and his AGI readiness team was disbanded

Garrison23 Oct 2024 23:42 UTC

57 points

4 comments7 min readEA link

(garrisonlovely.substack.com)

AI & wisdom 3: AI effects on amortised optimisation

L Rudolf L29 Oct 2024 13:37 UTC

14 points

0 comments1 min readEA link

(rudolf.website)

Motivation control

Joe_Carlsmith30 Oct 2024 17:15 UTC

18 points

0 comments1 min readEA link

The Compendium, A full argument about extinction risk from AGI

adamShimi31 Oct 2024 12:02 UTC

9 points

1 comment2 min readEA link

(www.thecompendium.ai)

Option control

Joe_Carlsmith4 Nov 2024 17:54 UTC

11 points

0 comments1 min readEA link

[Question] Can we train AI so that future philanthropy is more effective?

Ricardo Pimentel3 Nov 2024 15:08 UTC

3 points

0 comments1 min readEA link

The King and the Golem—The Animation

Writer8 Nov 2024 18:23 UTC

50 points

1 comment1 min readEA link

College technical AI safety hackathon retrospective—Georgia Tech

yixiong14 Nov 2024 13:34 UTC

18 points

0 comments5 min readEA link

(yixiong.substack.com)

Infinite Rewards, Finite Safety: New Models for AI Motivation Without Infinite Goals

Whylome Team12 Nov 2024 7:21 UTC

1 point

1 comment2 min readEA link

Incentive design and capability elicitation

Joe_Carlsmith12 Nov 2024 20:56 UTC

9 points

0 comments1 min readEA link

Linkpost: “Imagining and building wise machines: The centrality of AI metacognition” by Johnson, Karimi, Bengio, et al.

Chris Leong17 Nov 2024 15:00 UTC

8 points

0 comments1 min readEA link

(arxiv.org)

LLMs are weirder than you think

Derek Shiller20 Nov 2024 13:39 UTC

61 points

3 comments22 min readEA link

LLM chatbots have ~half of the kinds of “consciousness” that humans believe in. Humans should avoid going crazy about that.

Andrew Critch22 Nov 2024 3:26 UTC

11 points

2 comments1 min readEA link

The Animal Welfare Case for Open Access: Breaking Barriers to Scientific Knowledge and Enhancing LLM Training

Wladimir J. Alonso23 Nov 2024 13:07 UTC

32 points

2 comments3 min readEA link

Agentic Alignment: Navigating between Harm and Illegitimacy

LennardZ26 Nov 2024 21:27 UTC

2 points

1 comment9 min readEA link

[Question] Launching Applications for the Global AI Safety Fellowship 2025!

Impact Academy27 Nov 2024 15:33 UTC

9 points

1 comment1 min readEA link

Consider granting AIs freedom

Matthew_Barnett6 Dec 2024 0:55 UTC

75 points

21 comments5 min readEA link

OpenAI’s o1 tried to avoid being shut down, and lied about it, in evals

Greg_Colbourn6 Dec 2024 15:25 UTC

23 points

9 comments1 min readEA link

(www.transformernews.ai)

Biomimetic alignment: Alignment between animal genes and animal brains as a model for alignment between humans and AI systems.

Geoffrey Miller26 May 2023 21:25 UTC

32 points

1 comment16 min readEA link

[Question] Is it valuable to the field of AI Safety to have a neuroscience background?

Samuel Nellessen3 Apr 2022 19:44 UTC

18 points

3 comments1 min readEA link

Expected impact of a career in AI safety under different opinions

Jordan Taylor14 Jun 2022 14:25 UTC

42 points

16 comments11 min readEA link

Against GDP as a metric for timelines and takeoff speeds

kokotajlod29 Dec 2020 17:50 UTC

47 points

6 comments14 min readEA link

How could we know that an AGI system will have good consequences?

So8res7 Nov 2022 22:42 UTC

25 points

0 comments1 min readEA link

Long-Term Future Fund: Ask Us Anything!

AdamGleave3 Dec 2020 13:44 UTC

89 points

153 comments1 min readEA link

AI Forecasting Dictionary (Forecasting infrastructure, part 1)

jacobjacob8 Aug 2019 13:16 UTC

18 points

0 comments5 min readEA link

AI Benefits Post 1: Introducing “AI Benefits”

Cullen 🔸22 Jun 2020 16:58 UTC

10 points

2 comments3 min readEA link

Asya Bergal: Reasons you might think human-level AI is unlikely to happen soon

EA Global26 Aug 2020 16:01 UTC

24 points

2 comments17 min readEA link

(www.youtube.com)

Share your requests for ChatGPT

Kate Tran5 Dec 2022 18:43 UTC

8 points

5 comments1 min readEA link

Podcast: Krister Bykvist on moral uncertainty, rationality, metaethics, AI and future populations

Gus Docker21 Oct 2021 15:17 UTC

8 points

0 comments1 min readEA link

(www.utilitarianpodcast.com)

Good Futures Initiative: Winter Project Internship

a_e_r27 Nov 2022 23:27 UTC

67 points

7 comments3 min readEA link

[Question] Benefits/Risks of Scott Aaronson’s Orthodox/Reform Framing for AI Alignment

Jeremy21 Nov 2022 17:47 UTC

15 points

5 comments1 min readEA link

(scottaaronson.blog)

Introducing a New Course on the Economics of AI

akorinek21 Dec 2021 4:55 UTC

84 points

6 comments2 min readEA link

Interview with Tom Chivers: “AI is a plausible existential risk, but it feels as if I’m in Pascal’s mugging”

felix.h21 Feb 2021 13:41 UTC

16 points

1 comment7 min readEA link

[Link and commentary] Beyond Near- and Long-Term: Towards a Clearer Account of Research Priorities in AI Ethics and Society

MichaelA🔸14 Mar 2020 9:04 UTC

18 points

0 comments6 min readEA link

AI Safety Unconference NeurIPS 2022

Orpheus_Lummis7 Nov 2022 15:39 UTC

13 points

5 comments1 min readEA link

(aisafetyevents.org)

Critique of Superintelligence Part 5

Fods1213 Dec 2018 5:19 UTC

12 points

2 comments6 min readEA link

[Question] Who would you have on your dream team for solving AGI Alignment?

Greg_Colbourn25 Aug 2022 13:34 UTC

10 points

14 comments1 min readEA link

AI safety starter pack

mariushobbhahn28 Mar 2022 16:05 UTC

126 points

13 comments6 min readEA link

The alignment problem from a deep learning perspective

richard_ngo11 Aug 2022 3:18 UTC

58 points

0 comments26 min readEA link

AGI alignment results from a series of aligned actions

hanadulset27 Dec 2021 19:33 UTC

15 points

1 comment6 min readEA link

Against Agents as an Approach to Aligned Transformative AI

𝕮𝖎𝖓𝖊𝖗𝖆27 Dec 2022 0:47 UTC

4 points

0 comments1 min readEA link

Consider trying Vivek Hebbar’s alignment exercises

Akash24 Oct 2022 19:46 UTC

16 points

0 comments1 min readEA link

PIBBSS Fellowship: Bounty for Referrals & Deadline Extension

Anna_Gajdova17 Jan 2022 16:23 UTC

17 points

7 comments1 min readEA link

[Question] Should I force myself to work on AGI alignment?

Isaac Benson24 Aug 2022 17:25 UTC

19 points

17 comments1 min readEA link

Why The Focus on Expected Utility Maximisers?

𝕮𝖎𝖓𝖊𝖗𝖆27 Dec 2022 15:51 UTC

11 points

1 comment1 min readEA link

The Vitalik Buterin Fellowship in AI Existential Safety is open for applications!

Cynthia Chen14 Oct 2022 3:23 UTC

38 points

0 comments2 min readEA link

It’s (not) how you use it

Eleni_A7 Sep 2022 13:28 UTC

6 points

3 comments2 min readEA link

[Question] How can we secure more research positions at our universities for x-risk researchers?

Neil Crawford6 Sep 2022 14:41 UTC

3 points

2 comments1 min readEA link

Deception as the optimal: mesa-optimizers and inner alignment

Eleni_A16 Aug 2022 3:45 UTC

19 points

0 comments5 min readEA link

Short-Term AI Alignment as a Priority Cause

len.hoang.lnh11 Feb 2020 16:22 UTC

17 points

11 comments7 min readEA link

We Ran an AI Timelines Retreat

Lenny McCline17 May 2022 4:40 UTC

46 points

6 comments3 min readEA link

The Tree of Life: Stanford AI Alignment Theory of Change

GabeM2 Jul 2022 18:32 UTC

69 points

5 comments14 min readEA link

AI Safety Overview: CERI Summer Research Fellowship

Jamie B24 Mar 2022 15:12 UTC

29 points

0 comments2 min readEA link

A Quick List of Some Problems in AI Alignment As A Field

Nicholas / Heather Kross21 Jun 2022 17:09 UTC

16 points

10 comments6 min readEA link

(www.thinkingmuchbetter.com)

Alignment is hard. Communicating that, might be harder

Eleni_A1 Sep 2022 11:45 UTC

17 points

1 comment3 min readEA link

[Question] Why AGIs utility can’t outweigh humans’ utility?

Alex P20 Sep 2022 5:16 UTC

6 points

25 comments1 min readEA link

Birds, Brains, Planes, and AI: Against Appeals to the Complexity/Mysteriousness/Efficiency of the Brain

kokotajlod18 Jan 2021 12:39 UTC

27 points

2 comments1 min readEA link

LessWrong is now a book, available for pre-order!

jacobjacob4 Dec 2020 20:42 UTC

48 points

1 comment10 min readEA link

Distinguishing test from training

So8res29 Nov 2022 21:41 UTC

27 points

0 comments1 min readEA link

On Artificial General Intelligence: Asking the Right Questions

Heather Douglas2 Oct 2022 5:00 UTC

−1 points

7 comments3 min readEA link

Alignment’s phlogiston

Eleni_A18 Aug 2022 1:41 UTC

18 points

1 comment2 min readEA link

How Roodman’s GWP model translates to TAI timelines

kokotajlod16 Nov 2020 14:11 UTC

22 points

0 comments2 min readEA link

What is the role of Bayesian ML for AI alignment/safety?

mariushobbhahn11 Jan 2022 8:07 UTC

39 points

6 comments3 min readEA link

Being an individual alignment grantmaker

A_donor28 Feb 2022 16:39 UTC

34 points

20 comments2 min readEA link

Frontier AI systems have surpassed the self-replicating red line

Greg_Colbourn10 Dec 2024 16:33 UTC

30 points

14 comments1 min readEA link

(github.com)

[Question] Is there any research or forecasts of how likely AI Alignment is going to be a hard vs. easy problem relative to capabilities?

Jordan Arel14 Aug 2022 15:58 UTC

8 points

1 comment1 min readEA link

[Question] How do you talk about AI safety?

Eevee🔹19 Apr 2020 16:15 UTC

10 points

5 comments1 min readEA link

[Question] Why does (any particular) AI safety work reduce s-risks more than it increases them?

MichaelStJules3 Oct 2021 16:55 UTC

48 points

19 comments1 min readEA link

[Question] Is working on AI safety as dangerous as ignoring it?

jkmh20 Sep 2021 23:06 UTC

10 points

5 comments1 min readEA link

Distillation of “How Likely is Deceptive Alignment?”

NickGabs1 Dec 2022 20:22 UTC

10 points

1 comment10 min readEA link

Three scenarios of pseudo-alignment

Eleni_A5 Sep 2022 20:26 UTC

7 points

0 comments3 min readEA link

Cooperation and Alignment in Delegation Games: You Need Both!

Oliver Sourbut3 Aug 2024 10:16 UTC

4 points

1 comment1 min readEA link

(www.oliversourbut.net)

AI risk hub in Singapore?

kokotajlod29 Oct 2020 11:51 UTC

24 points

3 comments4 min readEA link

Book review: Architects of Intelligence by Martin Ford (2018)

Ofer11 Aug 2020 17:24 UTC

11 points

1 comment2 min readEA link

“Intro to brain-like-AGI safety” series—halfway point!

Steven Byrnes9 Mar 2022 15:21 UTC

8 points

0 comments2 min readEA link

7 traps that (we think) new alignment researchers often fall into

Akash27 Sep 2022 23:13 UTC

73 points

8 comments1 min readEA link

The first AI Safety Camp & onwards

Remmelt7 Jun 2018 18:49 UTC

25 points

2 comments8 min readEA link

A course for the general public on AI

LeandroD31 Aug 2020 1:29 UTC

1 point

0 comments1 min readEA link

The heterogeneity of human value types: Implications for AI alignment

Geoffrey Miller16 Sep 2022 21:21 UTC

27 points

2 comments10 min readEA link

AI Governance Career Paths for Europeans

careersthrowaway16 May 2020 6:40 UTC

83 points

1 comment12 min readEA link

Apply for the ML Winter Camp in Cambridge, UK [2-10 Jan]

Nathan_Barnard2 Dec 2022 19:33 UTC

50 points

11 comments2 min readEA link

DeepMind’s generalist AI, Gato: A non-technical explainer

frances_lorenz16 May 2022 21:19 UTC

128 points

13 comments6 min readEA link

[Creative Writing Contest] Metal or Mortal

Louis16 Oct 2021 16:24 UTC

7 points

0 comments7 min readEA link

Defusing AGI Danger

Mark Xu24 Dec 2020 23:08 UTC

23 points

0 comments2 min readEA link

(www.alignmentforum.org)

Critique of Superintelligence Part 2

Fods1213 Dec 2018 5:12 UTC

10 points

12 comments7 min readEA link

Skilling-up in ML Engineering for Alignment: request for comments

TheMcDouglas24 Apr 2022 6:40 UTC

8 points

0 comments1 min readEA link

Newsletter for Alignment Research: The ML Safety Updates

Esben Kran22 Oct 2022 16:17 UTC

30 points

0 comments7 min readEA link

Visible Thoughts Project and Bounty Announcement

So8res30 Nov 2021 0:35 UTC

35 points

2 comments13 min readEA link

[Creative Writing Contest] The Puppy Problem

Louis13 Oct 2021 14:01 UTC

13 points

0 comments7 min readEA link

A conversation with Rohin Shah

AI Impacts12 Nov 2019 1:31 UTC

27 points

8 comments33 min readEA link

(aiimpacts.org)

Loss of control of AI is not a likely source of AI x-risk

squek9 Nov 2022 5:48 UTC

8 points

0 comments1 min readEA link

European Master’s Programs in Machine Learning, Artificial Intelligence, and related fields

Master Programs ML/AI17 Jan 2021 20:09 UTC

17 points

4 comments1 min readEA link

[Extended Deadline: Jan 23rd] Announcing the PIBBSS Summer Research Fellowship

nora18 Dec 2021 16:54 UTC

36 points

1 comment1 min readEA link

A tough career decision

PabloAMC 🔸9 Apr 2022 0:46 UTC

68 points

13 comments4 min readEA link

Announcing the Cambridge Boston Alignment Initiative [Hiring!]

kuhanj2 Dec 2022 1:07 UTC

83 points

0 comments1 min readEA link

Critique of Superintelligence Part 4

Fods1213 Dec 2018 5:14 UTC

4 points

2 comments4 min readEA link

How to do theoretical research, a personal perspective

Mark Xu19 Aug 2022 19:43 UTC

132 points

7 comments15 min readEA link

[Question] How would a language model become goal-directed?

David M16 Jul 2022 14:50 UTC

113 points

20 comments1 min readEA link

Changes in funding in the AI safety field

Sebastian_Farquhar3 Feb 2017 13:09 UTC

34 points

10 comments7 min readEA link

“Taking AI Risk Seriously” – Thoughts by Andrew Critch

Raemon19 Nov 2018 2:21 UTC

26 points

9 comments1 min readEA link

(www.lesswrong.com)

Four reasons I find AI safety emotionally compelling

Kat Woods28 Jun 2022 14:01 UTC

32 points

5 comments4 min readEA link

Summary of Stuart Russell’s new book, “Human Compatible”

Rohin Shah19 Oct 2019 19:56 UTC

33 points

1 comment15 min readEA link

(www.alignmentforum.org)

AGI will arrive by the end of this decade either as a unicorn or as a black swan

Yuri Barzov21 Oct 2022 10:50 UTC

−4 points

7 comments3 min readEA link

You Understand AI Alignment and How to Make Soup

Leen Armoush28 May 2022 6:22 UTC

0 points

2 comments5 min readEA link

My (naive) take on Risks from Learned Optimization

Artyom K6 Nov 2022 16:25 UTC

5 points

0 comments1 min readEA link

A response to Matthews on AI Risk

RyanCarey11 Aug 2015 12:58 UTC

11 points

16 comments6 min readEA link

[Question] Are social media algorithms an existential risk?

Barry Grimes15 Sep 2020 8:52 UTC

24 points

13 comments1 min readEA link

AI Safety Career Bottlenecks Survey Responses Responses

Linda Linsefors28 May 2021 10:41 UTC

35 points

1 comment5 min readEA link

ML Safety Scholars Summer 2022 Retrospective

TW1231 Nov 2022 3:09 UTC

56 points

2 comments21 min readEA link

The role of academia in AI Safety.

PabloAMC 🔸28 Mar 2022 0:04 UTC

71 points

19 comments3 min readEA link

Annual AGI Benchmarking Event

Metaculus26 Aug 2022 21:31 UTC

20 points

2 comments2 min readEA link

(www.metaculus.com)

Analysis of AI Safety surveys for field-building insights

Ash Jafari5 Dec 2022 17:37 UTC

30 points

7 comments5 min readEA link

Resources that (I think) new alignment researchers should know about

Akash28 Oct 2022 22:13 UTC

20 points

2 comments1 min readEA link

[Question] Why not offer a multi-million / billion dollar prize for solving the Alignment Problem?

Aryeh Englander17 Apr 2022 16:08 UTC

15 points

9 comments1 min readEA link

13 Recent Publications on Existential Risk (Jan 2021 update)

HaydnBelfield8 Feb 2021 12:42 UTC

7 points

2 comments10 min readEA link

[Question] Why not to solve alignment by making superintelligent humans?

Pato16 Oct 2022 21:26 UTC

9 points

12 comments1 min readEA link

Report on Semi-informative Priors for AI timelines (Open Philanthropy)

Tom_Davidson26 Mar 2021 17:46 UTC

62 points

6 comments2 min readEA link

Enabling more feedback

JJ Hepburn10 Dec 2021 6:52 UTC

41 points

3 comments3 min readEA link

‘Force multipliers’ for EA research

Craig Drayton18 Jun 2022 13:39 UTC

18 points

7 comments4 min readEA link

Feedback Request on EA Philippines’ Career Advice Research for Technical AI Safety

BrianTan3 Oct 2020 10:39 UTC

19 points

5 comments4 min readEA link

Any further work on AI Safety Success Stories?

Krieger2 Oct 2022 11:59 UTC

4 points

0 comments1 min readEA link

[Question] Analogy of AI Alignment as Raising a Child?

Aaron_Scher19 Feb 2022 21:40 UTC

4 points

2 comments1 min readEA link

[Question] How long does it take to undersrand AI X-Risk from scratch so that I have a confident, clear mental model of it from first principles?

Jordan Arel27 Jul 2022 16:58 UTC

29 points

6 comments1 min readEA link

[Question] Does China have AI alignment resources/institutions? How can we prioritize creating more?

JakubK4 Aug 2022 19:23 UTC

18 points

9 comments1 min readEA link

New Speaker Series on AI Alignment Starting March 3

Zechen Zhang26 Feb 2022 10:58 UTC

5 points

0 comments1 min readEA link

Critique of Superintelligence Part 1

Fods1213 Dec 2018 5:10 UTC

22 points

13 comments8 min readEA link

[Closed] Prize and fast track to alignment research at ALTER

Vanessa18 Sep 2022 9:15 UTC

38 points

0 comments3 min readEA link

AI Alignment 2018-2019 Review

Habryka28 Jan 2020 21:14 UTC

28 points

0 comments6 min readEA link

(www.lesswrong.com)

How to store human values on a computer

oliver_siegel4 Nov 2022 19:36 UTC

1 point

2 comments1 min readEA link

Promoting compassionate longtermism

jonleighton7 Dec 2022 14:26 UTC

117 points

5 comments12 min readEA link

AI Benefits Post 2: How AI Benefits Differs from AI Alignment & AI for Good

Cullen 🔸29 Jun 2020 16:59 UTC

9 points

0 comments2 min readEA link

Defending against Adversarial Policies in Reinforcement Learning with Alternating Training

sergeivolodin12 Feb 2022 15:59 UTC

1 point

0 comments13 min readEA link

Aligning AI with Humans by Leveraging Legal Informatics

johnjnay18 Sep 2022 7:43 UTC

20 points

11 comments3 min readEA link

My Overview of the AI Alignment Landscape: A Bird’s Eye View

Neel Nanda15 Dec 2021 23:46 UTC

45 points

15 comments16 min readEA link

(www.alignmentforum.org)

Can we simulate human evolution to create a somewhat aligned AGI?

Thomas Kwa29 Mar 2022 1:23 UTC

19 points

0 comments7 min readEA link

Working at EA organizations series: Machine Intelligence Research Institute

SoerenMind1 Nov 2015 12:49 UTC

8 points

0 comments4 min readEA link

Student project for engaging with AI alignment

Per Ivar Friborg9 May 2022 10:44 UTC

35 points

1 comment1 min readEA link

[Question] What “defense layers” should governments, AI labs, and businesses use to prevent catastrophic AI failures?

alexlintz3 Dec 2021 14:24 UTC

37 points

3 comments1 min readEA link

Supporting global coordination in AI development: Why and how to contribute to international AI standards

pcihon17 Apr 2019 22:17 UTC

21 points

4 comments1 min readEA link

Critique of Superintelligence Part 3

Fods1213 Dec 2018 5:13 UTC

3 points

5 comments7 min readEA link

a casual intro to AI doom and alignment

Tamsin Leake2 Nov 2022 9:42 UTC

12 points

2 comments1 min readEA link

Metaculus is building a team dedicated to AI forecasting

christian18 Oct 2022 16:08 UTC

35 points

0 comments1 min readEA link

(apply.workable.com)

Centre for the Study of Existential Risk Four Month Report June—September 2020

HaydnBelfield2 Dec 2020 18:33 UTC

24 points

0 comments17 min readEA link

E.A. Megaproject Ideas

Tomer_Goloboy21 Mar 2022 1:23 UTC

15 points

4 comments4 min readEA link

Data collection for AI alignment—Career review

Benjamin Hilton3 Jun 2022 11:44 UTC

34 points

1 comment5 min readEA link

(80000hours.org)

LW4EA: Some cruxes on impactful alternatives to AI policy work

Jeremy17 May 2022 3:05 UTC

11 points

1 comment1 min readEA link

(www.lesswrong.com)

[Question] Updates on FLI’S Value Alignment Map?

QubitSwarm9919 Sep 2022 0:25 UTC

8 points

0 comments1 min readEA link

What if we don’t need a “Hard Left Turn” to reach AGI?

Eigengender15 Jul 2022 9:49 UTC

39 points

7 comments4 min readEA link

The True Story of How GPT-2 Became Maximally Lewd

Writer18 Jan 2024 21:03 UTC

23 points

1 comment1 min readEA link

(youtu.be)

Summaries: Alignment Fundamentals Curriculum

Leon_Lang19 Sep 2022 15:43 UTC

25 points

1 comment1 min readEA link

(docs.google.com)

Ought’s theory of change

stuhlmueller12 Apr 2022 0:09 UTC

43 points

4 comments3 min readEA link

AI safety scholarships look worth-funding (if other funding is sane)

anon-a19 Nov 2019 0:59 UTC

22 points

6 comments2 min readEA link

Follow along with Columbia EA’s Advanced AI Safety Fellowship!

RohanS2 Jul 2022 6:07 UTC

27 points

0 comments2 min readEA link

Takeaways from a survey on AI alignment resources

DanielFilan5 Nov 2022 23:45 UTC

18 points

9 comments6 min readEA link

(www.lesswrong.com)

The flaws that make today’s AI architecture unsafe and a new approach that could fix it

80000_Hours22 Jun 2020 22:15 UTC

3 points

0 comments86 min readEA link

(80000hours.org)

[Question] Donating against Short Term AI risks

Jan-Willem16 Nov 2020 12:23 UTC

6 points

10 comments1 min readEA link

Preserving and continuing alignment research through a severe global catastrophe

A_donor6 Mar 2022 18:43 UTC

40 points

11 comments5 min readEA link

Database of existential risk estimates

MichaelA🔸15 Apr 2020 12:43 UTC

130 points

37 comments5 min readEA link

AGI Safety Communications Initiative

Ines11 Jun 2022 16:30 UTC

35 points

6 comments1 min readEA link

What Should We Optimize—A Conversation

Johannes C. Mayer7 Apr 2022 14:48 UTC

1 point

0 comments14 min readEA link

What can the principal-agent literature tell us about AI risk?

ac10 Feb 2020 10:10 UTC

26 points

1 comment16 min readEA link

Amanda Askell: AI safety needs social scientists

EA Global4 Mar 2019 15:50 UTC

27 points

0 comments18 min readEA link

(www.youtube.com)

Stuart Russell Human Compatible AI Roundtable with Allan Dafoe, Rob Reich, & Marietje Schaake

Mahendra Prasad11 Feb 2021 7:43 UTC

16 points

0 comments1 min readEA link

[Question] Can we convince people to work on AI safety without convincing them about AGI happening this century?

BrianTan26 Nov 2020 14:46 UTC

8 points

3 comments2 min readEA link

Cortés, Pizarro, and Afonso as Precedents for Takeover

AI Impacts2 Mar 2020 12:25 UTC

27 points

17 comments11 min readEA link

(aiimpacts.org)

Redwood Research is hiring for several roles (Operations and Technical)

JJXWang14 Apr 2022 15:23 UTC

45 points

0 comments1 min readEA link

[DISC] Are Values Robust?

𝕮𝖎𝖓𝖊𝖗𝖆21 Dec 2022 1:13 UTC

4 points

0 comments1 min readEA link

Singapore’s Technical AI Alignment Research Career Guide

Yi-Yang26 Aug 2020 8:09 UTC

34 points

7 comments8 min readEA link

[Closed] Hiring a mathematician to work on the learning-theoretic AI alignment agenda

Vanessa19 Apr 2022 6:49 UTC

53 points

4 comments2 min readEA link

How might we solve the alignment problem? (Part 1: Intro, summary, ontology)

Joe_Carlsmith28 Oct 2024 21:57 UTC

11 points

0 comments1 min readEA link

AI Safety Ideas: A collaborative AI safety research platform

Apart Research17 Oct 2022 17:01 UTC

67 points

13 comments4 min readEA link

Podcast: Tamera Lanham on AI risk, threat models, alignment proposals, externalized reasoning oversight, and working at Anthropic

Akash20 Dec 2022 21:39 UTC

14 points

1 comment1 min readEA link

Announcing AI Alignment Awards: $100k research contests about goal misgeneralization & corrigibility

Akash22 Nov 2022 22:19 UTC

60 points

1 comment1 min readEA link

Reflections on the PIBBSS Fellowship 2022

nora11 Dec 2022 22:03 UTC

69 points

4 comments18 min readEA link

“Clean” vs. “messy” goal-directedness (Section 2.2.3 of “Scheming AIs”)

Joe_Carlsmith29 Nov 2023 16:32 UTC

7 points

0 comments1 min readEA link

“Normal accidents” and AI systems

Eleni_A8 Aug 2022 18:43 UTC

5 points

1 comment1 min readEA link

(www.achan.ca)

Three Biases That Made Me Believe in AI Risk

beth13 Feb 2019 23:22 UTC

41 points

20 comments3 min readEA link

How Josiah became an AI safety researcher

Neil Crawford29 Mar 2022 19:47 UTC

10 points

0 comments1 min readEA link

Re: Some thoughts on vegetarianism and veganism

Fai25 Feb 2022 20:43 UTC

46 points

3 comments8 min readEA link

Are Humans ‘Human Compatible’?

Matt Boyd6 Dec 2019 5:49 UTC

23 points

8 comments4 min readEA link

Criticism of the main framework in AI alignment

Michele Campolo31 Aug 2022 21:44 UTC

42 points

4 comments7 min readEA link

EA’s brain-over-body bias, and the embodied value problem in AI alignment

Geoffrey Miller21 Sep 2022 18:55 UTC

45 points

3 comments25 min readEA link

New series of posts answering one of Holden’s “Important, actionable research questions”

Evan R. Murphy12 May 2022 21:22 UTC

9 points

0 comments1 min readEA link

FYI: I’m working on a book about the threat of AGI/ASI for a general audience. I hope it will be of value to the cause and the community

Darren McKee17 Jun 2022 11:52 UTC

32 points

1 comment2 min readEA link

AI Alignment YouTube Playlists

jacquesthibs9 May 2022 21:31 UTC

16 points

2 comments1 min readEA link

[Question] Book recommendations for the history of ML?

Eleni_A28 Dec 2022 23:45 UTC

10 points

4 comments1 min readEA link

AI Forecasting Resolution Council (Forecasting infrastructure, part 2)

jacobjacob29 Aug 2019 17:43 UTC

28 points

0 comments3 min readEA link

You won’t solve alignment without agent foundations

MikhailSamin6 Nov 2022 8:07 UTC

14 points

0 comments1 min readEA link

[Question] Best introductory overviews of AGI safety?

JakubK13 Dec 2022 19:04 UTC

21 points

8 comments2 min readEA link

(www.lesswrong.com)

Crypto ‘oracle protocols’ for AI alignment with real-world data?

Geoffrey Miller22 Sep 2022 23:05 UTC

9 points

3 comments1 min readEA link

Key questions about artificial sentience: an opinionated guide

rgb25 Apr 2022 13:42 UTC

91 points

3 comments1 min readEA link

My summary of “Pragmatic AI Safety”

Eleni_A5 Nov 2022 14:47 UTC

14 points

0 comments5 min readEA link

(My suggestions) On Beginner Steps in AI Alignment

Joseph Bloom22 Sep 2022 15:32 UTC

36 points

3 comments9 min readEA link

The religion problem in AI alignment

Geoffrey Miller16 Sep 2022 1:24 UTC

54 points

28 comments11 min readEA link

Posit: Most AI safety people should work on alignment/safety challenges for AI tools that already have users (Stable Diffusion, GPT)

nonzerosum20 Dec 2022 17:23 UTC

12 points

3 comments1 min readEA link

How to Diversify Conceptual AI Alignment: the Model Behind Refine

adamShimi20 Jul 2022 10:44 UTC

43 points

0 comments9 min readEA link

(www.alignmentforum.org)

[Question] Does the idea of AGI that benevolently control us appeal to EA folks?

Noah Scales16 Jul 2022 19:17 UTC

6 points

20 comments1 min readEA link

Appendix to Bridging Demonstration

mako yass1 Jun 2022 20:30 UTC

18 points

2 comments28 min readEA link

[Discussion] Best intuition pumps for AI safety

mariushobbhahn6 Nov 2021 8:11 UTC

10 points

8 comments1 min readEA link

Our Current Directions in Mechanistic Interpretability Research (AI Alignment Speaker Series)

Group Organizer8 Apr 2022 17:08 UTC

3 points

0 comments1 min readEA link

AI Risk in Africa

Claude Formanek12 Oct 2021 2:28 UTC

18 points

0 comments10 min readEA link

Provably Honest—A First Step

Srijanak De5 Nov 2022 21:49 UTC

1 point

0 comments1 min readEA link

List of AI safety courses and resources

Daniel del Castillo6 Sep 2021 14:26 UTC

50 points

8 comments1 min readEA link

Anti-squatted AI x-risk domains index

plex12 Aug 2022 12:00 UTC

56 points

9 comments1 min readEA link

[Question] 1h-volunteers needed for a small AI Safety-related research project

PabloAMC 🔸16 Aug 2021 17:51 UTC

4 points

0 comments1 min readEA link

Introducing the Fund for Alignment Research (We’re Hiring!)

AdamGleave6 Jul 2022 2:00 UTC

74 points

3 comments4 min readEA link

On Solving Problems Before They Appear: The Weird Epistemologies of Alignment

adamShimi11 Oct 2021 8:21 UTC

28 points

0 comments15 min readEA link

fiction about AI risk

Ann Garth 🔸12 Nov 2020 22:36 UTC

8 points

1 comment1 min readEA link

Who ordered alignment’s apple?

Eleni_A28 Aug 2022 14:24 UTC

5 points

0 comments3 min readEA link

[3-hour podcast]: Joseph Carlsmith on longtermism, utopia, the computational power of the brain, meta-ethics, illusionism and meditation

Gus Docker27 Jul 2021 13:18 UTC

34 points

2 comments1 min readEA link

EA Berkeley Presents: Universal Ownership: Is Index Investing the New Socially Responsible Investing?

Mahendra Prasad10 Mar 2022 6:58 UTC

7 points

0 comments1 min readEA link

Contribute by facilitating the AGI Safety Fundamentals Programme

Jamie B6 Dec 2021 11:50 UTC

27 points

0 comments2 min readEA link

AI Forecasting Question Database (Forecasting infrastructure, part 3)

jacobjacob3 Sep 2019 14:57 UTC

23 points

2 comments4 min readEA link

Intro to caring about AI alignment as an EA cause

So8res14 Apr 2017 0:42 UTC

28 points

10 comments25 min readEA link

[Link] Thiel on GCRs

Milan_Griffes22 Jul 2019 20:47 UTC

28 points

11 comments1 min readEA link

AI Value Alignment Speaker Series Presented By EA Berkeley

Mahendra Prasad1 Mar 2022 6:17 UTC

2 points

0 comments1 min readEA link

General advice for transitioning into Theoretical AI Safety

Martín Soto15 Sep 2022 5:23 UTC

25 points

0 comments10 min readEA link

AI acceleration from a safety perspective: Trade-offs and considerations

mariushobbhahn19 Jan 2022 9:44 UTC

12 points

1 comment7 min readEA link

Desirable? AI qualities

brb24321 Mar 2022 22:05 UTC

7 points

0 comments2 min readEA link

Confused about AI research as a means of addressing AI risk

Eli Rose21 Feb 2019 0:07 UTC

31 points

15 comments1 min readEA link

[Link post] Promising Paths to Alignment—Connor Leahy | Talk

frances_lorenz14 May 2022 15:58 UTC

17 points

0 comments1 min readEA link

A stubborn unbeliever finally gets the depth of the AI alignment problem

aelwood13 Oct 2022 15:16 UTC

32 points

7 comments1 min readEA link

Discovering Language Model Behaviors with Model-Written Evaluations

evhub20 Dec 2022 20:09 UTC

25 points

0 comments1 min readEA link

But exactly how complex and fragile?

Katja_Grace13 Dec 2019 7:05 UTC

37 points

3 comments3 min readEA link

(meteuphoric.com)

Begging, Pleading AI Orgs to Comment on NIST AI Risk Management Framework

Bridges15 Apr 2022 19:35 UTC

87 points

3 comments2 min readEA link

[Question] How to get more academics enthusiastic about doing AI Safety research?

PabloAMC 🔸4 Sep 2021 14:10 UTC

25 points

19 comments1 min readEA link

Does generality pay? GPT-3 can provide preliminary evidence.

Eevee🔹12 Jul 2020 18:53 UTC

21 points

4 comments2 min readEA link

Implications of Quantum Computing for Artificial Intelligence alignment research (ABRIDGED)

Jaime Sevilla5 Sep 2019 14:56 UTC

25 points

4 comments2 min readEA link

My thoughts on OpenAI’s alignment plan

Akash30 Dec 2022 19:34 UTC

16 points

0 comments1 min readEA link

“AI” is an indexical

TW1233 Jan 2023 22:00 UTC

23 points

2 comments1 min readEA link

Solving alignment isn’t enough for a flourishing future

mic2 Feb 2024 18:22 UTC

27 points

0 comments22 min readEA link

(papers.ssrn.com)

ChatGPT understands, but largely does not generate Spanglish (and other code-mixed) text

Milan Weibel🔹4 Jan 2023 22:10 UTC

6 points

0 comments4 min readEA link

(www.lesswrong.com)

Learning as much Deep Learning math as I could in 24 hours

Phosphorous8 Jan 2023 2:19 UTC

58 points

6 comments7 min readEA link

David Krueger on AI Alignment in Academia and Coordination

Michaël Trazzi7 Jan 2023 21:14 UTC

32 points

1 comment3 min readEA link

(theinsideview.ai)

ML Summer Bootcamp Reflection: Aalto EA Finland

Aayush Kucheria12 Jan 2023 8:24 UTC

15 points

2 comments9 min readEA link

[Question] Any Philosophy PhD recommendations for students interested in Alignment Efforts?

rickyhuang.hexuan18 Jan 2023 5:54 UTC

7 points

6 comments1 min readEA link

Preparing for AI-assisted alignment research: we need data!

CBiddulph17 Jan 2023 3:28 UTC

11 points

0 comments11 min readEA link

Jan Kirchner on AI Alignment

birtes17 Jan 2023 15:11 UTC

5 points

0 comments1 min readEA link

Emerging Paradigms: The Case of Artificial Intelligence Safety

Eleni_A18 Jan 2023 5:59 UTC

16 points

0 comments19 min readEA link

UK AI Bill Analysis & Opinion

CAISID5 Feb 2024 0:12 UTC

18 points

0 comments15 min readEA link

11 heuristics for choosing (alignment) research projects

Akash27 Jan 2023 0:36 UTC

30 points

1 comment1 min readEA link

On value in humans, other animals, and AI

Michele Campolo31 Jan 2023 23:48 UTC

7 points

6 comments5 min readEA link

[Linkpost] Human-narrated audio version of “Is Power-Seeking AI an Existential Risk?”

Joe_Carlsmith31 Jan 2023 19:19 UTC

9 points

0 comments1 min readEA link

Alexander and Yudkowsky on AGI goals

Scott Alexander31 Jan 2023 23:36 UTC

29 points

1 comment1 min readEA link

Focus on the places where you feel shocked everyone’s dropping the ball

So8res2 Feb 2023 0:27 UTC

92 points

6 comments1 min readEA link

An audio version of the alignment problem from a deep learning perspective by Richard Ngo Et Al

Miguel3 Feb 2023 19:32 UTC

18 points

0 comments1 min readEA link

(www.whitehatstoic.com)

A discussion with ChatGPT on value-based models vs. large language models, etc..

Miguel4 Feb 2023 16:49 UTC

4 points

0 comments12 min readEA link

(www.whitehatstoic.com)

Interview with Roman Yampolskiy about AGI on The Reality Check

Darren McKee18 Feb 2023 23:29 UTC

27 points

0 comments1 min readEA link

(www.trcpodcast.com)

Overview | An Evaluative Evolution

Matt Keene10 Feb 2023 18:15 UTC

−9 points

0 comments5 min readEA link

(www.creatingafuturewewant.com)

AI Safety Info Distillation Fellowship

robertskmiles17 Feb 2023 16:16 UTC

80 points

1 comment1 min readEA link

AI alignment researchers don’t (seem to) stack

So8res21 Feb 2023 0:48 UTC

47 points

3 comments1 min readEA link

Seeking input on a list of AI books for broader audience

Darren McKee27 Feb 2023 22:40 UTC

49 points

14 comments5 min readEA link

AI alignment as a translation problem

Roman Leventov5 Feb 2024 14:14 UTC

3 points

1 comment1 min readEA link

Anthropic: Core Views on AI Safety: When, Why, What, and How

jonmenaster9 Mar 2023 17:30 UTC

107 points

6 comments22 min readEA link

(www.anthropic.com)

Questions about Conjecure’s CoEm proposal

Akash9 Mar 2023 19:32 UTC

19 points

0 comments1 min readEA link

Apply to a small iteration of MLAB to be run in Oxford

Rio P29 Aug 2023 19:39 UTC

11 points

0 comments1 min readEA link

AI safety and consciousness research: A brainstorm

Daniel_Friedrich15 Mar 2023 14:33 UTC

11 points

1 comment9 min readEA link

Why focus on schemers in particular (Sections 1.3 and 1.4 of “Scheming AIs”)

Joe_Carlsmith24 Nov 2023 19:18 UTC

10 points

1 comment1 min readEA link

Apply for MATS Winter 2023-24!

utilistrutil21 Oct 2023 2:34 UTC

34 points

2 comments5 min readEA link

(www.lesswrong.com)

Potential employees have a unique lever to influence the behaviors of AI labs

oxalis18 Mar 2023 20:58 UTC

139 points

1 comment5 min readEA link

Don’t Dismiss Simple Alignment Approaches

Chris Leong21 Oct 2023 12:31 UTC

12 points

0 comments1 min readEA link

[linkpost] Ten Levels of AI Alignment Difficulty

SammyDMartin4 Jul 2023 11:23 UTC

16 points

0 comments1 min readEA link

Announcing Timaeus

Stan van Wingerden22 Oct 2023 13:32 UTC

78 points

0 comments5 min readEA link

(www.lesswrong.com)

Announcing New Beginner-friendly Book on AI Safety and Risk

Darren McKee25 Nov 2023 15:57 UTC

112 points

9 comments1 min readEA link

Is scheming more likely in models trained to have long-term goals? (Sections 2.2.4.1-2.2.4.2 of “Scheming AIs”)

Joe_Carlsmith30 Nov 2023 16:43 UTC

6 points

1 comment1 min readEA link

Announcing #AISummitTalks featuring Professor Stuart Russell and many others

Otto24 Oct 2023 10:16 UTC

9 points

1 comment1 min readEA link

OpenAI is starting a new “Superintelligence alignment” team and they’re hiring

Alejandro Ortega5 Jul 2023 18:27 UTC

100 points

16 comments1 min readEA link

(openai.com)

Varieties of fake alignment (Section 1.1 of “Scheming AIs”)

Joe_Carlsmith21 Nov 2023 15:00 UTC

6 points

0 comments1 min readEA link

The Dissolution of AI Safety

Roko12 Dec 2024 10:46 UTC

−7 points

0 comments1 min readEA link

(www.transhumanaxiology.com)

[Question] What new psychology research could best promote AI safety & alignment research?

Geoffrey Miller13 Jul 2023 16:30 UTC

29 points

13 comments1 min readEA link

Winners of AI Alignment Awards Research Contest

Akash13 Jul 2023 16:14 UTC

49 points

1 comment1 min readEA link

“The Universe of Minds”—call for reviewers (Seeds of Science)

rogersbacon125 Jul 2023 16:55 UTC

4 points

0 comments1 min readEA link

AXRP Episode 24 - Superalignment with Jan Leike

DanielFilan27 Jul 2023 4:56 UTC

23 points

0 comments1 min readEA link

(axrp.net)

[Question] I’m interviewing Jan Leike, co-lead of OpenAI’s new Superalignment project. What should I ask him?

Robert_Wiblin18 Jul 2023 18:25 UTC

51 points

19 comments1 min readEA link

[Crosspost] An AI Pause Is Humanity’s Best Bet For Preventing Extinction (TIME)

Otto24 Jul 2023 10:18 UTC

36 points

3 comments7 min readEA link

(time.com)

Carl Shulman on AI takeover mechanisms (& more): Part II of Dwarkesh Patel interview for The Lunar Society

Alejandro Ortega25 Jul 2023 18:31 UTC

28 points

0 comments5 min readEA link

(www.dwarkeshpatel.com)

3 levels of threat obfuscation

Holden Karnofsky2 Aug 2023 17:09 UTC

31 points

0 comments6 min readEA link

(www.alignmentforum.org)

The Concept of Boundary Layer in Language Games and Its Implications for AI

Mirage24 Mar 2023 13:50 UTC

1 point

0 comments7 min readEA link

Sparks of Artificial General Intelligence: Early experiments with GPT-4 | Microsoft Research

𝕮𝖎𝖓𝖊𝖗𝖆23 Mar 2023 5:45 UTC

15 points

0 comments1 min readEA link

A stylized dialogue on John Wentworth’s claims about markets and optimization

So8res25 Mar 2023 22:32 UTC

18 points

0 comments1 min readEA link

[Question] Half-baked alignment idea

ozb28 Mar 2023 5:18 UTC

9 points

2 comments1 min readEA link

A rough and incomplete review of some of John Wentworth’s research

So8res28 Mar 2023 18:52 UTC

27 points

0 comments1 min readEA link

Want to win the AGI race? Solve alignment.

leopold29 Mar 2023 15:19 UTC

56 points

6 comments5 min readEA link

(www.forourposterity.com)

The fundamental human value is power.

Linyphia30 Mar 2023 15:15 UTC

−1 points

5 comments1 min readEA link

Recruit the World’s best for AGI Alignment

Greg_Colbourn30 Mar 2023 16:41 UTC

34 points

8 comments22 min readEA link

AI and Evolution

Dan H30 Mar 2023 13:09 UTC

41 points

1 comment2 min readEA link

(arxiv.org)

[Question] What are the biggest obstacles on AI safety research career?

jackchang11031 Mar 2023 14:53 UTC

2 points

1 comment1 min readEA link

Pessimism about AI Safety

Max_He-Ho2 Apr 2023 7:57 UTC

5 points

0 comments25 min readEA link

(www.lesswrong.com)

Two concepts of an “episode” (Section 2.2.1 of “Scheming AIs”)

Joe_Carlsmith27 Nov 2023 18:01 UTC

11 points

1 comment1 min readEA link

GPTs are Predictors, not Imitators

EliezerYudkowsky8 Apr 2023 19:59 UTC

74 points

12 comments1 min readEA link

[Question] Predictions for future AI governance?

jackchang1102 Apr 2023 16:43 UTC

4 points

1 comment1 min readEA link

Orthogonality is Expensive

𝕮𝖎𝖓𝖊𝖗𝖆3 Apr 2023 1:57 UTC

18 points

4 comments1 min readEA link

If interpretability research goes well, it may get dangerous

So8res3 Apr 2023 21:48 UTC

33 points

0 comments1 min readEA link

The Orthogonality Thesis is Not Obviously True

Omnizoid5 Apr 2023 21:08 UTC

18 points

12 comments9 min readEA link

AI Control idea: Give an AGI the primary objective of deleting itself, but construct obstacles to this as best we can. All other objectives are secondary to this primary goal.

Justausername3 Apr 2023 14:32 UTC

7 points

4 comments1 min readEA link

AI as a science, and three obstacles to alignment strategies

So8res25 Oct 2023 21:02 UTC

41 points

1 comment1 min readEA link

EA Explorer GPT: A New Tool to Explore Effective Altruism

Vlad_Tislenko12 Nov 2023 15:36 UTC

12 points

1 comment1 min readEA link

Podcast/video/transcript: Eliezer Yudkowsky—Why AI Will Kill Us, Aligning LLMs, Nature of Intelligence, SciFi, & Rationality

PeterSlattery9 Apr 2023 10:37 UTC

32 points

2 comments137 min readEA link

(www.youtube.com)

A New Model for Compute Center Verification

Damin Curtis🔹10 Oct 2023 19:23 UTC

21 points

2 comments5 min readEA link

Scalable And Transferable Black-Box Jailbreaks For Language Models Via Persona Modulation

soroushjp7 Nov 2023 18:00 UTC

10 points

0 comments2 min readEA link

(arxiv.org)

Developing a Calculable Conscience for AI: Equation for Rights Violations

Sean Sweeney12 Dec 2024 17:50 UTC

4 points

1 comment14 min readEA link

Perché il deep learning moderno potrebbe rendere difficile l’allineamento delle IA

EA Italy17 Jan 2023 23:29 UTC

1 point

0 comments16 min readEA link

Prevenire una catastrofe legata all’intelligenza artificiale

EA Italy17 Jan 2023 11:07 UTC

1 point

0 comments3 min readEA link

Not Just For Therapy Chatbots: The Case For Compassion In AI Moral Alignment Research

Kenneth_Diao29 Sep 2024 22:58 UTC

8 points

3 comments12 min readEA link

Orthogonal: A new agent foundations alignment organization

Tamsin Leake19 Apr 2023 20:17 UTC

38 points

0 comments1 min readEA link

Safety-First Agents/Architectures Are a Promising Path to Safe AGI

Brendon_Wong6 Aug 2023 8:00 UTC

6 points

0 comments12 min readEA link

Archetypal Transfer Learning: a Proposed Alignment Solution that solves the Inner x Outer Alignment Problem while adding Corrigible Traits to GPT-2-medium

Miguel26 Apr 2023 0:40 UTC

13 points

0 comments10 min readEA link

Research agenda: Supervising AIs improving AIs

Quintin Pope29 Apr 2023 17:09 UTC

16 points

0 comments1 min readEA link

A Guide to Forecasting AI Science Capabilities

Eleni_A29 Apr 2023 6:51 UTC

19 points

1 comment4 min readEA link

How useful for alignment-relevant work are AIs with short-term goals? (Section 2.2.4.3 of “Scheming AIs”)

Joe_Carlsmith1 Dec 2023 14:51 UTC

6 points

0 comments1 min readEA link

Call for Pythia-style foundation model suite for alignment research

Lucretia1 May 2023 20:26 UTC

10 points

0 comments1 min readEA link

The Hidden Complexity of Wishes—The Animation

Writer27 Sep 2023 17:59 UTC

7 points

0 comments1 min readEA link

(youtu.be)

Orthogonal’s Formal-Goal Alignment theory of change

Tamsin Leake5 May 2023 22:36 UTC

21 points

0 comments1 min readEA link

Why “just make an agent which cares only about binary rewards” doesn’t work.

Lysandre Terrisse9 May 2023 16:51 UTC

4 points

1 comment3 min readEA link

Unveiling the American Public Opinion on AI Moratorium and Government Intervention: The Impact of Media Exposure

Otto8 May 2023 10:49 UTC

28 points

5 comments6 min readEA link

Public Call for Interest in Mathematical Alignment

Davidmanheim22 Nov 2023 13:22 UTC

27 points

3 comments1 min readEA link

Why Is No One Trying To Align Profit Incentives With Alignment Research?

Prometheus23 Aug 2023 13:19 UTC

17 points

2 comments4 min readEA link

(www.lesswrong.com)

Origin and alignment of goals, meaning, and morality

FalseCogs24 Aug 2023 14:05 UTC

1 point

2 comments35 min readEA link

No comments.

AI alignment

Evaluation

Further reading

External links

Related entries