Otto

• OttoAug 25, 2023, 7:17 AM

  1 point

  0 ∶ 0

  in reply to: rileyharris’s comment on: [Cross­post] AI Reg­u­la­tion May Be More Im­por­tant Than AI Align­ment For Ex­is­ten­tial Safety

  Thanks for the comment. I think the ways an aligned AGI could make the world safer against unaligned AGIs can be divided in two categories: preventing unaligned AGIs from coming into existence or stopping already existing unaligned AGIs from causing extinction. The second is the offense/​defense balance. The first is what you point at.
  
  If an AGI would prevent people from creating AI, this would likely be against their will. A state would be the only actor who could do so legally, assuming there is regulation in place, and also most practically. Therefore, I think your option falls under what I described in my post as “Types of AI (hardware) regulation may be possible where the state actors implementing the regulation are aided by aligned AIs”. I think this is indeed a realistic option and it may reduce existential risk somewhat. Getting the regulation in place at all, however, seems more important at this point than developing what I see as a pretty far-fetched and—at the moment—intractable way to implement it more effectively.

[Cross­post] AI Reg­u­la­tion May Be More Im­por­tant Than AI Align­ment For Ex­is­ten­tial Safety

[Cross­post] An AI Pause Is Hu­man­ity’s Best Bet For Prevent­ing Ex­tinc­tion (TIME)

• OttoJun 1, 2023, 1:49 PM

  1 point

  0 ∶ 0

  in reply to: Peter Wildeford’s comment on: Un­veiling the Amer­i­can Public Opinion on AI Mo­ra­to­rium and Govern­ment In­ter­ven­tion: The Im­pact of Me­dia Exposure

  Hi Peter, thanks for your comment. We do think the conclusions we draw are robust based on our sample size. If course it depends on the signal: if there’s a change in e.g. awareness from 5% to 50%, a small sample size should be plenty to show that. However, if you’re trying to measure a signal of only 1% difference, your sample size should be much larger. While we stand by our conclusions, we do think there would be significant value in others doing similar research, if possible with larger sample sizes.

  Again, thanks for your comments, we take the input into account.

• OttoMay 16, 2023, 12:59 PM

  1 point

  0 ∶ 0

  in reply to: Greg_Colbourn ⏸️ ’s comment on: AGI ris­ing: why we are in a new era of acute risk and in­creas­ing pub­lic aware­ness, and what to do now

  Yes exactly!

• OttoMay 10, 2023, 10:08 AM

  2 points

  0 ∶ 0

  in reply to: GabeM’s comment on: Un­veiling the Amer­i­can Public Opinion on AI Mo­ra­to­rium and Govern­ment In­ter­ven­tion: The Im­pact of Me­dia Exposure

  Thanks Gabriel! Sorry for the confusion. TE stands for The Economist, so this item: https://​​www.youtube.com/​​watch?v=ANn9ibNo9SQ

• OttoMay 9, 2023, 10:41 AM

  1 point

  0 ∶ 0

  in reply to: jackva’s comment on: How Eng­ineers can Effec­tively Con­tribute to Tack­ling Cli­mate Change

  Thanks for your reply. I mostly agree with many of the things you say, but I still think work to reduce the amount of emission rights should at least be on the list of high-impact things to do, and as far as I’m concerned, significantly higher than a few other paths mentioned here.
  
  If you’d still want to do technology-specific work, I think offshore solar might also be impactful and neglected.

• OttoMay 8, 2023, 9:09 PM

  3 points

  0 ∶ 0

  on: How Eng­ineers can Effec­tively Con­tribute to Tack­ling Cli­mate Change

  As someone who has worked in sustainable energy technology for ten years (wind energy, modeling, smart charging, activism) before moving into AI xrisk, my favorite neglected topic is carbon emission trading schemes (ETS).

  ETSs such as implemented by the EU, China, and others, have a waterbed effect. The total amount of emissions is capped, and trading sets the price of those emissions for all sectors under the scheme (in the EU electricity, heavy industry, expanding to other sectors). That means that:

  1. Reducing emissions for sectors under an ETS is pointless, climate-wise.

  2. Deciding to reduce the amount of emission rights within an ETS should directly lead to lower emissions, without any need to understand the technologies involved.

  It’s just crazy to think about all the good-hearted campaigning, awareness creation, hard engineering work, money, etc that is being directed to decreasing emissions for a sector that’s covered by an ETS. To my best understanding, as long as ETS is working correctly, this effort is completely meaningless. At the same time, I knew of exactly one person trying to reduce ETS emission rights based in my country, the Netherlands. This was the only person potentially actually achieving something useful for the climate.

  If I would want to do something neglected in the climate space, I would try to inform all those people currently wasting their energy that what they should really do is trying to reduce the amount of ETS emission rights and let the market figure out the rest. (Note that several of the trajectories recommended above, such as working on nuclear power, reducing industry emissions, and deep geothermal energy (depending on use case) are all contained in ETS (at least in the EU) and improvements would therefore not benefit the climate).

  If countries or regions have an ETS system, successful emission reduction should really start (and basically stop) there. It’s also quite a neglected area so plenty of low hanging fruit!

Un­veiling the Amer­i­can Public Opinion on AI Mo­ra­to­rium and Govern­ment In­ter­ven­tion: The Im­pact of Me­dia Exposure

AI X-risk in the News: How Effec­tive are Re­cent Me­dia Items and How is Aware­ness Chang­ing? Our New Sur­vey Re­sults.

• OttoMay 4, 2023, 9:56 AM

  13 points

  2 ∶ 1

  on: AGI ris­ing: why we are in a new era of acute risk and in­creas­ing pub­lic aware­ness, and what to do now

  I don’t know if everyone should drop everything else right now, but I do agree that raising awareness about AI xrisks should be a major cause area. That’s why I quit my work on the energy transition about two years ago to found the Existential Risk Observatory, and this is what we’ve been doing since (resulting in about ten articles in leading Dutch newspapers, this one in TIME, perhaps the first comms research, a sold out debate, and a passed parliamentary motion in the Netherlands).
  
  I miss two significant things on the list of what people can do to help:

  1) Please, technical people, work on AI Pause regulation proposals! There is basically one paper now, possibly because everyone else thought a pause was too far outside the Overton window. Now we’re discussing a pause anyway and I personally think it might be implemented at some point, but we don’t have proper AI Pause regulation proposals, which is a really bad situation. Researchers (both policy and technical), please fix that, fix it publicly, and fix it soon!
  2) You can start institutes or projects that aim to inform the societal debate about AI existential risk. We’ve done that and I would say it worked pretty well so far. Others could do the same thing. Funders should be able to choose from a range of AI xrisk communication projects to spend their money most effectively. This is currently really not the case.

Or­ga­niz­ing a de­bate with ex­perts and MPs to raise AI xrisk aware­ness: a pos­si­ble blueprint

• OttoApr 14, 2023, 12:03 PM

  3 points

  0 ∶ 0

  in reply to: Vasco Grilo🔸’s comment on: Paper Sum­mary: The Effec­tive­ness of AI Ex­is­ten­tial Risk Com­mu­ni­ca­tion to the Amer­i­can and Dutch Public

  Hi Vasco, thank you for taking the time to read our paper!
  
  Although we did not specify this in the methodology section, we addressed the “mean variation in likelihood” between countries and surveys throughout the research such as in section 4.2.2. I hope this clarifies your question. This aspect should have been better specified in the methodology section.
  
  If you have any more questions, do not hesitate to ask.

• OttoApr 14, 2023, 12:02 PM

  1 point

  0 ∶ 0

  in reply to: Joshc’s comment on: Paper Sum­mary: The Effec­tive­ness of AI Ex­is­ten­tial Risk Com­mu­ni­ca­tion to the Amer­i­can and Dutch Public

  Hi Joshc, thanks and sorry for the slow reply, it’s a good idea! Unfortunately we don’t really have time right now, but we might do something like this in the future. Also, if you’re interested in receiving the raw data, let us know. Thanks again for the suggestion!

• OttoMar 30, 2023, 2:47 PM

  14 points

  2 ∶ 0

  on: Paus­ing AI Devel­op­ments Isn’t Enough. We Need to Shut it All Down by Eliezer Yudkowsky

  I hope that this article sends the signals that pausing the development of the largest AI-models is good, informing society about AGI xrisk is good, and that we should find a coordination method (regulation) to make sure we can effectively stop training models that are too capable.
  
  What I think we should do now is:
  
  1) Write good hardware regulation policy proposals that could reliably pause the development towards AGI.
  2) Campaign publicly to get the best proposal implemented, first in the US and then internationally.
  
  This could be a path to victory.

• OttoMar 24, 2023, 12:04 AM

  18 points

  7 ∶ 0

  on: The Over­ton Win­dow widens: Ex­am­ples of AI risk in the media

  Crossposting a comment: As co-author of one of the mentioned pieces, I’d say it’s really great to see the AGI xrisk message mainstreaming. It doesn’t nearly go fast enough, though. Some (Hawking, Bostrom, Musk) have already spoken out about the topic for close to a decade. So far, that hasn’t been enough to change common understanding. Those, such as myself, who hope that some form of coordination could save us, should give all they have to make this go faster. Additionally, those who think regulation could work should work on robust regulation proposals which are currently lacking. And those who can should work on international coordination, which is currently also lacking.
  
  A lot of work to be done. But the good news is that the window of opportunity is opening, and a lot of people could work on this which currently aren’t. This could be a path to victory.

• OttoMar 22, 2023, 8:01 PM

  9 points

  6 ∶ 0

  on: An­nounc­ing the Euro­pean Net­work for AI Safety (ENAIS)

  Great idea, congrats on the founding and looking forward to working with you!

• OttoMar 10, 2023, 10:45 PM

  3 points

  0 ∶ 0

  in reply to: PeterSlattery’s comment on: Paper Sum­mary: The Effec­tive­ness of AI Ex­is­ten­tial Risk Com­mu­ni­ca­tion to the Amer­i­can and Dutch Public

  Thanks Peter for the compliment! If there is something in particular you’re interested in, please let us know and perhaps we can take it into account in future research projects!

Paper Sum­mary: The Effec­tive­ness of AI Ex­is­ten­tial Risk Com­mu­ni­ca­tion to the Amer­i­can and Dutch Public

[Cross­post] Why Un­con­trol­lable AI Looks More Likely Than Ever