Guillaume Corlouer

• Guillaume Corlouer 13 Apr 2021 21:47 UTC

  1 point

  0 ∶ 0

  on: Tankrede’s Shortform

  The definition of existential risk as ‘humanity losing its long term potential’ in Toby Ord precipice could be specified further. Without (perhaps) loss of generality, assuming finite total value in our universe, one could specify existential risks into two broad categories of risks such as:

  • Extinction risks (X-risks): Human share of total value goes to zero. Examples could be extinction from pandemics, extreme climate change or some natural event.

  • Agential risks (A-risks): Human share of total value could be greater than in the X-risks scenarios but keeps being strictly dominated by the share of total value holded by misaligned agents. Examples could be misaligned institutions, AIs or loud alienscontrolling most of the value in the universe and with whom there would be little gain from trade to be hoped for.

• Guillaume Corlouer 5 Mar 2022 15:37 UTC

  9 points

  0 ∶ 0

  on: The Fu­ture Fund’s Pro­ject Ideas Competition

  Funding the AI alignment institute, a Manhattan project scale for AI alignment.

  Artificial intelligence

  Aligning AI with human interests could be very hard. The current growth in AI alignment research might be insufficient to align AI. To speed up alignment research, we want to fund an ambitious institute attracting hundreds to thousands of researchers and engineers to work full-time on aligning AI. The institute would enable these researchers to work with computing resources competitive with top AI industries. We could also slow down risky AI capability research by offering top AI capability researchers competitive wages and autonomy, draining them from top AI organizations. While small specialized teams would pursue innovative alignment research, the institute would enhance their collaboration, bridging AI alignment theory, experiment, and policy. The institute could also offer alignment fellowships optimized to speed up the onboarding of bright young students in alignment research. For example, we would fund stipends and mentorships competitive with doctoral programs or entry-level jobs in the industry. The institute would be located in a place safe from global catastrophic risks and facilitate access to high-quality healthcare, food, housing, transportation to optimize researchers well being and productivity.

• Guillaume Corlouer 5 Mar 2022 20:48 UTC

  7 points

  0 ∶ 0

  on: The Fu­ture Fund’s Pro­ject Ideas Competition

  Making AI alignment research among the most lucrative career path in the world.

  AI alignment

  Having the most productive researchers in AI alignment would increase our chances to develop competitive aligned models and agents. As of now, the most lucrative careers tend to be in top AI companies. They attract many bright graduate students and researchers. We want this to change and enable AI alignment research to become the most attractive career choice for excellent junior and senior engineers and researchers. We are willing to fund AI alignment workers with wages higher than top AI companies’ standards. For example, wages could start around 250k$/​year and grow with productivity and experience.

• Guillaume Corlouer 6 Mar 2022 16:49 UTC

  4 points

  0 ∶ 0

  on: The Fu­ture Fund’s Pro­ject Ideas Competition

  Funding AI policy proposals to slow down high-risk AI capability research.

  AI alignment, AI policy

  We want AI alignment research to catch up and surpass AI capability research. Among others, AI capability research requires a friendly political environment. We would be interested in funding AI policy proposals that would increase the chance of obtaining effective regulations slowing down highly risky AI capability R&D. For example, some regulations could impose large language models to pass a thorough safety audit before deployment or scaling in parameters above determined safety thresholds. Another example would be funding AI policy projects increasing the chance of banning research aiming to build generally capable AI before solving the AI alignment problem. Such regulations would probably need to be implemented on a national and international scale to be effective.

• Guillaume Corlouer 6 Mar 2022 18:37 UTC

  13 points

  0 ∶ 0

  on: The Fu­ture Fund’s Pro­ject Ideas Competition

  Regulating AI consciousness.

  Artificial intelligence, Values and reflective process

  The probability that AIs will be capable of conscious processing in the incoming decades is not negligible. With the right information dynamics, some artificial cognitive architecture could support conscious experiences. The global neural workspace is an example of a leading theory of consciousness compatible with this view. Furthermore, if it turns out that conscious processing improves learning efficiency then building AI capable of consciousness might become an effective path toward more generally capable AI. Building conscious AIs would have crucial ethical implications given their high expected population. To decrease the chance of bad moral outcomes we could follow two broad strategies. First, we could fund policy projects aiming to work with regulators to ban or slow down research that poses a substantial risk to building conscious AI. Regulations slowing the arrival of conscious AIs could be in place until we gain more moral clarity and a solid understanding of machine consciousness. For example, philosopher Thomas Metzinger advocated a moratorium on synthetic phenomenology in a previously published paper. Second, we need to fund more research in machine consciousness and philosophy of mind improving our understanding of synthetic phenomenology in AIs and their moral status. Note that machine consciousness is currently very neglected as an academic field.

• Guillaume Corlouer 7 Mar 2022 10:28 UTC

  1 point

  0 ∶ 0

  in reply to: Chris Leong’s comment on: The Fu­ture Fund’s Pro­ject Ideas Competition

  Yes. To reduce that risk we could aim for an international agreement on banning high-risk AI capability research but might not be satisfying. I have the impression that very few people (if any) are working on that flavor of regulations and could be useful to explore it more. Ideally, if we could simply coordinate to not produce direct work on producing generally capable AI until we figure out safety it could be an important win.