ac

Longter­mist rea­sons to work for in­no­va­tive governments

What can the prin­ci­pal-agent liter­a­ture tell us about AI risk?

• ac 29 May 2022 10:03 UTC

  5 points

  0 ∶ 0

  in reply to: bgarfinkel’s comment on: Ben Garfinkel’s Shortform

  I think that’s mostly right, with a couple of caveats:

  • You only mentioned non-profits, but I think most of this applies to other longtermists organizations with pretty illegible missions. Maybe Anthropic is an example.

  • Some organizations with longtermists missions should not aim to maximise something particularly illegible. In these cases, entrepreneurialism will often be very important, including in highly autonomous roles. For example, some biosecurity organization could be trying to design and produce, at very large scales, “Super PPE”, such as masks, engineered with extreme events in mind.

    • Like SpaceX, which initially aimed to significantly reduce the cost, and improve the supply, of routine space flight, the Super PPE project would need to improve upon existing PPE designed for use in extreme events, which is “ bulky, highly restrictive, and insufficiently abundant”. (Alvea might be another example, but I don’t know enough about them).

    • This suggests a division of labour where project missions are defined by individuals outside the organization, as with Super PPE, before being executed by others, who are high on entrepreneurialism. Note that, in hiring for leadership roles in the organization, this will mean placing more weight on entrepreneurialism than on self-skepticism and reflectiveness. While Musk did a poor job defining SpaceX’s mission, he did an excellent job executing it.

  Ultimately, outside of earning-to-give ventures, we probably shouldn’t expect the longtermist community (or at least the best version of it) to house many extremely entrepreneurial people. There will be occasional leaders who are extremely high on both entrepreneurialism and reflectiveness (I can currently think of at least a couple); however, since these two traits don’t seem to be strongly correlated, this will probably only happen pretty rarely.

  
  This seems true. It also suggests that if you can be extremely high on both traits, you’ll bring significant counterfactual value.

• ac 15 Oct 2020 10:47 UTC

  1 point

  0 ∶ 0

  in reply to: tylermjohn’s comment on: Longter­mist rea­sons to work for in­no­va­tive governments

  Thanks for this Tyler!

  The references are great, I wasn’t aware of them. Re the first, how exactly do you think low institutional path-dependence and institutional innovativeness interact? They seem like related but distinct concepts to me.

  I agree that it would be great to see more research on those questions, though I wonder if a thorough review of the policy diffusion literature might be sufficient. I definitely would like a clearer characterization of governmental innovativeness; I felt kind of hand-wavey in this post.

• ac 15 Oct 2020 11:53 UTC

  0 points

  0 ∶ 0

  in reply to: jia’s comment on: Longter­mist rea­sons to work for in­no­va­tive governments

  Hi Jia, I don’t know much about policy diffusion, unfortunately. From the brief reading I have done, both seem to occur. Learning from other governments appears to be important; according to some researchers, exchange via membership in international organisations is a central causal factor. “Intentional” governments could make active engagement in these a higher priority.