Thanks for the clarification!
ac
Karma: 118
Survey on AI existential risk scenarios
Some AI Governance Research Ideas
Hey Cullen, I’m a bit confused about why you should assume that you would do traditional lawyering for so long. The fact it’s the modal path could just mean most law-schoolers want to be lawyers—which seems probably true. So maybe if you want to do policy, you can jump in straight away.
Hi Jia, I don’t know much about policy diffusion, unfortunately. From the brief reading I have done, both seem to occur. Learning from other governments appears to be important; according to some researchers, exchange via membership in international organisations is a central causal factor. “Intentional” governments could make active engagement in these a higher priority.
Thanks for this Tyler!
The references are great, I wasn’t aware of them. Re the first, how exactly do you think low institutional path-dependence and institutional innovativeness interact? They seem like related but distinct concepts to me.
I agree that it would be great to see more research on those questions, though I wonder if a thorough review of the policy diffusion literature might be sufficient. I definitely would like a clearer characterization of governmental innovativeness; I felt kind of hand-wavey in this post.
Longtermist reasons to work for innovative governments
When to use quantitative vs qualitative research
Without a framework for thinking about this, I’m often unsure what I should be learning from qualitative studies, and I don’t always know when it makes sense to conduct them. (This seems related to the debate between cleometricians and counterfactual narrative historians; some discussion here, page 18)
I doubt that there is any one answer re the marginal value of such projects, because the value depends on what is being governed. For instance, I think a successful implementation of regulatory markets for AI safety would be very valuable, but regulatory markets for corporate law wouldn’t be; yet the same basic framework is being implemented.
For this reason, I’d be more interested in analysis of governance innovation for a particular cause area.
I think that’s mostly right, with a couple of caveats:
You only mentioned non-profits, but I think most of this applies to other longtermists organizations with pretty illegible missions. Maybe Anthropic is an example.
Some organizations with longtermists missions should not aim to maximise something particularly illegible. In these cases, entrepreneurialism will often be very important, including in highly autonomous roles. For example, some biosecurity organization could be trying to design and produce, at very large scales, “Super PPE”, such as masks, engineered with extreme events in mind.
Like SpaceX, which initially aimed to significantly reduce the cost, and improve the supply, of routine space flight, the Super PPE project would need to improve upon existing PPE designed for use in extreme events, which is “ bulky, highly restrictive, and insufficiently abundant”. (Alvea might be another example, but I don’t know enough about them).
This suggests a division of labour where project missions are defined by individuals outside the organization, as with Super PPE, before being executed by others, who are high on entrepreneurialism. Note that, in hiring for leadership roles in the organization, this will mean placing more weight on entrepreneurialism than on self-skepticism and reflectiveness. While Musk did a poor job defining SpaceX’s mission, he did an excellent job executing it.
This seems true. It also suggests that if you can be extremely high on both traits, you’ll bring significant counterfactual value.