Thanks for posting your attempt! Yeah, it does seem like you ran into some of those issues in your attempt, and it’s useful information to know that this task is very hard. I guess one lesson here is that we probably won’t be able to build perfect institutions on the first try, even in safety-critical cases like AGI governance.
Just stumbled upon this post—I like the general vein in which you’re thinking. Not sure if you’re aware of it already, but this post by Paul Christiano addresses the “inevitable dangerous technology” argument as it relates to AI alignment.
- “First-principles design is intractable and misses important situation-specific details”—This could easily be true, I don’t have a strong opinion on it, just intutions.
I think this objection is pretty compelling. The specific tools that an institution can use to ensure that a technology is deployed safely will ultimately depend on the nature of that technology itself, its accessibility/difficulty of replication, the political/economic systems it’s integrated into, and the incentives surrounding its deployment. (Not an exhaustive list.)
Usually, any type of regulation or “responsible power-wielding” comes with tradeoffs (to freedom, efficiency, equitability, etc.), and it’ll be hard to assess whether these accepting these tradeoffs is prudent without a specific technology in mind.
That said, I think it can still be a worthwhile exercise to think about how we can build governance practices that are robust to worst-case scenarios for all of the above. I can imagine some useful insights coming out of that kind of exercise!
Thanks for posting your attempt! Yeah, it does seem like you ran into some of those issues in your attempt, and it’s useful information to know that this task is very hard. I guess one lesson here is that we probably won’t be able to build perfect institutions on the first try, even in safety-critical cases like AGI governance.
Just stumbled upon this post—I like the general vein in which you’re thinking. Not sure if you’re aware of it already, but this post by Paul Christiano addresses the “inevitable dangerous technology” argument as it relates to AI alignment.
I think this objection is pretty compelling. The specific tools that an institution can use to ensure that a technology is deployed safely will ultimately depend on the nature of that technology itself, its accessibility/difficulty of replication, the political/economic systems it’s integrated into, and the incentives surrounding its deployment. (Not an exhaustive list.)
Usually, any type of regulation or “responsible power-wielding” comes with tradeoffs (to freedom, efficiency, equitability, etc.), and it’ll be hard to assess whether these accepting these tradeoffs is prudent without a specific technology in mind.
That said, I think it can still be a worthwhile exercise to think about how we can build governance practices that are robust to worst-case scenarios for all of the above. I can imagine some useful insights coming out of that kind of exercise!