This looks super interesting, thanks for posting! I especially appreciate the “How to apply” section
One thing I’m interested in is seeing how this actually looks in practice—specifying real exogenous uncertainties (e.g. about timelines, takeoff speeds, etc), policy levers (e.g. these ideas, different AI safety research agendas, etc), relations (e.g. between AI labs, governments, etc) and performance metrics (e.g “p(doom)”, plus many of the sub-goals you outline). What are the conclusions? What would this imply about prioritization decisions? etc
I appreciate this would be super challenging, but if you are aware of any attempts to do it (even if using just a very basic, simplifying model), I’d be curious to hear how it’s gone
Thank you for your thoughtful comment! You’ve highlighted some of the key aspects that I believe make this kind of AI governance model potentially valuable. I too am eager to see how these concepts would play out in practice.
Firstly, on specifying real exogenous uncertainties, I believe this is indeed a crucial part of this approach. As you rightly pointed out, uncertainties around AI development timelines, takeoff speeds, and others are quite significant. A robust AI governance framework should indeed have the ability to effectively incorporate these uncertainties.
Regarding policy levers, I agree that an in-depth understanding of different AI safety research agendas is essential. In my preliminary work, I have started exploring a variety of such agendas. The goal is not only to understand these different research directions but also to identify which might offer the most promise given different (but also especially bleak) future scenarios.
In terms of relations between different entities like AI labs, governments, etc., this is another important area I’m planning to looking into. The nature of these relations can significantly impact the deployment and governance of AI, and we would need to develop models to help us better understand these dynamics.
Regarding performance metrics like p(doom), I’m very much in the early stages of defining and quantifying these. It’s quite challenging because it requires balancing a number of competing factors. Still, I’m confident that our approach will eventually enable us to develop robust metrics for assessing different policy options. An interesting notion here is that it p(doom), is quite an aggregated variable. The DMDU approach would provide us with the opportunity to have a set of (independent) metrics that we can attempt to optimize all at the same time (think Pareto-optimality here).
As to the conclusions and the implications for prioritization decisions, it’s hard to say before running optimizations or simulations, let alone before formally modeling. Given the nature of ABMs for example, we would expect emerging macro-behaviors and phenomena that arise from the defined micro-behaviors. Feedback and system dynamics are something to discover when running the model. That makes it very hard to predict what we would likely see. However, given that Epoch (and maybe others) are working on finding good policies themselves, we could include these policies into our models as well and check whether different modeling paradigms (ABMs in this case) yield similar results. Keep in mind that this would entail simply running the model with some policy inputs. There is no (multi-objective) optimization involved at this stage yet. The optimization in combination with subsets of vulnerable scenarios would add even more value to using models.
In terms of attempts to implement this approach, there is nothing out there. Previous modeling on AI governance has mostly focused on traditional modeling, e.g. (evolutionary) game theory and neoclassical macro-economic models (with e.g. Nordhaus’ DICE). At the moment, there are simple first attempts to use complexity modeling for AI governance. More could be done. Beyond the pure modeling, decision-making under deep uncertainty as a framework has not been used for such purposes yet.
What I would love to see is that there is more awareness of such methodologies, their power, and potential usefulness. Ideally, some (existing or new) organization would get some funding and pick up the challenge of creating such models and conducting the corresponding analyses. I strongly believe, this could be very useful. For example, the Odyssean Institute (which I’m a part of as well) has the intention to apply this methodology (+ more) to a wide range of problems. If particular funding would be available for AI governance application, I’m sure, they would go for it.
This looks super interesting, thanks for posting! I especially appreciate the “How to apply” section
One thing I’m interested in is seeing how this actually looks in practice—specifying real exogenous uncertainties (e.g. about timelines, takeoff speeds, etc), policy levers (e.g. these ideas, different AI safety research agendas, etc), relations (e.g. between AI labs, governments, etc) and performance metrics (e.g “p(doom)”, plus many of the sub-goals you outline). What are the conclusions? What would this imply about prioritization decisions? etc
I appreciate this would be super challenging, but if you are aware of any attempts to do it (even if using just a very basic, simplifying model), I’d be curious to hear how it’s gone
Thank you for your thoughtful comment! You’ve highlighted some of the key aspects that I believe make this kind of AI governance model potentially valuable. I too am eager to see how these concepts would play out in practice.
Firstly, on specifying real exogenous uncertainties, I believe this is indeed a crucial part of this approach. As you rightly pointed out, uncertainties around AI development timelines, takeoff speeds, and others are quite significant. A robust AI governance framework should indeed have the ability to effectively incorporate these uncertainties.
Regarding policy levers, I agree that an in-depth understanding of different AI safety research agendas is essential. In my preliminary work, I have started exploring a variety of such agendas. The goal is not only to understand these different research directions but also to identify which might offer the most promise given different (but also especially bleak) future scenarios.
In terms of relations between different entities like AI labs, governments, etc., this is another important area I’m planning to looking into. The nature of these relations can significantly impact the deployment and governance of AI, and we would need to develop models to help us better understand these dynamics.
Regarding performance metrics like p(doom), I’m very much in the early stages of defining and quantifying these. It’s quite challenging because it requires balancing a number of competing factors. Still, I’m confident that our approach will eventually enable us to develop robust metrics for assessing different policy options. An interesting notion here is that it p(doom), is quite an aggregated variable. The DMDU approach would provide us with the opportunity to have a set of (independent) metrics that we can attempt to optimize all at the same time (think Pareto-optimality here).
As to the conclusions and the implications for prioritization decisions, it’s hard to say before running optimizations or simulations, let alone before formally modeling. Given the nature of ABMs for example, we would expect emerging macro-behaviors and phenomena that arise from the defined micro-behaviors. Feedback and system dynamics are something to discover when running the model. That makes it very hard to predict what we would likely see. However, given that Epoch (and maybe others) are working on finding good policies themselves, we could include these policies into our models as well and check whether different modeling paradigms (ABMs in this case) yield similar results. Keep in mind that this would entail simply running the model with some policy inputs. There is no (multi-objective) optimization involved at this stage yet. The optimization in combination with subsets of vulnerable scenarios would add even more value to using models.
In terms of attempts to implement this approach, there is nothing out there. Previous modeling on AI governance has mostly focused on traditional modeling, e.g. (evolutionary) game theory and neoclassical macro-economic models (with e.g. Nordhaus’ DICE). At the moment, there are simple first attempts to use complexity modeling for AI governance. More could be done. Beyond the pure modeling, decision-making under deep uncertainty as a framework has not been used for such purposes yet.
What I would love to see is that there is more awareness of such methodologies, their power, and potential usefulness. Ideally, some (existing or new) organization would get some funding and pick up the challenge of creating such models and conducting the corresponding analyses. I strongly believe, this could be very useful. For example, the Odyssean Institute (which I’m a part of as well) has the intention to apply this methodology (+ more) to a wide range of problems. If particular funding would be available for AI governance application, I’m sure, they would go for it.