This looks super interesting, thanks for posting! I especially appreciate the âHow to applyâ section
One thing Iâm interested in is seeing how this actually looks in practiceâspecifying real exogenous uncertainties (e.g. about timelines, takeoff speeds, etc), policy levers (e.g. these ideas, different AI safety research agendas, etc), relations (e.g. between AI labs, governments, etc) and performance metrics (e.g âp(doom)â, plus many of the sub-goals you outline). What are the conclusions? What would this imply about prioritization decisions? etc
I appreciate this would be super challenging, but if you are aware of any attempts to do it (even if using just a very basic, simplifying model), Iâd be curious to hear how itâs gone
Thank you for your thoughtful comment! Youâve highlighted some of the key aspects that I believe make this kind of AI governance model potentially valuable. I too am eager to see how these concepts would play out in practice.
Firstly, on specifying real exogenous uncertainties, I believe this is indeed a crucial part of this approach. As you rightly pointed out, uncertainties around AI development timelines, takeoff speeds, and others are quite significant. A robust AI governance framework should indeed have the ability to effectively incorporate these uncertainties.
Regarding policy levers, I agree that an in-depth understanding of different AI safety research agendas is essential. In my preliminary work, I have started exploring a variety of such agendas. The goal is not only to understand these different research directions but also to identify which might offer the most promise given different (but also especially bleak) future scenarios.
In terms of relations between different entities like AI labs, governments, etc., this is another important area Iâm planning to looking into. The nature of these relations can significantly impact the deployment and governance of AI, and we would need to develop models to help us better understand these dynamics.
Regarding performance metrics like p(doom), Iâm very much in the early stages of defining and quantifying these. Itâs quite challenging because it requires balancing a number of competing factors. Still, Iâm confident that our approach will eventually enable us to develop robust metrics for assessing different policy options. An interesting notion here is that it p(doom), is quite an aggregated variable. The DMDU approach would provide us with the opportunity to have a set of (independent) metrics that we can attempt to optimize all at the same time (think Pareto-optimality here).
As to the conclusions and the implications for prioritization decisions, itâs hard to say before running optimizations or simulations, let alone before formally modeling. Given the nature of ABMs for example, we would expect emerging macro-behaviors and phenomena that arise from the defined micro-behaviors. Feedback and system dynamics are something to discover when running the model. That makes it very hard to predict what we would likely see. However, given that Epoch (and maybe others) are working on finding good policies themselves, we could include these policies into our models as well and check whether different modeling paradigms (ABMs in this case) yield similar results. Keep in mind that this would entail simply running the model with some policy inputs. There is no (multi-objective) optimization involved at this stage yet. The optimization in combination with subsets of vulnerable scenarios would add even more value to using models.
In terms of attempts to implement this approach, there is nothing out there. Previous modeling on AI governance has mostly focused on traditional modeling, e.g. (evolutionary) game theory and neoclassical macro-economic models (with e.g. Nordhausâ DICE). At the moment, there are simple first attempts to use complexity modeling for AI governance. More could be done. Beyond the pure modeling, decision-making under deep uncertainty as a framework has not been used for such purposes yet.
What I would love to see is that there is more awareness of such methodologies, their power, and potential usefulness. Ideally, some (existing or new) organization would get some funding and pick up the challenge of creating such models and conducting the corresponding analyses. I strongly believe, this could be very useful.
This looks super interesting, thanks for posting! I especially appreciate the âHow to applyâ section
One thing Iâm interested in is seeing how this actually looks in practiceâspecifying real exogenous uncertainties (e.g. about timelines, takeoff speeds, etc), policy levers (e.g. these ideas, different AI safety research agendas, etc), relations (e.g. between AI labs, governments, etc) and performance metrics (e.g âp(doom)â, plus many of the sub-goals you outline). What are the conclusions? What would this imply about prioritization decisions? etc
I appreciate this would be super challenging, but if you are aware of any attempts to do it (even if using just a very basic, simplifying model), Iâd be curious to hear how itâs gone
Thank you for your thoughtful comment! Youâve highlighted some of the key aspects that I believe make this kind of AI governance model potentially valuable. I too am eager to see how these concepts would play out in practice.
Firstly, on specifying real exogenous uncertainties, I believe this is indeed a crucial part of this approach. As you rightly pointed out, uncertainties around AI development timelines, takeoff speeds, and others are quite significant. A robust AI governance framework should indeed have the ability to effectively incorporate these uncertainties.
Regarding policy levers, I agree that an in-depth understanding of different AI safety research agendas is essential. In my preliminary work, I have started exploring a variety of such agendas. The goal is not only to understand these different research directions but also to identify which might offer the most promise given different (but also especially bleak) future scenarios.
In terms of relations between different entities like AI labs, governments, etc., this is another important area Iâm planning to looking into. The nature of these relations can significantly impact the deployment and governance of AI, and we would need to develop models to help us better understand these dynamics.
Regarding performance metrics like p(doom), Iâm very much in the early stages of defining and quantifying these. Itâs quite challenging because it requires balancing a number of competing factors. Still, Iâm confident that our approach will eventually enable us to develop robust metrics for assessing different policy options. An interesting notion here is that it p(doom), is quite an aggregated variable. The DMDU approach would provide us with the opportunity to have a set of (independent) metrics that we can attempt to optimize all at the same time (think Pareto-optimality here).
As to the conclusions and the implications for prioritization decisions, itâs hard to say before running optimizations or simulations, let alone before formally modeling. Given the nature of ABMs for example, we would expect emerging macro-behaviors and phenomena that arise from the defined micro-behaviors. Feedback and system dynamics are something to discover when running the model. That makes it very hard to predict what we would likely see. However, given that Epoch (and maybe others) are working on finding good policies themselves, we could include these policies into our models as well and check whether different modeling paradigms (ABMs in this case) yield similar results. Keep in mind that this would entail simply running the model with some policy inputs. There is no (multi-objective) optimization involved at this stage yet. The optimization in combination with subsets of vulnerable scenarios would add even more value to using models.
In terms of attempts to implement this approach, there is nothing out there. Previous modeling on AI governance has mostly focused on traditional modeling, e.g. (evolutionary) game theory and neoclassical macro-economic models (with e.g. Nordhausâ DICE). At the moment, there are simple first attempts to use complexity modeling for AI governance. More could be done. Beyond the pure modeling, decision-making under deep uncertainty as a framework has not been used for such purposes yet.
What I would love to see is that there is more awareness of such methodologies, their power, and potential usefulness. Ideally, some (existing or new) organization would get some funding and pick up the challenge of creating such models and conducting the corresponding analyses. I strongly believe, this could be very useful.