Skimmed and this looks like a good list, thanks for writing this up and sharing! I also appreciate you sharing your reasoning.
Minor question: I’m curious about your emphasis on statistical work, particularly wrt AI regulation and standards. I think mostly I’m unsure that the additional time and difficulty of doing robust statistical work would be worth it, relative to the current case study/best guess/experts’ estimates approach. What am I missing?
I think my case for more focus on good statistical work when looking at governance is that when doing good statistical work on interventions, we often find very high degrees of variation in effect sizes that are enough to justify the extra work of the intervention. I’m personally very unsure of the effect size of changes in liability law, soft law, and various corporate governance interventions on accident rates.
There’s been lots of great case study/best guess/expert consensus work on these questions which I think is often great and I’m very happy exists, but leaves me with large uncertainties about effect sizes, and so on the current margin, I want more good statistical work.
I think the case for good statistical work on areas like degree of misuse risk and forecasting is stronger because I think that people’s takes on these questions are pretty grounded in quite theoretical arguments (unlike governance interventions which are much more empirically grounded) that I think would benefit a lot from more grounded statistical work.
Skimmed and this looks like a good list, thanks for writing this up and sharing! I also appreciate you sharing your reasoning.
Minor question: I’m curious about your emphasis on statistical work, particularly wrt AI regulation and standards. I think mostly I’m unsure that the additional time and difficulty of doing robust statistical work would be worth it, relative to the current case study/best guess/experts’ estimates approach. What am I missing?
Thanks for your comment Ceb.
I think my case for more focus on good statistical work when looking at governance is that when doing good statistical work on interventions, we often find very high degrees of variation in effect sizes that are enough to justify the extra work of the intervention. I’m personally very unsure of the effect size of changes in liability law, soft law, and various corporate governance interventions on accident rates.
There’s been lots of great case study/best guess/expert consensus work on these questions which I think is often great and I’m very happy exists, but leaves me with large uncertainties about effect sizes, and so on the current margin, I want more good statistical work.
I think the case for good statistical work on areas like degree of misuse risk and forecasting is stronger because I think that people’s takes on these questions are pretty grounded in quite theoretical arguments (unlike governance interventions which are much more empirically grounded) that I think would benefit a lot from more grounded statistical work.