Thanks for running this survey. I find these results extremely implausibly bearish on public policy—I do not think we should be even close to indifferent between improving the AI policy of the country that can make binding rules on all of the leading labs plus many key hardware inputs and has a $6 trillion budget and the most powerful military on earth by 5% and having $8.1 million more dollars for a good grantmaker, or having 32.5 “good video explainers,” or having 13 technical AI academics. I’m biased, of course, but IMO the surveyed population is massively overrating the importance of the alignment community relative to the US government.
I mostly agree with this. The counterargument I can come up with is that the best AI think tanks right now are asking for grants in the range of $2 - $5 million and seem to be pretty influential, so it’s possible that a grantmaker who got $8 million could improve policy by 5%, in which case it’s correct to equate those two.
I’m not sure how that fits with the relative technical/policy questions.
I think “5%” is just very badly defined. If I just go with the most intuitive definition to me, then 32.5 good video explainers would probably improve the AI x-risk relevant competence of the US government by more than 5% (which currently is very close to 0, and 5% of a very small number is easy to achieve).
But like, any level of clarification would probably wildly swing whatever estimates I give you. Disagreement on this question seems like it will inevitably just lead to arguing over definitions.
“Improve US AI policy 5 percentage points” was defined as
Instead of buying think tanks, this option lets you improve AI policy directly. The distribution of possible US AI policies will go from being centered on the 50th-percentile-good outcome to being centered on the 55th-percentile-good outcome, as per your personal definition of good outcomes. The variance will stay the same.
Hmm, yeah, that is better-defined. I don’t have a huge amount of variance within those percentiles, so I think I would probably take the 32.5 video explainers, but I really haven’t thought much about it.
Thanks for running this survey. I find these results extremely implausibly bearish on public policy—I do not think we should be even close to indifferent between improving the AI policy of the country that can make binding rules on all of the leading labs plus many key hardware inputs and has a $6 trillion budget and the most powerful military on earth by 5% and having $8.1 million more dollars for a good grantmaker, or having 32.5 “good video explainers,” or having 13 technical AI academics. I’m biased, of course, but IMO the surveyed population is massively overrating the importance of the alignment community relative to the US government.
I mostly agree with this. The counterargument I can come up with is that the best AI think tanks right now are asking for grants in the range of $2 - $5 million and seem to be pretty influential, so it’s possible that a grantmaker who got $8 million could improve policy by 5%, in which case it’s correct to equate those two.
I’m not sure how that fits with the relative technical/policy questions.
How are the best AI think tanks “pretty influential?”
I think “5%” is just very badly defined. If I just go with the most intuitive definition to me, then 32.5 good video explainers would probably improve the AI x-risk relevant competence of the US government by more than 5% (which currently is very close to 0, and 5% of a very small number is easy to achieve).
But like, any level of clarification would probably wildly swing whatever estimates I give you. Disagreement on this question seems like it will inevitably just lead to arguing over definitions.
“Improve US AI policy 5 percentage points” was defined as
(This is still poorly defined.)
Hmm, yeah, that is better-defined. I don’t have a huge amount of variance within those percentiles, so I think I would probably take the 32.5 video explainers, but I really haven’t thought much about it.