There are a lot of things I’m uncertain about, but I should say that I expect most research aimed at resolving these uncertainties not to provide strong enough evidence to change my funding decisions (though some research definitely could!) I do think weaker evidence could change my decisions if we had a larger number of high-quality applications to choose from. On the current margin, I’d be more excited about research aimed at identifying new interventions that could be promising.
Here’s a small sample of the things that feel particularly relevant to grants I’ve considered recently. I’m not sure if I would say these are the most crucial:
What sources of existential risk are plausible?
If I thought that AI capabilities were perfectly entangled with their ability to learn human preferences, I would be unlikely to fund AI alignment work.
If I thought institutional incentives were such that people wouldn’t create AI systems that could be existentially threatening without taking maximal precautions, I would be unlikely to fund AI risk work at all.
If I thought our lightcone was overwhelmingly likely to be settled by another intelligent species similar to us, I would be unlikely to fund existential risk mitigation outside of AI.
What kind of movement-building work is effective?
Adam writes above how he thinks movement-building work that sacrifices quality for quantity is unlikely to be good. I agree with him, but I could be wrong about that. If I changed my mind here, I’d be more likely to fund a larger number of movement-building projects.
It seems possible to me that work that’s explicitly labeled as ‘movement-building’ is generally not as effective for movement-building as high-quality direct work, and could even be net-negative. If I decided this was true, I’d be less likely to fund movement-building projects at all.
What strands of AI safety work are likely to be useful?
I currently take a fairly unopinionated approach to funding AI safety work—I feel willing to fund anything that I think a sufficiently large subset of smart researchers would think is promising. I can imagine becoming more opinionated here, and being less likely to fund certain kinds of work.
If I believed that it was certain that very advanced AI systems were coming soon and would look like large neural networks, I would be unlikely to fund speculative work focused on alternate paths to AGI.
If I believed that AI systems were overwhelmingly unlikely to look like large neural networks, this would have some effect on my funding decisions, but I’d have to think more about the value of near-term work from an AI safety field-building perspective.
Edit: I really like Adam’s answer
There are a lot of things I’m uncertain about, but I should say that I expect most research aimed at resolving these uncertainties not to provide strong enough evidence to change my funding decisions (though some research definitely could!) I do think weaker evidence could change my decisions if we had a larger number of high-quality applications to choose from. On the current margin, I’d be more excited about research aimed at identifying new interventions that could be promising.
Here’s a small sample of the things that feel particularly relevant to grants I’ve considered recently. I’m not sure if I would say these are the most crucial:
What sources of existential risk are plausible?
If I thought that AI capabilities were perfectly entangled with their ability to learn human preferences, I would be unlikely to fund AI alignment work.
If I thought institutional incentives were such that people wouldn’t create AI systems that could be existentially threatening without taking maximal precautions, I would be unlikely to fund AI risk work at all.
If I thought our lightcone was overwhelmingly likely to be settled by another intelligent species similar to us, I would be unlikely to fund existential risk mitigation outside of AI.
What kind of movement-building work is effective?
Adam writes above how he thinks movement-building work that sacrifices quality for quantity is unlikely to be good. I agree with him, but I could be wrong about that. If I changed my mind here, I’d be more likely to fund a larger number of movement-building projects.
It seems possible to me that work that’s explicitly labeled as ‘movement-building’ is generally not as effective for movement-building as high-quality direct work, and could even be net-negative. If I decided this was true, I’d be less likely to fund movement-building projects at all.
What strands of AI safety work are likely to be useful?
I currently take a fairly unopinionated approach to funding AI safety work—I feel willing to fund anything that I think a sufficiently large subset of smart researchers would think is promising. I can imagine becoming more opinionated here, and being less likely to fund certain kinds of work.
If I believed that it was certain that very advanced AI systems were coming soon and would look like large neural networks, I would be unlikely to fund speculative work focused on alternate paths to AGI.
If I believed that AI systems were overwhelmingly unlikely to look like large neural networks, this would have some effect on my funding decisions, but I’d have to think more about the value of near-term work from an AI safety field-building perspective.