Infrastructure: Tooling, mentorship, training, or legal support for researchers.
New AI Safety Organizations: New labs or fellowship programs.
Advocacy Organizations: Raising awareness about the field.
Where would, for example, insurance for AI products fit in this? This is a for-profit idea that creates a natural business incentive to understand & research risks from AI products at a very granular level, and if it succeeds, it puts you into position to influence the entire industry (e.g. “we will lower your premiums if you implement safety measure X”).
I agree that if you restrict yourself to either supporting AIS researchers, launching field-building projects or research labs, or doing advocacy, then you will in fact not find good startup ideas, for the structural reasons you do a good job of listing in your post, as well as the fact that these are all things people are already doing.
METR is a very good AIS org. In addition to just being really solid and competent, a lot of why they succeeded was that they started doing something that few people were thinking about at the time. Everyone and their dog is launching an evals startup today, but the real value is finding ideas like METR before they are widespread. If the startup ideas you consider are all about doing the same thing that existing orgs do, you will miss out on the most important ones.
I do agree that the intersection of impact & profit & bootstrappability is small and hard to hit, and there’s no law of nature that says something should definitely exist there. But something exists in that corner, it will be a novel type of thing.
You discuss three types of AI safety ventures:
Where would, for example, insurance for AI products fit in this? This is a for-profit idea that creates a natural business incentive to understand & research risks from AI products at a very granular level, and if it succeeds, it puts you into position to influence the entire industry (e.g. “we will lower your premiums if you implement safety measure X”).
I agree that if you restrict yourself to either supporting AIS researchers, launching field-building projects or research labs, or doing advocacy, then you will in fact not find good startup ideas, for the structural reasons you do a good job of listing in your post, as well as the fact that these are all things people are already doing.
METR is a very good AIS org. In addition to just being really solid and competent, a lot of why they succeeded was that they started doing something that few people were thinking about at the time. Everyone and their dog is launching an evals startup today, but the real value is finding ideas like METR before they are widespread. If the startup ideas you consider are all about doing the same thing that existing orgs do, you will miss out on the most important ones.
I do agree that the intersection of impact & profit & bootstrappability is small and hard to hit, and there’s no law of nature that says something should definitely exist there. But something exists in that corner, it will be a novel type of thing.
(reposted from a Slack thread)