Acquire and repurpose new AI startups for AI safety
Artificial intelligence
As ML performance has recently improved there is a new wave of startups coming. Some are composed of top talent, carefully engineered infrastructure, a promising product, well-coordinated teams, with existing workflows and management capacity. All of these are bottlenecks for AI safety R&D.
It should be possible to acquire some appropriate startups and middle-sized companies. Examples include HuggingFace, AI21, Cohere, and smaller, newer startups. The idea is to repurpose the mission of some select companies to align them more closely with socially beneficial and safety-oriented R&D. This is sometimes feasible since their missions are often broad, still in flux, and their product could benefit from improving safety and alignment.
Trying this could have very high information value. If it works, it has enormous potential upside as many new AI startups are being created now that could be acquired in the future. It could potentially more than double the size of the AI alignment R&D.
Paying existing employees to do safety R&D seems easier than paying academics. Academics often like to follow their own ideas but employees are already doing what their superior tells them to. In fact, they may find alignment and safety R&D more motivating than their company’s existing mission. Additionally, some founders may be more willing to sell to a non-profit org with a social-good mission than to Big Tech.
Big tech companies acquire small companies all the time. The reasons for this vary (e.g. killing competition), but overall it suggests that it can be feasible and even profitable.
Caveats:
1) A highly qualified replacement may be needed for the top-level management.
2) Some employees may leave after an acquisition. This seems more likely if the pivot towards safety is a big change to the skills and workflows. Or if the employees don’t like the new mission. It seems possible to partially avoid both of these by acquiring the right companies and steering them towards a mission that is relevant to their existing work. For example, natural language generation startups would usually benefit from fine-tuning their models with alignment techniques.
Thanks, I think that’s a really interesting and potentially great idea. I’d encourage you to post it as a short stand-alone post, I’d be interested in hearing other people’s thoughts.
Acquire and repurpose new AI startups for AI safety
Artificial intelligence
As ML performance has recently improved there is a new wave of startups coming. Some are composed of top talent, carefully engineered infrastructure, a promising product, well-coordinated teams, with existing workflows and management capacity. All of these are bottlenecks for AI safety R&D.
It should be possible to acquire some appropriate startups and middle-sized companies. Examples include HuggingFace, AI21, Cohere, and smaller, newer startups. The idea is to repurpose the mission of some select companies to align them more closely with socially beneficial and safety-oriented R&D. This is sometimes feasible since their missions are often broad, still in flux, and their product could benefit from improving safety and alignment.
Trying this could have very high information value. If it works, it has enormous potential upside as many new AI startups are being created now that could be acquired in the future. It could potentially more than double the size of the AI alignment R&D.
Paying existing employees to do safety R&D seems easier than paying academics. Academics often like to follow their own ideas but employees are already doing what their superior tells them to. In fact, they may find alignment and safety R&D more motivating than their company’s existing mission. Additionally, some founders may be more willing to sell to a non-profit org with a social-good mission than to Big Tech.
Big tech companies acquire small companies all the time. The reasons for this vary (e.g. killing competition), but overall it suggests that it can be feasible and even profitable.
Caveats:
1) A highly qualified replacement may be needed for the top-level management.
2) Some employees may leave after an acquisition. This seems more likely if the pivot towards safety is a big change to the skills and workflows. Or if the employees don’t like the new mission. It seems possible to partially avoid both of these by acquiring the right companies and steering them towards a mission that is relevant to their existing work. For example, natural language generation startups would usually benefit from fine-tuning their models with alignment techniques.
Thanks, I think that’s a really interesting and potentially great idea. I’d encourage you to post it as a short stand-alone post, I’d be interested in hearing other people’s thoughts.