TL;DR: At least in my experience, AISC was pretty positive for most participants I know and it’s incredibly cheap. It also serves a clear niche that other programs are not filling and it feels reasonable to me to continue the program.
I’ve been a participant in the 2021⁄22 edition. Some thoughts that might make it easier to decide for funders/donors. 1. Impact-per-dollar is probably pretty good for the AISC. It’s incredibly cheap compared to most other AI field-building efforts and scalable. 2. I learned a bunch during AISC and I did enjoy it. It influenced my decision to go deeper into AI safety. It was less impactful than e.g. MATS for me but MATS is a full-time in-person program, so that’s not surprising. 3. AISC fills a couple of important niches in the AI safety ecosystem in my opinion. It’s online and part-time which makes it much easier to join for many people, it implies a much lower commitment which is good for people who want to find out whether they’re a good fit for AIS. It’s also much cheaper than flying everyone to the Bay or London. This also makes it more scalable because the only bottleneck is mentoring capacity without physical constraints. 4. I think AISC is especially good for people who want to test their fit but who are not super experienced yet. This seems like an important function. MATS and ARENA, for example, feel like they target people a bit deeper into the funnel with more experience who are already more certain that they are a good fit. 5. Overall, I think AISC is less impactful than e.g. MATS even without normalizing for participants. Nevertheless, AISC is probably about ~50x cheaper than MATS. So when taking cost into account, it feels clearly impactful enough to continue the project. I think the resulting projects are lower quality but the people are also more junior, so it feels more like an early educational program than e.g. MATS. 6. I have a hard time seeing how the program could be net negative unless something drastically changed since my cohort. In the worst case, people realize that they don’t like one particular type of AI safety research. But since you chat with others who are curious about AIS regularly, it will be much easier to start something that might be more meaningful. Also, this can happen in any field-building program, not just AISC. 7. Caveat: I have done no additional research on this. Maybe others know details that I’m unaware of. See this as my personal opinion and not a detailed research analysis.
Nevertheless, AISC is probably about ~50x cheaper than MATS
~50x is a big difference, and I notice the post says:
We commissioned Arb Research to do an impact assessment. One preliminary result is that AISC creates one new AI safety researcher per around $12k-$30k USD of funding.
Multiplying that number (which I’m agnostic about) by 50 gives $600k-$1.5M USD. Does your ~50x still seem accurate in light of this?
I’m guessing that what Marius means by “AISC is probably about ~50x cheaper than MATS” is that AISC is probably ~50x cheaper per participant than MATS.
Our cost per participant is $0.6k - $3k USD
50 times this would be 30k − 150k per participant. I’m guessing that MATS is around 50k per person (including stipends).
Here’s where the $12k-$30k USD comes from:
Dollar cost per new researcher produced by AISC
The organizers have proposed $60–300K per year in expenses.
The number of non-RL participants of programs have increased from 32 (AISC4) to 130 (AISC9). Let’s assume roughly 100 participants in the program per year given the proposed size of new camps.
Researchers are produced at a rate of 5–10%.
Optimistic estimate: $60K / (10% * 100) = $6K per new researcher
Middle estimate 1: $60K / (5% * 100) = $12K per new researcher
Middle estimate 2: $300K / (10% * 100) = $30K per new researcher
Pessimistic estimate: $300K / (5% * 100) = $60K per new researcher
I’m curious if you or the other past participants you know had a good experience with AISC are in a position to help fill the funding gap AISC currently has. Even if you (collectively) can’t fully fund the gap, I’d see that as a pretty strong signal that AISC is worth funding. Or, if you do donate but you prefer other giving opportunities instead (whether in AIS or other cause areas) I’d find that valuable to know too.
I was the private donor who gave €5K. My reaction to hearing that AISC was not getting funding was that this seemed insane. The iteration I was in two years ago was fantastic for me, and the research project I got started on there is basically still continuing at Apollo now. Without AISC, I think there’s a good chance I would never have become an AI notkilleveryoneism researcher.
5. Overall, I think AISC is less impactful than e.g. MATS even without normalizing for participants. Nevertheless, AISC is probably about ~50x cheaper than MATS. So when taking cost into account, it feels clearly impactful enough to continue the project. I think the resulting projects are lower quality but the people are also more junior, so it feels more like an early educational program than e.g. MATS.
This seems correct to me. MATS is investing a lot in few people. AISC is investing a little in many people.
TL;DR: At least in my experience, AISC was pretty positive for most participants I know and it’s incredibly cheap. It also serves a clear niche that other programs are not filling and it feels reasonable to me to continue the program.
I’ve been a participant in the 2021⁄22 edition. Some thoughts that might make it easier to decide for funders/donors.
1. Impact-per-dollar is probably pretty good for the AISC. It’s incredibly cheap compared to most other AI field-building efforts and scalable.
2. I learned a bunch during AISC and I did enjoy it. It influenced my decision to go deeper into AI safety. It was less impactful than e.g. MATS for me but MATS is a full-time in-person program, so that’s not surprising.
3. AISC fills a couple of important niches in the AI safety ecosystem in my opinion. It’s online and part-time which makes it much easier to join for many people, it implies a much lower commitment which is good for people who want to find out whether they’re a good fit for AIS. It’s also much cheaper than flying everyone to the Bay or London. This also makes it more scalable because the only bottleneck is mentoring capacity without physical constraints.
4. I think AISC is especially good for people who want to test their fit but who are not super experienced yet. This seems like an important function. MATS and ARENA, for example, feel like they target people a bit deeper into the funnel with more experience who are already more certain that they are a good fit.
5. Overall, I think AISC is less impactful than e.g. MATS even without normalizing for participants. Nevertheless, AISC is probably about ~50x cheaper than MATS. So when taking cost into account, it feels clearly impactful enough to continue the project. I think the resulting projects are lower quality but the people are also more junior, so it feels more like an early educational program than e.g. MATS.
6. I have a hard time seeing how the program could be net negative unless something drastically changed since my cohort. In the worst case, people realize that they don’t like one particular type of AI safety research. But since you chat with others who are curious about AIS regularly, it will be much easier to start something that might be more meaningful. Also, this can happen in any field-building program, not just AISC.
7. Caveat: I have done no additional research on this. Maybe others know details that I’m unaware of. See this as my personal opinion and not a detailed research analysis.
~50x is a big difference, and I notice the post says:
Multiplying that number (which I’m agnostic about) by 50 gives $600k-$1.5M USD. Does your ~50x still seem accurate in light of this?
I’m guessing that what Marius means by “AISC is probably about ~50x cheaper than MATS” is that AISC is probably ~50x cheaper per participant than MATS.
Our cost per participant is $0.6k - $3k USD
50 times this would be 30k − 150k per participant.
I’m guessing that MATS is around 50k per person (including stipends).
Here’s where the $12k-$30k USD comes from:
I’m curious if you or the other past participants you know had a good experience with AISC are in a position to help fill the funding gap AISC currently has. Even if you (collectively) can’t fully fund the gap, I’d see that as a pretty strong signal that AISC is worth funding. Or, if you do donate but you prefer other giving opportunities instead (whether in AIS or other cause areas) I’d find that valuable to know too.
From Lucius Bushnaq:
Full comment here: This might be the last AI Safety Camp — LessWrong
This seems correct to me. MATS is investing a lot in few people. AISC is investing a little in many people.
Also agreement on all the other points.