I’d like to throw my hat in the ring and indicate that I’d at least find it very interesting to take over for you to ensure that QURI’s mission continues! I’m currently trying to get back into the AI safety grantmaking space, but that’ll most likely fail, in which case I would welcome a plan B.
I imagine that the grantmaking bottleneck is overblown – that a 100–1000x increase in grantmaking capacity is easily achievable through hiring obvious candidates (10x) and streamlining the processes through retroactive funding (10–100x). If the funds actually end up doing that, it’ll be better again to contribute as a charity entrepreneur, and the plan B would become a plan A.
I’d prefer to expand QURI to projects that have less to do with quantification and more with ranking and clustering, and to adopt more of an incubator-like approach where successful projects turn into spin-offs with their own legal structure over time to introduce more resilience through redundancy. (And more Python.)
That’ll probably require about $1m in funding over ~2 years. Is that realistic? Also I’m not sure if it clashes with your vision?
Quickly (and to give context to others reading this):
I think that ‘taking over QURI’ makes the most sense for people who want to take over Guesstimate/Squiggle, and even then I’d default to being hesitant about that.
QURI doesn’t really have much to take over, especially for research. There’s a simple brand, there’s a niche community that knows its history. There aren’t staff or much IP at all. It’s unclear to me what the value is of doing related work ‘inside QURI’ vs. making a new org.
“$1m in funding over ~2 years” → I’d personally be excited to see what you do with this, but I’m unsure how easy it would be to raise this. In our experience fundraising was definitely not trivial, and we’ve mostly been asking for much less than $1m (though year by year). Most EA funders now are focused on other dedicated areas like technical AI safety or bio risk, I think our category of work has been more challenging. For a while the FTX Future Fund was interested in these areas, then later Coefficient Giving had a Forecasting fund, but both closed fairly quickly. It’s quite possible that Anthropic money and similar will change the situation, but I haven’t seen that yet. Related, I’d recommend very much deeply understanding the funding climate before making an incubator, as I think it’s easy to create a bunch of orgs that basically have no where to go.
In terms of incubators, some reading this might be interested in Surplus! It’s for-profit only, but definitely related to our work.
If you want a 501(c)(3) sponsor, I think there’s a lot of good options. QURI was itself sponsored by Rethink Priorities for much of its life.
I think everything hinges on the funding unfortunately…
Most of the projects on my list require some $200–500k in the first year to get started, and then can scale to a few million per year over time. The large-scale retrofunding needs to start higher – $10m might work, $100m works for the XPrize, $1b could be the goal.
The natural starting point is the incubator itself, which falls into the $200–500k range, but more towards the upper end to provide seed funding for the incubated projects.
Why did Guesstimate/Squiggle as for-profit not work out?
I’d love to have a call and catch up in any case! I’m curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors).
“Why did Guesstimate/Squiggle as for-profit not work out?” I’ve been asked this question a bunch of times before, happy to give some quick thoughts here. Most of my answer is here.
If it were the case that it seemed very easy to make these as startups, and ideally in which there were some clear cofounders who could handle the business side, I think the for-profit side would have been more promising.
In my case: 1. I realized this was a niche/small space, that getting a large market would likely mean having to pivot. 2. I wasn’t sure what might take off. I think Squiggle has a lot going for it, but am not sure if the value proposition is strong enough for a large company now, especially as LLMs have been getting much more powerful (changing the trade-offs around decision tools). 3. My main goal was helping effective altruist / rationalist researchers, and I suspected that focusing much more on commercialization would have hurt that. 4. Related to (3), I just found myself much more motivated to go after altruists/nonprofits than to go after standard enterprise deals
“I’d love to have a call and catch up in any case! I’m curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors).”
Quickly: 1. I’ll schedule a call sometime, once I understand my schedule a bit better. 2. I have incredibly little info on DeepMind right now. As you may imagine, personally I’m excited about a lot of these areas, but I can’t speak for DeepMind.
I’d like to throw my hat in the ring and indicate that I’d at least find it very interesting to take over for you to ensure that QURI’s mission continues! I’m currently trying to get back into the AI safety grantmaking space, but that’ll most likely fail, in which case I would welcome a plan B.
I imagine that the grantmaking bottleneck is overblown – that a 100–1000x increase in grantmaking capacity is easily achievable through hiring obvious candidates (10x) and streamlining the processes through retroactive funding (10–100x). If the funds actually end up doing that, it’ll be better again to contribute as a charity entrepreneur, and the plan B would become a plan A.
I’d prefer to expand QURI to projects that have less to do with quantification and more with ranking and clustering, and to adopt more of an incubator-like approach where successful projects turn into spin-offs with their own legal structure over time to introduce more resilience through redundancy. (And more Python.)
That’ll probably require about $1m in funding over ~2 years. Is that realistic? Also I’m not sure if it clashes with your vision?
Hi Dawn!
I’d be happy to discuss this.
Quickly (and to give context to others reading this):
I think that ‘taking over QURI’ makes the most sense for people who want to take over Guesstimate/Squiggle, and even then I’d default to being hesitant about that.
QURI doesn’t really have much to take over, especially for research. There’s a simple brand, there’s a niche community that knows its history. There aren’t staff or much IP at all. It’s unclear to me what the value is of doing related work ‘inside QURI’ vs. making a new org.
“$1m in funding over ~2 years” → I’d personally be excited to see what you do with this, but I’m unsure how easy it would be to raise this. In our experience fundraising was definitely not trivial, and we’ve mostly been asking for much less than $1m (though year by year). Most EA funders now are focused on other dedicated areas like technical AI safety or bio risk, I think our category of work has been more challenging. For a while the FTX Future Fund was interested in these areas, then later Coefficient Giving had a Forecasting fund, but both closed fairly quickly. It’s quite possible that Anthropic money and similar will change the situation, but I haven’t seen that yet. Related, I’d recommend very much deeply understanding the funding climate before making an incubator, as I think it’s easy to create a bunch of orgs that basically have no where to go.
In terms of incubators, some reading this might be interested in Surplus! It’s for-profit only, but definitely related to our work.
If you want a 501(c)(3) sponsor, I think there’s a lot of good options. QURI was itself sponsored by Rethink Priorities for much of its life.
I’d be happy to chat / help mentor either way.
Surplus sounds useful!
I think everything hinges on the funding unfortunately…
Most of the projects on my list require some $200–500k in the first year to get started, and then can scale to a few million per year over time. The large-scale retrofunding needs to start higher – $10m might work, $100m works for the XPrize, $1b could be the goal.
The natural starting point is the incubator itself, which falls into the $200–500k range, but more towards the upper end to provide seed funding for the incubated projects.
Why did Guesstimate/Squiggle as for-profit not work out?
I’d love to have a call and catch up in any case! I’m curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors).
“Why did Guesstimate/Squiggle as for-profit not work out?”
I’ve been asked this question a bunch of times before, happy to give some quick thoughts here. Most of my answer is here.
If it were the case that it seemed very easy to make these as startups, and ideally in which there were some clear cofounders who could handle the business side, I think the for-profit side would have been more promising.
In my case:
1. I realized this was a niche/small space, that getting a large market would likely mean having to pivot.
2. I wasn’t sure what might take off. I think Squiggle has a lot going for it, but am not sure if the value proposition is strong enough for a large company now, especially as LLMs have been getting much more powerful (changing the trade-offs around decision tools).
3. My main goal was helping effective altruist / rationalist researchers, and I suspected that focusing much more on commercialization would have hurt that.
4. Related to (3), I just found myself much more motivated to go after altruists/nonprofits than to go after standard enterprise deals
“I’d love to have a call and catch up in any case! I’m curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors).”
Quickly:
1. I’ll schedule a call sometime, once I understand my schedule a bit better.
2. I have incredibly little info on DeepMind right now. As you may imagine, personally I’m excited about a lot of these areas, but I can’t speak for DeepMind.