Good critique, my main conclusion is that redwood seems reasonable overall and not far out of line from other ai safety orgs. Benchmarked against non-ai safety orgs, I would have my usual critique that redwood (and other longtermist orgs) seems unreasonably expensive for reasons I don’t quite understand. Does salary really make that big a difference in attracting talent? If that is the case, what does that say about our community’s values?
In any case, remember that every org has issues. When listing every issue an org has in a row it can give an impression of things being worse than they really are. Would love a similar critique be made of the organization I co-founded once we grow to a similar size. More critique is good for the community.
We should be able to write scathing criticisms without getting mad at each other. We need to be able to read criticisms and not go completely ham and want to see the org and everyone associated guillotined.
Benchmarked against non-ai safety orgs, I would have my usual critique that redwood (and other longtermist orgs) seems unreasonably expensive for reasons I don’t quite understand. Does salary really make that big a difference in attracting talent? If that is the case, what does that say about our community’s values?
Can you say more about what your implicit benchmark actually is here? Taken literally, “non-ai safety orgs” possibly describes almost all human organizations.
Startups would be another good reference class. VCs are incentivized to scale as fast as possible so they can cash out and reinvest their money, but they rarely give a new organization as much money as Redwood received.
Startups usually receive a seed round of ~$2M cash to cover the first year or two of business, followed by ~$10M for Series A to cover another year or two. Even Stripe, a VC wunderkind that’s raised billions privately while scaling to thousands of employees around the world, began with $2M for their first year, $38M for the next three years (2012-2014), and $70M for the next two years after that.
I’m not sure how long Redwood’s $21M is meant to cover, but if it’s less than a period of 4 years, then they’re spending more than the typical 5M/year for a Series A startup. There’s a good argument to be made that OP can be more risk tolerant than most VCs and take a big swing on scaling Redwood quickly. But beyond cost-effectiveness, another downside of fast funding is that scaling organizations effectively is very difficult, and it could be counterproductive to hire quickly before you have senior management in place with clear lines of tractable work.
Thanks, this is helpful. One thing to flag is that I wouldn’t find the 2012-2014 numbers very convincing; my impression is that VC funding increased a lot until 2022, and 2021 was a year where capital was particularly cheap, for reasons that in hindsight were not entirely dissimilar to why longtermist EA was (relatively) well-funded in the last two years.
Yep that’s a good point. Here’s one source on it, funding amounts definitely increased throughout the 2010s. An alternative explanation could be that valuations have increased more than funding amounts. There’s some data to support this, but you’d need a more careful comparison of startups within the same reference class to be sure.
Maybe his own org and other global development orgs? I think it’s almost always a mistake for a non-profit to get this much money this quickly, regardless of how much potential they have or the good reputation of their founders. It is difficult to gradually build an org and organically make the inevitable mistakes when you are given 10 million dollars in the first year.
I won’t speak for @MathiasKB , but these agree some of my benchmarks outside the AI realm—he can share what he means :).
The Center for Effective aid policy Matthias and co is a brand new org, so don’t have evidence of outputs or financials yet. They were given 170,000US to start up. To many in the development world even 170k might still seem like a lot for an NGO to start with, but it’s still a lot less than 10 million.
Last year our org OneDay Health which has a decent chance of being effective employed 43 staff, launched 8 Health centers, treated 50,000 patients in the most remote rural parts of Uganda, and our total expenditure for the year was $104,000 US dollars.
If we are looking at a development org with a budget on a similar scale, Last Mile Health has a 10 year track record, grew steadily, has won countless awards (Social innovation, TED prizes etc), has been a crucial part of the global movement for rolling out community health workers impacting improving health access to millions of people accross 5+ countries, and employs hundreds of people both in the US and developing world. They spent about 26 million dollars last year. Which is a lot of money, but in the ballpark of Redwood research and only after many years of high performance, proven recognition and growth.
Even as a global development guy, I think AI alignment research is important, but it is somewhat hard to understand why it’s a good idea for a new, small org like this to get this much money from the getgo. Perhaps start with 1 million in the first year with the CEO and co-founder taking a low-ish salary while the org builds their reputation then ramp things up after that?
Mind you if we really do only have 5-20 years before potentially dangerous GAI, maybe we have to sacrifice sustainable growth and stewardship of money at the altar of having a chance to save the world?
Good critique, my main conclusion is that redwood seems reasonable overall and not far out of line from other ai safety orgs. Benchmarked against non-ai safety orgs, I would have my usual critique that redwood (and other longtermist orgs) seems unreasonably expensive for reasons I don’t quite understand. Does salary really make that big a difference in attracting talent? If that is the case, what does that say about our community’s values?
In any case, remember that every org has issues. When listing every issue an org has in a row it can give an impression of things being worse than they really are. Would love a similar critique be made of the organization I co-founded once we grow to a similar size. More critique is good for the community.
We should be able to write scathing criticisms without getting mad at each other. We need to be able to read criticisms and not go completely ham and want to see the org and everyone associated guillotined.
Can you say more about what your implicit benchmark actually is here? Taken literally, “non-ai safety orgs” possibly describes almost all human organizations.
Startups would be another good reference class. VCs are incentivized to scale as fast as possible so they can cash out and reinvest their money, but they rarely give a new organization as much money as Redwood received.
Startups usually receive a seed round of ~$2M cash to cover the first year or two of business, followed by ~$10M for Series A to cover another year or two. Even Stripe, a VC wunderkind that’s raised billions privately while scaling to thousands of employees around the world, began with $2M for their first year, $38M for the next three years (2012-2014), and $70M for the next two years after that.
I’m not sure how long Redwood’s $21M is meant to cover, but if it’s less than a period of 4 years, then they’re spending more than the typical 5M/year for a Series A startup. There’s a good argument to be made that OP can be more risk tolerant than most VCs and take a big swing on scaling Redwood quickly. But beyond cost-effectiveness, another downside of fast funding is that scaling organizations effectively is very difficult, and it could be counterproductive to hire quickly before you have senior management in place with clear lines of tractable work.
Some numbers here (https://www.investopedia.com/articles/personal-finance/102015/series-b-c-funding-what-it-all-means-and-how-it-works.asp) and here (https://www.fundz.net/what-is-series-a-funding-series-b-funding-and-more). For Stripe funding numbers, google crunchbase Stripe Seed / Series A / Series B.
Thanks, this is helpful. One thing to flag is that I wouldn’t find the 2012-2014 numbers very convincing; my impression is that VC funding increased a lot until 2022, and 2021 was a year where capital was particularly cheap, for reasons that in hindsight were not entirely dissimilar to why longtermist EA was (relatively) well-funded in the last two years.
Yep that’s a good point. Here’s one source on it, funding amounts definitely increased throughout the 2010s. An alternative explanation could be that valuations have increased more than funding amounts. There’s some data to support this, but you’d need a more careful comparison of startups within the same reference class to be sure.
Thanks, appreciate the concrete data!
I appreciate this comment for giving concrete data that improves my model of the world. Thanks.
Maybe his own org and other global development orgs? I think it’s almost always a mistake for a non-profit to get this much money this quickly, regardless of how much potential they have or the good reputation of their founders. It is difficult to gradually build an org and organically make the inevitable mistakes when you are given 10 million dollars in the first year.
I won’t speak for @MathiasKB , but these agree some of my benchmarks outside the AI realm—he can share what he means :).
The Center for Effective aid policy Matthias and co is a brand new org, so don’t have evidence of outputs or financials yet. They were given 170,000US to start up. To many in the development world even 170k might still seem like a lot for an NGO to start with, but it’s still a lot less than 10 million.
Last year our org OneDay Health which has a decent chance of being effective employed 43 staff, launched 8 Health centers, treated 50,000 patients in the most remote rural parts of Uganda, and our total expenditure for the year was $104,000 US dollars.
If we are looking at a development org with a budget on a similar scale, Last Mile Health has a 10 year track record, grew steadily, has won countless awards (Social innovation, TED prizes etc), has been a crucial part of the global movement for rolling out community health workers impacting improving health access to millions of people accross 5+ countries, and employs hundreds of people both in the US and developing world. They spent about 26 million dollars last year. Which is a lot of money, but in the ballpark of Redwood research and only after many years of high performance, proven recognition and growth.
Even as a global development guy, I think AI alignment research is important, but it is somewhat hard to understand why it’s a good idea for a new, small org like this to get this much money from the getgo. Perhaps start with 1 million in the first year with the CEO and co-founder taking a low-ish salary while the org builds their reputation then ramp things up after that?
Mind you if we really do only have 5-20 years before potentially dangerous GAI, maybe we have to sacrifice sustainable growth and stewardship of money at the altar of having a chance to save the world?
ha good point! I specifically had non-ai EA orgs in mind, could have made that clearer!