Other Rethink Priorities clients (including at Open Phil) might disagree, but my hunch is that if anything, higher quality and lower quantity is the way to go, because a client like me has less context on consultants doing some project than I do on someone I’ve directly managed (internally) on research projects for 2 years. So e.g. Holden vetted my Open Phil work pretty closely for 2 years and now feels less need to do so because he has a sense of what my strengths and weaknesses are, where he can just defer to me and where he should make sure to develop his own opinion, etc. That’s part of the massive cost of hiring, training, and managing internal talent, but it eventually gets you to a place where you don’t need to be so nervous about major crippling flaws (of some kinds) in someone’s work. But a major purpose of outsourcing analysis work is to get some information you need without needing to first have built up months or years of deep context with them. But how can I trust the work of someone I have so little context with? I think “go kinda overboard on legibility / reasoning transparency” and “go kinda overboard on quality / thoroughness / vetting” are two major strategies, especially when the client is far more time-constrained than funding constrained (as Open Phil is).
In this case, do you think RP should focus more on quality and less on quantity as we scale, by satisficing on quantity and focusing/optimizing on research quality (concretely, this may mean being very slow to add additional researchers and primarily using them as additional quality checks on existing work, over trying to have more output in novel work)? This is very much not the way we currently plan to scale, which is closer to focusing on maintaining research quality and trying to increase quantity/output.
(reiterating that all impressions here are my own).
I don’t feel strongly. You all have more context than I do on what seems feasible here. My hunch is in favor of RP maintaining current quality (or raising it only a tiny bit) and scaling quickly for a while — I mostly wanted to give some counterpoints to your suggestion that maybe RP should lower its quality to get more quantity.
Other Rethink Priorities clients (including at Open Phil) might disagree, but my hunch is that if anything, higher quality and lower quantity is the way to go, because a client like me has less context on consultants doing some project than I do on someone I’ve directly managed (internally) on research projects for 2 years. So e.g. Holden vetted my Open Phil work pretty closely for 2 years and now feels less need to do so because he has a sense of what my strengths and weaknesses are, where he can just defer to me and where he should make sure to develop his own opinion, etc. That’s part of the massive cost of hiring, training, and managing internal talent, but it eventually gets you to a place where you don’t need to be so nervous about major crippling flaws (of some kinds) in someone’s work. But a major purpose of outsourcing analysis work is to get some information you need without needing to first have built up months or years of deep context with them. But how can I trust the work of someone I have so little context with? I think “go kinda overboard on legibility / reasoning transparency” and “go kinda overboard on quality / thoroughness / vetting” are two major strategies, especially when the client is far more time-constrained than funding constrained (as Open Phil is).
In this case, do you think RP should focus more on quality and less on quantity as we scale, by satisficing on quantity and focusing/optimizing on research quality (concretely, this may mean being very slow to add additional researchers and primarily using them as additional quality checks on existing work, over trying to have more output in novel work)? This is very much not the way we currently plan to scale, which is closer to focusing on maintaining research quality and trying to increase quantity/output.
(reiterating that all impressions here are my own).
I don’t feel strongly. You all have more context than I do on what seems feasible here. My hunch is in favor of RP maintaining current quality (or raising it only a tiny bit) and scaling quickly for a while — I mostly wanted to give some counterpoints to your suggestion that maybe RP should lower its quality to get more quantity.