Some quick thoughts. I work at Rethink Priorities and also started recently as a (very) part-time guest manager at the Long-Term Future Fund, but all thoughts are my own. Note that I’m actively thinking about this right now, so these are just my working thoughts, and they’re subject to change in months or even days.
Here are factors I use to identify whether a project is worth doing (keeping personnel factors constant, which is true if I’m advising someone/myself whether to do a project, and not for e.g. funding decisions).
Note that I’m less rigorous/principled than the below list will suggest in practice, as sometimes a single factor may dominate (e.g., if a project has really strong learning value, or the story for decreasing x-risk is unusually clear, or if the researcher just seems so viscerally excited about the project that it’d be hard to convince them to do anything else).
Does this project have a strong direct line/clean story for improving the long-term future (especially via reducing existential risk), and/or multiple fuzzier stories?
E.g. at similar levels of cost-effectiveness, I’m much more excited about projects that can scale to >$100M of spending, without losing (much) cost-effectiveness, than projects that “top out” at $100k or $1M.
Relatedly, what does the learning/exploration value of this project look like?
To the researcher/entrepreneur?
To the institution? (if they’re working in an EA-institutional context)
To the EA or longtermist ecosystem as a whole?
Does the researcher/entrepreneur working at this project seem viscerally excited about this? Are they likely to pursue it with passion and vigor?
Am I personally excited about this project? Can I imagine working on the project myself, or spending multiple hours a week advising this project?
Do other people I respect like this project (or other projects like it), and want there to be more of this project in the world?
Does the project have fast ways to fail? Can you de-risk your investment by realizing the project is not worth it with <10% of the time/money investment?
What does the project look like in terms of explicit cost-effectiveness?
This matters much more for larger projects than smaller ones, and more for projects where the learning/exploration value is low than ones where the exploration value is high.
By cost-effectiveness, I mean something like “At scale, how many dollars or equivalent in human capital do you need to avert one microextinction/microdoom or basis point in x-risk”
Thanks Linch! This list is really helpful. One clarifying question on this point:
Relatedly, what does the learning/exploration value of this project look like?
To the researcher/entrepreneur?
To the institution? (if they’re working in an EA-institutional context)
To the EA or longtermist ecosystem as a whole?
For 1) and 2), I assume you’re referring to the skills gained by the person/institution completing the project, which they could then apply to future projects.
For 3), are you referring to the possibility of “ruling out intervention X as a feasible way to tackle x-risks”? That’s what I’m assuming, but I’m just asking to make sure I understand properly.
Thanks for the question! Hmm, I don’t think there’s a hard cutoff of person/institution vs. ecosystem. For 3), skills learned from completing a project (or trying to complete a project) might also be generalizable elsewhere (so there’s value other than ruling out specific interventions).
For example, learning how to do a biosecurity ballot initiative in California can be useful for doing future biosecurity ballot initiatives in California, or AI safety ones. Some of the skills and knowledge acquired here can be passed on to other individuals or orgs.
Some quick thoughts. I work at Rethink Priorities and also started recently as a (very) part-time guest manager at the Long-Term Future Fund, but all thoughts are my own. Note that I’m actively thinking about this right now, so these are just my working thoughts, and they’re subject to change in months or even days.
Here are factors I use to identify whether a project is worth doing (keeping personnel factors constant, which is true if I’m advising someone/myself whether to do a project, and not for e.g. funding decisions).
Note that I’m less rigorous/principled than the below list will suggest in practice, as sometimes a single factor may dominate (e.g., if a project has really strong learning value, or the story for decreasing x-risk is unusually clear, or if the researcher just seems so viscerally excited about the project that it’d be hard to convince them to do anything else).
Does this project have a strong direct line/clean story for improving the long-term future (especially via reducing existential risk), and/or multiple fuzzier stories?
Does it have leverage/scalability potential?
E.g. at similar levels of cost-effectiveness, I’m much more excited about projects that can scale to >$100M of spending, without losing (much) cost-effectiveness, than projects that “top out” at $100k or $1M.
Relatedly, what does the learning/exploration value of this project look like?
To the researcher/entrepreneur?
To the institution? (if they’re working in an EA-institutional context)
To the EA or longtermist ecosystem as a whole?
Does the researcher/entrepreneur working at this project seem viscerally excited about this? Are they likely to pursue it with passion and vigor?
Am I personally excited about this project? Can I imagine working on the project myself, or spending multiple hours a week advising this project?
Do other people I respect like this project (or other projects like it), and want there to be more of this project in the world?
Though I try to be careful about what my own independent impressions are, and not contribute to information cascades.
Does the project have fast ways to fail? Can you de-risk your investment by realizing the project is not worth it with <10% of the time/money investment?
What does the project look like in terms of explicit cost-effectiveness?
This matters much more for larger projects than smaller ones, and more for projects where the learning/exploration value is low than ones where the exploration value is high.
By cost-effectiveness, I mean something like “At scale, how many dollars or equivalent in human capital do you need to avert one microextinction/microdoom or basis point in x-risk”
Thanks Linch! This list is really helpful. One clarifying question on this point:
For 1) and 2), I assume you’re referring to the skills gained by the person/institution completing the project, which they could then apply to future projects.
For 3), are you referring to the possibility of “ruling out intervention X as a feasible way to tackle x-risks”? That’s what I’m assuming, but I’m just asking to make sure I understand properly.
Thanks again!
Thanks for the question! Hmm, I don’t think there’s a hard cutoff of person/institution vs. ecosystem. For 3), skills learned from completing a project (or trying to complete a project) might also be generalizable elsewhere (so there’s value other than ruling out specific interventions).
For example, learning how to do a biosecurity ballot initiative in California can be useful for doing future biosecurity ballot initiatives in California, or AI safety ones. Some of the skills and knowledge acquired here can be passed on to other individuals or orgs.