1) Convergence Analysis: The idea here is to create a Berkeley affiliated research institute that operates mainly in two fronts 1)Strategy on the long term future 2)Finding Crucial Considerations that have not been considered or researched yet. We have an interesting group of academics and I would take a mixed position of CEO and researcher.
2) Altruism: past, present, propagation: this is a book whose table of contents I already wrote, and would need further research and spelling out each of the 250 sections I have in mind. It is very different in nature from Will’s book, or Singer’s book. The idea here is not to introduce to EA, but to reason about the history of cooperation and altruism that led to us, and where this can be taken in the future, inclusive by the EA movement. This would be major intellectual undertaking, likely consuming my next three years and doubling as a PHD dissertation. Perhaps, tripling as a series of blog posts, for quick feedback loops and reliable writer motivation.
3) FLI grant proposal: Our proposal intended to increase our understanding psychological theories of human morality in order to facilitate later work in formalizing moral cognition to AIs, a subset of the value loading and control problems of Artificial Generalized Intelligence. We didn’t win, so the plan here would be to try to find other funding sources for this research.
4) Accelerate the PHD: For that I need to do 3 field statements, one about the control problem in AI with Stuart, one about altruism with Deacon, and one to be determined, then only the dissertation would be still on the to do list.
All these plans scored sufficiently high in my calculations that it is hard to decide between them. Accelerating the PHD has a major disadvantage because it does not increase my funding. The book (via blog posts or not) has a strong advantage in that I think it will have sufficiently new material that it satisfies goal 1 best of all, it is probably the best for the world if I manage to get to the end of it and do it well. But again, it doesn’t increase funding. Convergence has the advantage of co-working with very smart people, and if it takes off sufficiently well, it could solve the problem of continuing to live in Berkeley and that of financial constraints all at once, putting me in a stable position to continue doing research in relevant topics almost indeterminately, instead of having to make ends meet by downsizing the EA goal substantially among my priorities. So very high stakes, but uncertain probabilities.
If AI is (nearly) all that matters, then the FLI grant will be the highest impact, followed by Convergence, the book and the acceleration.
In any event all of those are incredible opportunities which I feel lucky to even have in my consideration space. It is a privilege to be making that choice, but it is also very hard. So conditional on the goals I stated before:
1)Making the world better by the most effective means possible. 2)Continuing to live in Berkeley 3)Receive more funding 4)Not stop PHD 5)Use my knowledge and background to do (1).
I am looking for some light, some perspective from the outside that will make me lean one way or another. I have been uncomfortably indecisive for months, and maybe your analysis can help.
Three of your projects rank highest on personal interest. I think I would attempt a more granular analysis of this keeping in mind your current uncertainty about your model of your maintained future interest.
Some ideas:
Pretend that you are handing off the project to someone else and writing a short guide to what you think their curriculum of research and work will look like.
Brainstorm the key assumptions underlying the model that assumes a value for each project and see if any of those key assumptions are very cheaply testable (this can be a surprising exercise IME)
Premortem (murphyjitsu) each project and compare the likelihood of different failure modes.
If there’s a way to encourage Russell to write or teach a bit more about AI safety (even just in his textbook, or maybe in other domains), I would think that would be quite important. But you probably have a better picture of how (in)feasible that is.
Sorry that I don’t have strong opinions on the other options....
All of these seem potentially valuable. I suspect the best choice is the one you’ll be most motivated to pursue. My suggestion is that you should consider who your ‘customers’ are for each project, and figure out which group you’d most like to work with and generate deliverables for.
Also, some of these may lend themselves better to intermediate/incremental deliverables, which would be a big plus.
All of the above is fully general advice- my low-resolution take on your specific situation is that Convergence Analysis seems by far both the highest leverage (and certainly the largest variance), though the fact that you seem unsure whether to dive down that path may imply there may be some difficult hurdles or complications down that path that you’re dreading?
As Luke and Nate would tell you, the shift from researcher to CEO is a hard one to make, even when you want to do good, as Hanson puts it “Yes, thinking is indeed more fun.”
I have directed an institute in Brazil before, and that was already somewhat a burden.
The main reason for the high variance though is that setting up an institute requires substantial funding. The people most likely to fundraise would be me, Stephen Frey (who is not on the website), and Daniel, and fundraising is taxing in many ways. Would be great if we had for instance the REG fundraisers to aid us (Liv, Ruari, Igor, wink wink) either by fundraising for us, or finding someone to, or teaching us to.
1) Convergence Analysis: The idea here is to create a Berkeley affiliated research institute that operates mainly in two fronts 1)Strategy on the long term future 2)Finding Crucial Considerations that have not been considered or researched yet. We have an interesting group of academics and I would take a mixed position of CEO and researcher.
2) Altruism: past, present, propagation: this is a book whose table of contents I already wrote, and would need further research and spelling out each of the 250 sections I have in mind. It is very different in nature from Will’s book, or Singer’s book. The idea here is not to introduce to EA, but to reason about the history of cooperation and altruism that led to us, and where this can be taken in the future, inclusive by the EA movement. This would be major intellectual undertaking, likely consuming my next three years and doubling as a PHD dissertation. Perhaps, tripling as a series of blog posts, for quick feedback loops and reliable writer motivation.
3) FLI grant proposal: Our proposal intended to increase our understanding psychological theories of human morality in order to facilitate later work in formalizing moral cognition to AIs, a subset of the value loading and control problems of Artificial Generalized Intelligence. We didn’t win, so the plan here would be to try to find other funding sources for this research.
4) Accelerate the PHD: For that I need to do 3 field statements, one about the control problem in AI with Stuart, one about altruism with Deacon, and one to be determined, then only the dissertation would be still on the to do list.
All these plans scored sufficiently high in my calculations that it is hard to decide between them. Accelerating the PHD has a major disadvantage because it does not increase my funding. The book (via blog posts or not) has a strong advantage in that I think it will have sufficiently new material that it satisfies goal 1 best of all, it is probably the best for the world if I manage to get to the end of it and do it well. But again, it doesn’t increase funding. Convergence has the advantage of co-working with very smart people, and if it takes off sufficiently well, it could solve the problem of continuing to live in Berkeley and that of financial constraints all at once, putting me in a stable position to continue doing research in relevant topics almost indeterminately, instead of having to make ends meet by downsizing the EA goal substantially among my priorities. So very high stakes, but uncertain probabilities. If AI is (nearly) all that matters, then the FLI grant will be the highest impact, followed by Convergence, the book and the acceleration.
In any event all of those are incredible opportunities which I feel lucky to even have in my consideration space. It is a privilege to be making that choice, but it is also very hard. So conditional on the goals I stated before: 1)Making the world better by the most effective means possible. 2)Continuing to live in Berkeley 3)Receive more funding 4)Not stop PHD 5)Use my knowledge and background to do (1).
I am looking for some light, some perspective from the outside that will make me lean one way or another. I have been uncomfortably indecisive for months, and maybe your analysis can help.
Three of your projects rank highest on personal interest. I think I would attempt a more granular analysis of this keeping in mind your current uncertainty about your model of your maintained future interest.
Some ideas:
Pretend that you are handing off the project to someone else and writing a short guide to what you think their curriculum of research and work will look like.
Brainstorm the key assumptions underlying the model that assumes a value for each project and see if any of those key assumptions are very cheaply testable (this can be a surprising exercise IME)
Premortem (murphyjitsu) each project and compare the likelihood of different failure modes.
Thanks for sharing. :)
If there’s a way to encourage Russell to write or teach a bit more about AI safety (even just in his textbook, or maybe in other domains), I would think that would be quite important. But you probably have a better picture of how (in)feasible that is.
Sorry that I don’t have strong opinions on the other options....
All of these seem potentially valuable. I suspect the best choice is the one you’ll be most motivated to pursue. My suggestion is that you should consider who your ‘customers’ are for each project, and figure out which group you’d most like to work with and generate deliverables for.
Also, some of these may lend themselves better to intermediate/incremental deliverables, which would be a big plus.
All of the above is fully general advice- my low-resolution take on your specific situation is that Convergence Analysis seems by far both the highest leverage (and certainly the largest variance), though the fact that you seem unsure whether to dive down that path may imply there may be some difficult hurdles or complications down that path that you’re dreading?
As Luke and Nate would tell you, the shift from researcher to CEO is a hard one to make, even when you want to do good, as Hanson puts it “Yes, thinking is indeed more fun.”
I have directed an institute in Brazil before, and that was already somewhat a burden.
The main reason for the high variance though is that setting up an institute requires substantial funding. The people most likely to fundraise would be me, Stephen Frey (who is not on the website), and Daniel, and fundraising is taxing in many ways. Would be great if we had for instance the REG fundraisers to aid us (Liv, Ruari, Igor, wink wink) either by fundraising for us, or finding someone to, or teaching us to.
Money speaks. And it spells high variance.