I love this! Reminds me of Vavilov guarding the seed bank.
One of the worries I’ve heard shared about longtermism is that it’s hard to find interventions that keep on being cost effective when there’s a lot of funding to throw at them—like where the last dollar is as valuable as the first dollar? How do you think that would play out here? To what extent are there diminishing returns?
Thank you for this. I love Star Trek Discovery too.
Something I’m a bit puzzled by—what would we do to make benevolent AI over and above what we already do? We already train LLMs to do things we (for a given value of “we”) find useful and not useless. But a plan for a benevolent AI must require more than that. If you had to sketch out how to teach an AI benevolence, how would you do it?