“You can’t prove it” isn’t the type of argument I expect to see if we’re truthseeking. All of these are positions taken by various other experts, and are at least reasonable. No, I’m not certain, but I don’t need to be when others are risking my life in pursuit of short term military advantage.
Davidmanheim
Is what the US has done or supported in Iraq, Syria, Israel and elsewhere materially or obviously less bad?
Do you feel the same way if AGI is created by the Trump administration, which has openly opposed a variety of human right?
(I’m not entirely disagreeing directionally, I’m hoping to ask honestly to understand your views, not attack them.)
It’s a game of chicken, and I don’t really care which side is hitting the accelerator if I’m stuck in one of the cars. China getting uncontrolled ASI first kills me the same way that the US getting it does.
Edit to add: I would be very interested in responses instead of disagree votes. I think this should be the overwhelming consideration for anyone who cares about the future more than, say, 10 years out. If people disagree, I would be interested in understanding why.
Excited to see people capitalizing on opportunities like this!
I’d wonder how much harder it would be to get large commercial product vendors to consider the same thing. (Approaching them is probably harder, and they likely already have food chemists, etc. who would have vested interests and/or would need to be convinced.)
On AIxCyber field building, you might be interested in knowing about Heron, which launched this past year with Openphil support.
Permanence should already be under cost effectiveness over time, but I agree it’s not obvious where it goes in the simplified ITN framework. if we added it, I’d suggest ‘persistence’ as the more general framing for how long the fix lasts. And fistulas won’t ever get fixed permanently globally the way eliminating diseases globally can permanently fix an entire problem, so I’d say it’s fairly persistent, but not fully permanent.
Close enough not to have any cyclic components that would lead to infinite cycles for the nonsatiable component of their utility.
Humans are neither coherent, nor do they necessarily have a nonsatiable goal—though some might. But they have both to a far greater extent than less intelligent creatures.
Are you willing to posit that advanced systems are coherent, with at least one non-satiable component? Because that’s pretty minimal as an assumption, but implies with probability one that they prefer paperclipping of some sort.
Where and when are these supposed to occur, and how can we track that for our respective countries?
“EA has always bee[n] rather demanding,”
I want to clarify that this is a common but generally incorrect reading of EA’s views. EA leaders have repeatedly clarified that you don’t need to dedicate your life to it, and can simply donate to causes that others have identified as highly effective, and otherwise live your life.
If you want to do more than that, great, good for you—but EA isn’t utilitarianism, so please don’t confuse the demandingness of the two.
First, Utilitarianism doesn’t traditionally require the type of extreme species neutrality you propose here.Singer and many EAs gave a somewhat narrower view of what ‘really counts’ as Utilitarian, but your argument assumes that narrow view without really justifying it.
Second, you assume future AIs will have rich inner lives that are valuable, instead of paperclipping the universe. You say “one would need to provide concrete evidence about what kinds of objectives advanced AIs are actually expected to develop”—but Eliezer has done that quite explicitly.
Very much in favor of posts clarifying that cause neutrality doesn’t require value neutrality or deference to others’ values.
I very much appreciate that you are thinking about this, and the writing is great. That said, without trying to address the arguments directly, I worry that the style here is justifying a conclusion you’ve come to and explores analogies you like rather than exploring the arguments and trying to decide what side to be on, and it fails to embrace scout mindset sufficiently to be helpful.
I think that replaceability is very high, so the counterfactual impact is minimal. But that said, there is very little possibility in my mind that even helping with RLHF for compliance with their “safety” guidelines is more beneficial for safety than for accelerating capabilities racing, so any impact is negative.
Thank you for fighting the good fight!
I don’t think multiperson disagreements are in general a tractable problem for one hour sessions. It sounds like you need someone in charge to enable disagree then commit, rather than a better way to argue.
How much of the money raised by effectiv-spenden, etc. is a essentially pass through to Givewell? (I know Israel now has a similar initiative, but is in large part passing the money to the same orgs.)
I’m confused why people think certainty is needed to characterize this as a game of chicken! It’s certainly not needed in order for the game theoretic dynamics to apply.
I can make a decision about whether to oppose something given that there is substantial uncertainty, and I have done so.