This post is a great exemplar for why the term “AI alignment” has proven a drag on AI x-risk safety. The concern is and has always been that AI would dominate humanity like humans dominate animals. All of the talk about aligning AI to “human values” leads to pedantic posts like this one arguing about what “human values” are and how likely AIs are to pursue them.
Gil
Karma: 1,182
Altman, like most people with power, doesn’t have a totally coherent vision for why him gaining power is beneficial for humanity, but can come up with some vague values or poorly-thought-out logic when pressed. He values safety, to some extent, but is skeptical of attempts to cut back on progress in the name of safety.
Hmm, I still don’t think this response quite addresses the intuition. Various groups yield outsized political influence owing to their higher rates of voting—seniors, a lot of religious groups, post-grad degree ppl, etc. Nonetheless, they vote in a lot of uncompetitive races where it would seem their vote doesn’t matter. It seems wrong that an individual vote of theirs has much EV in an uncompetitive race. On the other hand, it seems basically impossible to mediate strategy such that there is still a really strong norm of voting in competitive races but not in uncompetitive races (and besides it’s not clear that that would even suffice given that uncompetitive races would become competitive in the absence of a very large group). I think all the empirical evidence shows that groups that turn out more in competitive races also do so in uncompetitive races.
Sorry, I shouldn’t have used the phrase “the fact that”. Rephrased, the sentence should say “why would the universe taking place in an incomputable continuous setting mean it’s not implemented”. I have no confident stance on if the universe is continuous or not, just that I find the argument presented unconvincing.
That and/or acausal decision theory is at play for this current election
I will say that I think most of this stuff is really just dancing around the fundamental issue, which is that expected value of your single vote really isn’t the best way of thinking about it. Your vote “influences” other people’s vote, either through acausal decision theory or because of norms that build up (elections are repeated games, after all!).
I may go listen to the podcast if you think it settles this more, but on reading it I’m skeptical of Joscha’s argument. It seems to skip the important leap from “implemented” to “computable”. Why does the fact that our universe takes place in an incomputable continuous setting mean it’s not implemented? All it means is that it’s not being implemented on a computer, right?
I think there’s a non-negligible chance we survive until the heat death of the sun or whatever, maybe even after, which is not well-modelled by any of this.
To clarify: the point of this parenthetical was to state reasons why a world without transhumanist progress may be terrible. I don’t think animal welfare concerns disappear or even are remedied much with transhumanism in the picture. As long as animal welfare concerns don’t get much worse however, transhumanism changes the world either from good to amazing (if we figure out animal welfare) or terrible to good (if we don’t). Assuming AI doesn’t kill us obviously.
I think the simplest answer is not that such a world would be terrible (except for factory farming and wild animal welfare, which are major concerns), but that a world with all these transhumanist initiatives would be much better
I am glad somebody wrote this post. I often have the inclination to write posts like these, but I feel like advice like this is sometimes good and sometimes bad and it would be disingenuous for me to stake out a claim in any direction. Nonetheless, I think it’s a good mental exercise to explicitly state the downsides of comparative claims and the upsides of absolute claims, and then people in the comments will (and have) assuredly explain the opposite.
″...for most professional EA roles, and especially for “thought leadership”, English-language communication ability is one of the most critical skills for doing the job well”
Is it, really? Like, this is obviously true to some extent. But I’m guessing that English communication ability isn’t much more important for most professional EA roles than it is for eg academics or tech startup founders. These places are much more diverse in native language than EA I think.
How did he deal with two-envelope considerations in his calculation of moral weights for OpenPhil?
This consideration is something I had never thought of before and blew my mind. Thank you for sharing.
Hopefully I can summarize it (assuming I interpreted it correctly) in a different way that might help people who were as befuddled as I was.
The point is that, when you have probabilistic weight to two different theories of sentience being true, you have to assign units to sentience in these different theories in order to compare them.
Say you have two theories of sentience that are similarly probable, one dependent on intelligence and one dependent on brain size. Call these units IQ-qualia and size-qualia. If you assign fruit flies a moral weight of 1, you are implicitly declaring a conversion rate of (to make up some random numbers) 1000 IQ-qualia = 1 size-qualia. If you assign elephants however to have a moral weight of 1, you implicitly declare a conversion rate of (again, made-up) 1 IQ-qualia = 1000 size-qualia, because elephant brains are much larger but not much smarter than fruit flies. These two different conversion rates are going to give you very different numbers for the moral weight of humans (or as Shulman was saying, of each other).
Rethink Priorities assigned humans a moral weight of 1, and thus assumed a certain conversion rate between different theories that made for a very small-animal-dominated world by sentience.
If OpenPhil’s allocation is really so dependent on moral weight numbers, you should be spending significant money on research in this area, right? Are you doing this? Do you plan on doing more of this given the large divergence from Rethink’s numbers?
Yeah, I think there’s a big difference between how Republican voters feel about it and how their elites do. Romney is, uhh, not representative of most elite Republicans, so I’d be cautious there
Do we have any idea how Republican elites feel about AI regulation?
This seems like the biggest remaining question mark which will determine how much AI regulation we get. It’s basically guaranteed that Republicans will have to agree to AI regulation legislation, and Biden can’t do too much without funding in legislation. Also there’s a very good chance Trump wins next year and will control executive AI Safety regulation.
Well, this is our moment.
Politics is really important, so thank you for recognizing that and adding to discussion about Pause.
But this post confuses me. You start by talking about how protests are stronger when they are centered on something people care about rather than simply policy advocacy. Which, I don’t know if I agree with, but it’s an argument that you can make. But then you shift toward advocating for regulation rather than pause. Which is also just policy advocacy, right? And I don’t understand why you’d expect it to have better politics than a pause. Your points about needing companies to prove they are safe is pretty much the same point that Holly Elmore has been making, and I don’t know why they apply better to regulation than a Pause.
Love the post, don’t love the names given.
I think “capacity growth” is a bit too vague, something like “tractable, common-sense global interventions” seems better.
I also think “moonshots” is a bit derogatory, something like “speculative, high-uncertainty causes” seems better.