Anything I write here is written purely on my own behalf, and does not represent my employerās views (unless otherwise noted).
Erich_Grunewald šø
Karma: 2,925
Why You Should AlĀmost Never Use AI to Write AnyĀthing Substantive
I agree that animal welfare is underfunded, but who is this post trying to convince? If funders should fund the most cost-effective interventions or cause areas, then the argument needs to be made on the merits, which are not covered here. Animals donāt just automatically win because there is lots of animal suffering today (unless maybe you are a fairly near-termist negative utilitarian).
Did you have any system prompt or other instructions active when you asked these things? As someone else mentioned in another comment, incognito mode just means that Anthropic doesnāt save the chat, but your general instructions for Claude are still accessible to it in incognito mode.
I agree that the Forum is (fairly) dead; I donāt read it much anymore, and feel somewhat sad about this. I also think a lot of energy has moved to Substack, and would guess that this is a stronger effect than LessWrong cross-posting.
Itās true that technological progress so far has been largely good for humans and bad for animals (due to factory farms, but the effects on wild animals complicate this a lot).
But I also think human values towards animals have improved compared to how they were historically, and e.g., house and work animals are likely treated better on the whole now than in the past. So I think thereās been some moral progress, but this improvement has been dominated by technology simultaneously making animal food production much more cost-effective (and, as a side-effect, more suffering-producing).
I think eventually technological progress will make it cheaper to act on animal-friendly values, because Iām guessing the taste/āprice/āconvenience/āfriction of animal meat is hitting diminishing returns, while thereās much more room for improvement with non-animal-based foods. So I think there will be a crossover point at some point, sort of the way it was with the Enlightenment and the industrial revolution, where the economic effects of technology first made many people probably worse off, but eventually the better values won out and people were left better off on the whole.
I separately also think AGI and especially ASI, if aligned with human values, can wisely advise us on good courses of actions and help improve our values and promote good values, which would also help. ASI could also help a lot with wild animal welfare, where we are currently quite at a loss.
I think this is beautiful and touching and a little scary.
Yet nobody is building the I-O equivalent for AI compute governance.
Iām kind of skeptical that the data exists to create a good input-output model for AI chip export controls. For example, we know very little about which companies (especially in China) rent which AI chips from which cloud providers, and in which volumes; about any of the substation elasticities; or about smuggling volumes. We know more about AI chip sales now, but still little about which companies are buying them. But Iād be very interested in knowing if Iām wrong about this lack of data being a problem for input-output models?
Also, I feel quite confident, based on style, that this post was at least partially AI-written, in a way that made me like it less and take it less seriously. Make of that what you will!
A much better order of operations would be to 1) try to negotiate with China to establish an international regulatory framework (plan A), with export control and other stuff being imposed as something that is explicitly linked to China not agreeing to that framework, in the same way sanctions on Russia are imposed explicitly because its aggression against Ukraine, and 2) only if they refuse, try to crush them (plan B).
Maybe if you are President of the United States you can first try the one thing, and then the other. But from the perspective of an individual, you have to assume thereās some probability of each of these plans (and other strategies) being executed, and that everything will be really messy (e.g., different actors having different strategies in mind, even within the US). Softening export controls seems like something you could do as part of executing Plan A, but as I mentioned above, itās very unclear to me whether unilaterally doing so makes Plan A more likely to be the chosen strategy, and it does likely make Plan B and Plan C go worse.
When political will in the US to try for plan A is lacking, I think waiting until circumstances make that plan realistic while preparing the groundwork for it is a better strategy than going straight ahead for plan B.
I think youāre thinking people have more control over which strategy is adopted than I think they do? Or, what circumstances do you have in mind? Because waiting seems pretty costly.
But I think maybe the cruxiest bits are (a) I think export controls seem great in Plan B/āC worlds, which seem much likelier than Plan A worlds, and (b) I think unilaterally easing export controls is unlikely to substantially affect the likelihood of Plan A happening (all else equal). It seems like you disagree with both, or at least with (b)?
I think if your sole objective is to enact a bilateral pause, then easing export controls may be the best option, or maybe not. Itās pretty unclear to me how that shakes out, I could definitely also see unilateral concessions as being quite detrimental (for reasons similar to those Peter mention in the other comment).
But I would guess most of the people you are responding to think enacting a cooperative pause is some combination of very unlikely and/āor undesirable, and also that export controls help a lot in the absence of such an agreement. The main way export controls help for other plans are by giving the US more slack to (one can hope) spend on safety, and/āor because superintelligence developed in the US would imo likely be safer (cf. this comment), and/āor because imo US values are better, and this would likely be reflected in the AIs (cf. Claude versus Grok).
If you thought Yudkowsky and Soares used overly confident language and would have taken the āQEDā as further evidence of that, but this particular example turns out not to have been written by Yudkowsky and Soares, thatās some evidence against your hypothesis. But instead of updating away a little, you seemed to dismiss that evidence and double down. (I think you originally replied to the original comment approvingly or at least non-critically, but then deleted that comment after I replied to it, but I could be misremembering that.)
For what itās worth, I think youāre right that Yudkowsky at least uses overly confident language sometimesāor I should say, is overly confident sometimes, because I think his language generally reflects his beliefsābut I wouldāve been surprised to see him use āQEDā in that way, which is why I reacted to the original comment here with skepticism and checked whether āQEDā actually appeared in the book (it didnāt). I take that to imply I was better calibrated than anyone who did not so react.
Interesting!
Given that these failures were predictable, it should be possible to systematically predict many analogous failures that might result from training AI systems on specific data sets or (simulated) environments.
Your framework seems to work for simple cases like āice cream, sucralose, or sex with contraceptionā, but I donāt think it works for more complex cases like āpeacocks would like giant colorful tailsā?
There is so much human behaviour also that would have been essentially impossible to predict just from first principles and natural selection under constraints: poetry, chess playing, comedy, monasticism, sports, philosophy, effective altruism. These behaviours seem further removed from your detectors for instrumentally important subgoals, and/āor to have a more complex relationship to those detectors, but theyāre still widespread and important parts of human life. This seems to support the argument that the relationship between how a mind was evolved (e.g., by natural selection) and what it ends up wanting is unpredictable, possibly in dangerous ways.
Your model might still tell us that generalisation failures are very likely to occur, even if, as I am suggesting, it canāt predict many of the specific ways things will misgeneralise. But Iām not sure this offers much practical guidance when trying to develop safer AI systems. But maybe Iām wrong about that?
Who are ātheyā? If you mean Yudkowsky and Soares, āQEDā is something that Hanson (the author of this critique) includes in his paraphrase of Yudkowsky and Soares, but I donāt think itās anything Yudkowsky and Soares wrote in their book. The quoted argument is not actually a quote, but a paraphrase.
For what itās worth, I would guess that though the āfunnessā of AI safety research, or maybe especially technical AI safety research, is probably a factor in determining how many people are interested in working on that, I would be surprised if itās a factor in determining how much money is allocated towards that as a field.
(To be clear, I do think many of these charities do some good and are run with the best of intentions, etc. But I still also stand by the statement in the parent comment.)
That is the most PR-optimized list of donations I have ever seen in my life.
And also to include the link? Or maybe Iām too dumb to see it.
Thanks for sharing this. I did an Erasmus exchange year in Italy in 2010-11 that was very important for my personal growth, although it was not particularly beneficial professionally or academically.
Nice work!
On AI chip smuggling, rather than the report you listed, which is rather outdated now, I recommend reading Countering AI Chip Smuggling Has Become a National Security Priority, which is essentially a Pareto improvement over the older one.
I also think Chris Millerās How US Export Controls Have (and Havenāt) Curbed Chinese AI provides a good overview of the AI chip export controls, and it is still quite up-to-date.
(I have already read these posts and much prefer them over the post weāre commenting on here.)