Here to talk about phytomining for now.
Benjamin M.
Karma: 176
I agree with large chunks of this post, but I’m weakly confident (75ish%) that the claim about how newspapers work is wrong. Most newspapers that I am familiar with give their reporters specified beats (topics that they focus on), to at least some extent, although I think there are also reporters that don’t have specific beats. So if there’s an important tech story that needs covered, like AI x-risk, some of that is going to be as a replacement for other tech stories and some of that is going to be taken from people who write on pretty much anything. That still might mean more AI present-risk coverage, because it’s hard to talk about one without talking about the other, and because there’s a lot of other room in tech to take away stories from, but I don’t think it’s as simple as it appears.
However, I’m basing this mostly off info from newspapers that wouldn’t write important AI x-risk stories. Maybe they behave differently? Some other people here probably know more than I do.
I cut out a bit that I had on how much it could scale to make it shorter, but there’s still a bit about that in the practical section. In general, phytomining could potentially produce the vast majority of some of the less common elements (thallium), and Krol-Sinclair and Hale say that the same is true for cobalt. It could also produce a pretty large chunk of nickel- I don’t have an exact number but I’m pretty confident that it could be a double-digit percentage of the world’s consumption. So yes, it scales, but figuring out how much it scales would be a good target for future research.
I’m not an expert on most of the evidence in this post, but I’m extremely suspicious of the claim that GPT-4 represents AI that is “~ human level at language”, unless you mean something by this that is very different from what most people would expect.
Technically, GPT-4 is superhuman at language because whatever task you are giving it is in English, and the median human’s English proficiency is roughly nil. But a more commonsense interpretation of this statement is that a prompt-engineered AI and a trained human can do the task roughly as well.
What you link to shows the results of how GPT-4 performs on a bunch of different exams. This doesn’t really show how language is used in the real world, especially since the exams very closely match past exams that were in the training data. It’s good at some of them, but also extremely bad at others (AP English Literature and Codeforces in particular), which is an issue if you’re making a claim that it’s roughly human level.
Furthermore, language isn’t just putting words together in the right order and with the right inflection. It also includes semantic information (what the actual meaning of the sentences is) and pragmatic information (is the language conveying what it is trying to convey, not just the literal meaning). I’m not sure whether pragmatics in particular would be relevant for AI risk, but the fact that anecdotally even GPT-4 is pretty bad at pragmatics prevents a literal interpretation of your statement.
In my opinion, the best evidence for GPT-4 not being human level at language is that, in the real world, GPT-4 is much cheaper than a human but consistently unable to outcompete humans. News organizations have a strong incentive to overhype GPT-caused automation, but the examples that they’ve found are mostly of people saying that either GPT-4 or GPT-3 (it’s not always clear which) did their job much worse than them, but good enough for clients. Take https://www.washingtonpost.com/technology/2023/06/02/ai-taking-jobs/ as a typical story.
Exams aren’t exactly the real world, but the popular example of GPT-4 doing well on exams is https://www.slowboring.com/p/chatgpt-goes-to-harvard. This both ignores citations (which is a very important part of college writing, and one that GPT-3 couldn’t do whatsoever and which GPT-4 still is significantly below what I would expect from a human) and relies on the false belief that Harvard is a hard school to do well at (grade inflation!)
I still agree with two big takeaways of your post, that an AI pause would be good and that we don’t necessarily need AGI for a good future, but that’s more because it’s robust to a lot of different beliefs about AI than because I agree with the evidence provided. Again, a lot of the evidence is stuff that I don’t feel particularly knowledgeable about, I picked this claim because I’ve had to think about it before and because it just feels false from my experience using GPT-4.