I’m a computational physicist, I generally donate to global health. I am skeptical of AI x-risk and of big R Rationalism, and I intend explaining why in great detail.
titotal
Karma: 7,656
This again seems like another “bubble” thing. The vast majority of conservatives do not draw a distinction between USAID and foreign aid in general. And I would guess they do associate foreign aid with “woke”, because “woke” is a word that is usually assigned based on vibes alone, for the things perceived as taking away from the average american to give to some other minority. Foreign aid involves spending american money to help foreigners, it’s absolutely perceieved as “woke”.
Look, I wish we lived in a world where people were rational and actually defined their terms and made their decisions accordingly, but that’s not the world we live in.
The original post is only 700 words, and this is like half that length. Can you not give people the dignity of reading their actual parting words?
I don’t think foreign aid is at risk of being viewed as woke. Even the conservative criticisms of USAID tend to focus on things that look very ideological and very not like traditional foreign aid.
This just isn’t true. Yes, exaggerated claims of “wastefulness” are one of the reasons they are against it, but there are many more who are ideologically opposed to foreign aid altogether.
I can link you to this exchange I had with a conservative, where they explictly stated that saving the lives of a billion foreigners would not be worth increasing the national deficit by 4%, because they are ideologically opposed to american taxpayer money saving foreign lives, no matter how efficiently they do it. Or see the insanely aggressive responses to this seemingly innocuous scott alexander tweet. Or here is a popular right wing meme specifically mocking liberals for having large moral circles.
I suspect that you are in a bubble, where the conservatives you know are fine with foreign aid, so you extend that to the rest of conservatives. But in a broader context, 73% of republicans want to cut foreign aid, while only 33% of democrats do.
I think the term “goodharting” is great. All you have to do is look up goodharts law to understand what is talked about: the AI is optimising for the metric you evaluated it on, rather than the thing you actually want it to do.
Your suggestions would rob this term of the specific technical meaning, which makes thing much vaguer and harder to talk about.
I think employers should be obligated to have at least a kitchen area where people can heat up or make their own food. People should not be forced to buy lunch.
I don’t mind you using LLMs for elucidating discussion, although I don’t think asking it to rate arguments is very valuable.
The additional details of having subfield specific auditors that are opt-in does lessen my objections significantly. Of course, the issue of what counts as a subfield is kinda thorny. It would make most sense for, as claude suggests, journals to have an “auditor verified” badge, but then maybe you’re giving too much power over content to the journals, which usually stick to accept/reject decisions (and even that can get quite political).
Coming back to your original statement, ultimately I just don’t buy that any of this can lead to “incredibly low rates of fraud/bias”. If someone wants to do fraud or bias, they will just game the tools, or submit to journals with weak/nonexistent auditors. Perhaps the black box nature of AI might even make it easier to hide this kind of thing.
Next: there are large areas of science where a tool telling you the best techniques to use will never be particularly useful. On the one hand there is research like mine, where it’s so frontier that the “best practices” to put into such an auditor don’t exist yet. On the other, you have statistics stuff that is so well known that there already exist software tools that implement the best practices: you just have to load up a well documented R package. What does an AI auditor add to this?
If I was tasked with reducing bias and fraud, I would mainly push for data transparency requirements in journal publications, and in beefing up the incentives for careful peer review, which is currently unpaid and unrewarding labour. Perhaps AI tools could be useful in parts of that process, but I don’t see it as anywhere near as important than those other two things.
My obvious answer is that the auditors should be held up to higher standards than the things they are auditing. This means that these should be particularly open, and should be open to other auditing. For example, the auditing code could be open-source, highly tested, and evaluated by both humans and AI systems.
Yeah, I just don’t buy that we could ever establish such a code in a way that would make it viable. Science chases novel projects and experiments, what is “meant” to happen will be different for each miniscule subfield of each field. If you release an open source code that has been proven to work for subfields A,B,C,D,E,F, someone in subfield G will immediately object that it’s not transferable, and they may very well be right. And the only people who can tell if it works on subfield G is people who are in subfield G.
You cannot avoid social and political aspects to this: Imagine if the AI-auditor code starts declaring that a controversial and widely used technique in, say, evolutionary psychology, is bad science. Does the evo-psych community accept this and abandon the technique, or do they say that the auditor code is flawed due to the biases of the code creators, and fork/reject the code? Essentially you are allowing whoever is controlling the auditor code to suppress fields they don’t agree with. It’s a centralization of science that is at odds with what allows science to actually work.
I don’t think Eliezer Yudkowsky and the rationalists should be throwing stones here. Sam Altman himself claimed that “eliezer has IMO done more to accelerate AGI than anyone else”. They’ve spent decades trying to convince people of the miraculous powers of AI, and now are acting shocked that this motivated people to try and build it.
As a working scientist, I strongly doubt that any of this will happen.
First, existing AI’s are nowhere near being able to do any of the things with an accuracy that makes them particularly useful. AI’s are equipped to do things similar to their training set, but all science is on the frontier: it is a much harder task to figure out the correct experimental setup for something that has never been done before in the history of humanity.
Right now I’m finishing up an article about how my field acually uses AI, and it’s nothing like anything you proposed here: LLMs are used for BS grant applications and low-level coding, almost exclusively. I don’t find it very useful for anything else.
The bigger issue here is with the “auditors” themselves: who’s in charge of them? If a working scientist disagrees with what the “auditor” says, what happens? What happens if someone like Elon is in charge, and decides to use the auditors for a political crusade against “woke science”, as is currently literally happening right now?
Catching errors in science is not something that can be boiled down to a formula: a massive part of the process is socio-cultural. You push out AI auditors, people are just going to game them, like they have with p-values, etc. This is not a problem with a technological solution.
If I’m reading this right, there was a ~40% drop in the number of respondents in 2024 compared to 2022?
I think this gives cause to be careful when interpreting the results: for example, from the graphs it might look like EA has succesfully recruited a few more centre-left people to the movement: but in absolute terms the number of centre leftists responding to the survey has decreased by about 300 people.
I think the decline in left identifying people is unlikely to be a case of mass political conversion: I think it’s more likely that a large number of leftists have left the movement due to it’s various recent scandals.
Thank you for laying that out, that is elucidatory. And behind all this I guess is the belief that if we don’t suceed in “technical alignment”, the default is that the AI will be “aligned” to an alien goal, the pursuit of which will involve humanities disempowerment or destruction? If this was the belief, I could see why you would find technical alignment superior.
I, personally, don’t buy that this will be the default: I think the default will be some shitty approximation of the goals of the corporation that made it, localised mostly to the scenarios it was trained in. From the point of view of someone like me, technical alignment actually sounds dangerous to pursue: it would allow someone to imbue an AI with world domination plans and potentially actually succeed.
I think EA has been taken in too far by “mistake theory”, with the idea that surely everyone values saving lives, they just disagree with each other on how to do it, and if we just explain that PEPFAR saves lives to the right people, they’ll change their minds.
But like… look at the ridiculously hostile replies to this tweet by Scott Alexander. There is an influential section of the Right that is ideologically against any tax money going to help save non-american lives, and this section appears to be currently in charge of the US government. These people cannot be reasoned out of their positions: instead the only path is to rip them away from power and influence. These anti-human policies must be hung over the head of the Republican party, and they must bleed politically for them: so that future politicians are warned away from such cruelty.
I think this would be a useful taxonomy to use when talking about the subject. Part of the problem seems to be that different people are using the same term to mean different things: which is not unsurprising when the basis is an unprecise and vague idea like “align AI to human or moral goals” (which humans? Which morals?).
I get the impression that Yud and company are looking for a different kind of alignment: where the AI is aligned to a moral code, and will disobey both the company making the model and the end user if they try to make it do something immoral.
Epistemologically speaking, it’s just not a good idea to have opinions relying on the conclusions of a single organization, no matter how trustworthy it is.
EA in general does not have very strong mechanisms for incentivising fact-checking: the use of independent evaluators seems like a good idea.
I assume they saw it at low karma. The first internet archive snapshot of this page had it at −4 karma.
I don’t think it’s “politically wise” to be associated with someone like Musk who is increasingly despised worldwide, especially among the educated, intelligent population that is EA’s primary recruitment ground. This goes quintuple for agreed upon racists like Hanania.
Elon has directly attacked every value I hold dear, and has directly screwed over life-saving aid to the third world. He is an enemy of effective altruist principles, and I don’t think we should be ashamed to loudly and openly say so.
A) there is no concrete proof that ASI is actually on the near-term horizon.
B) There is no concrete proof that if “uncontrolled” ASI is made, it is certain to kill us.
C) There is no concrete proof that the US and china will be equally bad if obtaining ASI. We have limited information as to what each country will look like decades in the future.
Many outlets don’t take the possibility of rapid AI development seriously, treating AGI discussions as mere marketing hype.
I think it would be a huge mistake to condition support for AI journalism on object level views like this. Being skeptical of rapid AI development is a perfectly valid opinion to have: and I think it’s pretty easy to make a case that the actions of some AI leaders don’t align with their words. Both of the articles you linked seem perfectly fine and provide evidence for their views: you just disagree with the conclusions of the authors.
If you want journalism to be accurate, you can’t prematurely cut off the skeptical view from the conversation. And I think skeptical blogs like Pivot-to-AI do a good job at compiling examples of failures, harms, and misdeployments of AI systems: if you want to build a coalition against harms from AI, excluding skeptics is a foolish thing to do.
I have not seen a lot of evidence that EA skills are very transferable to the realm of politics. As counterexamples, look at the botched Altman ouster, or the fact that AI safety people ended up helping start an AI arms race: these partially seem to come from a place of poor political instincts. EA is also disproportionately STEM background, which are generally considered comparatively poor at people skills (accurately, in my experience).
I think combating authoritarianism is important, but EA would probably be better off identifying other people who are good at politics and sending support their way.
1-4 is only unreasonable because you’ve written a strawman version of 4. Here is a version that makes total sense:
1. You make a superficially compelling argument for invading Iraq
2. A similar argument, if you squint, can be used to support invading Vietnam
3. This argument for invading vietnam was wrong because it made mistakes X, Y, and Z
4. Your argument for invading Iraq also makes mistakes X, Y and Z
5. Therefore, your argument is also wrong.
Steps 1-3 are not strictly necessary here, but they add supporting evidence to the claims.
As far as I can tell from the article, they are saying that you can make a counting argument that argues that it’s impossible to make a working SGD model. They are using this a jumping off point to explain the mistakes that would lead to flawed counting arguments, and then they spend the rest of the article trying to prove that the AI misalignment counting argument is making these same mistakes.
You can disagree with whether or not they have actually proved that AI misalignment made a comparable mistake, but that’s a different problem to the one you claim is going on here.