(I am mostly thinking about the AI section, and disagree with your categorization there:
NO: Developed RLHF, a technique for controlling AI output widely considered the key breakthrough behind ChatGPT
Yep, agree with a NO here
NO: …and other major AI safety advances, including RLAIF and the foundations of AI interpretability10.
Yep, agree with a NO here
NO: Founded the field of AI safety, and incubated it from nothing up to the point where Geoffrey Hinton, Yoshua Bengio, Demis Hassabis, Sam Altman, Bill Gates, and hundreds of others have endorsed it and urged policymakers to take it seriously.
I think this should be a YES. This is clearly about ending up in an influential position in a field.
NO: Helped convince OpenAI to dedicate 20% of company resources to a team working on aligning future superintelligences.
This should also be a YES. This is really quite centrally about EAs ending up with more power and influence.
NO: Gotten major AI companies including OpenAI to work with ARC Evals and evaluate their models for dangerous behavior before releasing them.
This should also be a YES. Working with AI companies is about power and influence (which totally might be used for good, but it’s not an intellectual achievement).
YES: Got two seats on the board of OpenAI, held majority control of OpenAI for one wild weekend, and still apparently might have some seats on the board of OpenAI, somehow?12
Agree
YES: Helped found, and continue to have majority control of, competing AI startup Anthropic, a $30 billion company widely considered the only group with technology comparable to OpenAI’s.13
Agree
YES: Become so influential in AI-related legislation that Politico accuses effective altruists of having “[taken] over Washington” and “largely dominating the UK’s efforts to regulate advanced AI”.
Agree
NO: Helped (probably, I have no secret knowledge) the Biden administration pass what they called “the strongest set of actions any government in the world has ever taken on AI safety, security, and trust.”
What we have seen here so far is institutes being founded and funding being promised, with some very extremely preliminary legislation that might help. Most of this achievement is about ending up with people in positions of power. So should be a YES.
NO: Helped the British government create its Frontier AI Taskforce.
This seems like a clear YES? The task force is very centrally putting EAs and people concerned about safety in power. No legislation has been passed.
NO: Won the PR war: a recent poll shows that 70% of US voters believe that mitigating extinction risk from AI should be a “global priority”.
Agree
Overall, for AI in-particular, I count 8⁄11. I think some of these are ambiguous or are clearly relevant for more than just being in power, but this list of achievements is really quite hugely weighted towards measuring the power that EA and AI Safety have achieved as a social movement, and not its achievements towards making AI actually safer.
I think basically all of our disagreements here have the same form, so lets just focus on the first one:
NO: Founded the field of AI safety, and incubated it from nothing up to the point where Geoffrey Hinton, Yoshua Bengio, Demis Hassabis, Sam Altman, Bill Gates, and hundreds of others have endorsed it and urged policymakers to take it seriously.
I think this should be a YES. This is clearly about ending up in an influential position in a field.
To my ear, this is not a claim about becoming influential. It is a claim about creating something in the past, and getting a bunch of people to agree. It’s similar to saying “Karl Marx founded Communism and got many people to agree”—the claim is not that Marx is in an influential position (he cannot be, since he is dead) but that he created something. There are related claims that are about influence, like if it had instead said “Founded and remain influential in the field”, but the actual specific claim here is fully consistent with the EAs birthing and then seeing their relevance come to an end. The thing that the hundreds of people endorsed wasn’t the EA movement itself, or even specific EAs, but the abstract claim that AI is risky.
We might mostly be arguing about semantics. In a similar discussion a few days ago I was making the literal analogy of “if you were worried about EA having bad effects on the world via the same kind of mechanism as the rise of communism, a large fraction of the things under the AI section should go into the ‘concern’ column, not the ‘success’ column”. Your analogy with Marx illustrates that point.
I do disagree with your last sentence. The thing that people are endorsing is very much both a social movement as well as some object level claims. I think it differs between people, but there is a lot of endorsing AI Safety as a social movement. Social proof is usually the primary thing evoked these days in order to convince people.
(I am mostly thinking about the AI section, and disagree with your categorization there:
Overall, for AI in-particular, I count 8⁄11. I think some of these are ambiguous or are clearly relevant for more than just being in power, but this list of achievements is really quite hugely weighted towards measuring the power that EA and AI Safety have achieved as a social movement, and not its achievements towards making AI actually safer.
I think basically all of our disagreements here have the same form, so lets just focus on the first one:
To my ear, this is not a claim about becoming influential. It is a claim about creating something in the past, and getting a bunch of people to agree. It’s similar to saying “Karl Marx founded Communism and got many people to agree”—the claim is not that Marx is in an influential position (he cannot be, since he is dead) but that he created something. There are related claims that are about influence, like if it had instead said “Founded and remain influential in the field”, but the actual specific claim here is fully consistent with the EAs birthing and then seeing their relevance come to an end. The thing that the hundreds of people endorsed wasn’t the EA movement itself, or even specific EAs, but the abstract claim that AI is risky.
We might mostly be arguing about semantics. In a similar discussion a few days ago I was making the literal analogy of “if you were worried about EA having bad effects on the world via the same kind of mechanism as the rise of communism, a large fraction of the things under the AI section should go into the ‘concern’ column, not the ‘success’ column”. Your analogy with Marx illustrates that point.
I do disagree with your last sentence. The thing that people are endorsing is very much both a social movement as well as some object level claims. I think it differs between people, but there is a lot of endorsing AI Safety as a social movement. Social proof is usually the primary thing evoked these days in order to convince people.