I kinda like that we’re back (so back?) to “a new movement called effective altruism”.
Tobias Häberli
Karma: 1,656
Important Update: Pivotal(TM) Trademark Announcement
I think the context of the Jack Clarke quote matters:
What if we’re right about AI timelines? What if we’re wrong?
Recently, I’ve been thinking a lot about AI timelines and I find myself wanting to be more forthright as an individual about my beliefs that powerful AI systems are going to arrive soon – likely during this Presidential Administration. But I’m struggling with something – I’m worried about making short-timeline-contingent policy bets.So far, the things I’ve advocated for are things which are useful in both short and long timeline worlds. Examples here include:
Building out a third-party measurement and evaluation ecosystem.
Encouraging governments to invest in further monitoring of the economy so they have visibility on AI-driven changes.
Advocating for investments in chip manufacturing, electricity generation, and so on.
Pushing on the importance of making deeper investments in securing frontier AI developers.
All of these actions are minimal “no regret” actions that you can do regardless of timelines. Everything I’ve mentioned here is very useful to do if powerful AI arrives in 2030 or 2035 or 2040 – it’s all helpful stuff that either builds institutional capacity to see and deal with technology-driven societal changes, or equips companies with resources to help them build and secure better technology.
But I’m increasingly worried that the “short timeline” AI community might be right – perhaps powerful systems will arrive towards the end of 2026 or in 2027. If that happens we should ask: are the above actions sufficient to deal with the changes we expect to come? The answer is: almost certainly not!
[Section that Mikhail quotes.]
Loudly talking about and perhaps demonstrating specific misuses of AI technology: If you have short timelines you might want to ‘break through’ to policymakers by dramatizing the risks you’re worried about. If you do this you can convince people that certain misuses are imminent and worthy of policymaker attention – but if these risks subsequently don’t materialize, you could seem like you’ve been Chicken Little and claimed the sky is falling when it isn’t – now you’ve desensitized people to future risks. Additionally, there’s a short- and long-timeline risk here where by talking about a specific misuse you might inspire other people in the world to pursue this misuse – this is bound up in broader issues to do with ‘information hazards’.
These are incredibly challenging questions without obvious answers. At the same time, I think people are rightly looking to people like me and the frontier labs to come up with answers here. How we get there is going to be, I believe, by being more transparent and discursive about these issues and honestly acknowledging that this stuff is really hard and we’re aware of the tradeoffs involved. We will have to tackle these issues, but I think it’ll take a larger conversation to come up with sensible answers.
In context Jack Clark seems to be arguing that he should be considering short timeline, ‘regretful actions’ more seriously.
Applications Open: Pivotal 2025 Q3 Research Fellowship
Some simplifying assumptions:
£50k starting net worth
Only employed for the next 4 years
£300k salary, £150k after tax, £110k after personal consumption
10% interest on your savings for 4 years
Around £635k at end of 2030
This is only slightly more than the average net worth of for UK 55 to 64 year olds.
Overall, if this plan worked out near perfectly, it would place you in around the 92 percentile of wealth in the UK.
This would put you in a good, but not great position to invest to give.
Overall it seems to me as if you’re trying to speedrun getting incredibly wealthy in 4 years. This is generally not possible with salaried work (the assumption above put you around the 99-99.5 percentile of salaries), but might be more feasible through entrepreneurship.
Some other considerations:Working in such a high paying job, even in financial services, will probably not allow you to study and practice investing. You will not be an expert on AI investing and investing in general in 2030, which would be a problem if you believe such expertise was necessary for you to invest to give.
Quite a lot of EAs will be richer than this in 2030. My rough guess is more than 500. Your position might be useful but is likely to be far from unique.
You might want to think through your uncertainties about how useful money will be to achieve your goals in 2030-2040. If there’s no more white collar jobs in 2030, in 2035 the world might be very weird and confusing.
If there is a massive increase of overall wealth in 2030-2040 due to fast technological progress, a lot of problems you might care about will get solved by non-EAs. Charity is a luxury good for the rich, more people will be rich, charity on average solves much more problems than it creates.
Technological progress itself will potentially solve a lot of the problems you care about.
(Also agree with Marcus’s point.)
Thanks for the comment! I might be missing something, but GPT-type chatbots are based on large language models, which play a key role in scaling toward AGI. I do think that extrapolating progress from them is valuable but also agree that tying discussions about future AI systems too closely to current models’ capabilities can be misleading.
That said, my post intentionally assumes a more limited claim: that AI will transform the world in significant ways relatively soon. This assumption seems both more likely and increasingly foreseeable. In contrast, assumptions about a world ‘incredibly radically’ transformed by superintelligence are less likely and less foreseeable. There have been lots of arguments around why you should work on AI Safety, and I agree with many of them. I’m mainly trying to reach the EAs who buy into the limited claim but currently act as if they don’t.
Regarding the example: It would likely be a mistake to focus only on current AI capabilities for education. However, it could be important to seriously evaluate scenarios like, ‘AI teachers better than every human teacher soon’.
Thanks for the thoughtful comment!
Re point 1: I agree that the likelihood and expected impact of transformative AI exist on a spectrum. I didn’t mean to imply certainty about timelines, but I chose not to focus on arguing for specific timelines in this post.Regarding the specific points: they seem plausible but are mostly based on base rates and social dynamics. I think many people’s views, especially those working on AI, have shifted from being shaped primarily by abstract arguments to being informed by observable trends in AI capabilities and investments.
One GHW example: The impact of AI tutoring on educational interventions (via Arjun Panickssery on LessWrong).
There have been at least 2 studies/impact evaluations of AI tutoring in African countries finding extraordinarily large effects:
Summer 2024 — 15–16-year olds in Nigeria
They had 800 students total. The treatment group studied with GPT-based Microsoft Copilot twice weekly for six weeks, studying English. They were just provided an initial prompt to start chatting—teachers had a minimal “orchestra conductor” role—but they achieved “the equivalent of two years of typical learning in just six weeks.”February–August 2023 — 8–14-year-olds in Ghana
An educational network called Rising Academies tested their WhatsApp-based AI math tutor called Rori with 637 students in Ghana. Students in the treatment group received AI tutors during study hall. After eight months, 25% of the subjects attrited from inconsistent school attendance. Of the remainder, the treatment group increased their scores on a 35-question assessment by 5.13 points versus 2.12 points for the control group. This difference was “approximately equivalent to an extra year of learning” for the treatment group.
Should this significantly change how excited EAs are about educational interventions? I don’t know, but I’ve also not seen a discussion of this on the forum (this post about MOOC & AI tutors that received ~zero engagement).
Thanks for adding this! I definitely didn’t want to suggest the list of reasons was exhaustive or that the division between the two ‘camps’ is clear-cut.
Preparing Effective Altruism for an AI-Transformed World
Exciting to hear about your growing team! I noticed the team page hasn’t been updated recently – are you able to share more about the new team members and their roles?
We’ve extended application to the 2025 Q1 Pivotal Research Fellowship!
We think this could be a great opportunity for many on the Forum!
Deadline: Tuesday 26. November.
Deadline Extended to Tuesday 26. November!
You can recommend others who may be a good fit. We’ll give you $100 for each accepted candidate we contact through you.
Since Longtermism as a concept doesn’t seem widely appealing, I wonder how other time-focused ethical frameworks fare, such as Shorttermism (Focusing on immediate consequences), Mediumtermism (Focusing on foreseeable future), or Atemporalism (Ignoring time horizons in ethical considerations altogether).
I’d guess these concepts would also be unpopular, perhaps because ethical considerations centered on timeframes feel confusing, too abstract or even uncomfortable for many people.
If true, it could mean that any theory framed in opposition, such as a critique of Shorttermism or Longtermism, might be more appealing than the time-focused theory itself. Critizising short-term thinking is an applause light in many circles.
Announcing the 2025 Q1 Pivotal Research Fellowship
Pivotal Research is Hiring Research Managers
You probably know much more about U.S. politics than I do, so I can’t engage deeply on whether these things are really happening or how unusual they might be.
However, I suspect that much of what you’re attributing to the Democratic party is actually due to a broader trend of U.S. elites becoming more left-leaning and Democrat-voting. Even if I agreed that this shift was bad for democracy, I’m not sure how voting for Trump would fix it in the long run. A Trump presidency would likely push elites even further toward left-leaning politics.
Regarding point 1.
You’re framing the situation as a choice between ‘Trump, who is willing to subvert democracy’ and ‘the Democratic Party, who is willing to subvert democracy’. This framing implicitly acknowledges that Harris is not (especially) willing to subvert democracy.It’s very plausible to believe that both the Democratic Party and the Republican Party are roughly equally willing to subvert democracy, especially given the significant influence Trump has on the Republican Party.
It then becomes a choice between:
Trump and the Republican Party, who are both willing to subvert democracy
vs.
The Democratic Party, who are willing to subvert democracy, and Harris, who is not.
In this comparison, Harris’s apparent commitment to democratic norms becomes the deciding factor in how you evaluate the overall democraticness of the choices.
But I feel also sad that the ideas have largely not slipped into the public consciousness over the last 14 years.