Currently doing local AI safety Movement Building in Australia and NZ.
Chris Leong
Karma: 6,198
It knows the concept of cruxes? I suppose that isn’t that surprising in retrospect.
This is a great project idea!
In case you were wondering, Secure Software Development Practices for Generative AI and Dual-Use Foundation Models completely avoids any discussion of the fact that releasing a dual-use model could potentially be dangerous or that the impacts of any such models should be evaluated before use. This is a truly stunning display of ball-dropping.
Update: I just checked NIST AI 600-1 as well: the report is extremely blaise about CBRN hazards from general AI (admitting though that chemical and biological design tools might pose risks to society or national security”). They quote the RAND report that claims the current generation doesn’t pose any such risks beyond web search, neglecting to mention that these results only applied to the release of a model over an API. As far as they’re concerned, these risks just need to be “carefully monitored”.
Okay, I guess parts of that framework make a bit more sense now that you’ve explained it.
At the same time, it feels that people can always decide to earn to give if they fail to land an EA-relevant gig, so I’m not sure why you’re modeling it as a $5k annual donation vs. a one-time $5k donation for someone spending a year focusing on upskilling for EA roles. Maybe you could add an extra factor for the slowdown in their career advancement, but $50k extra per year is unrealistic.
I think it’s also worth considering that there are selection effects here. So insofar as EA promotes direct work, people with higher odds of being successful in landing a direct work position are more likely to pursue that and people with better earn-to-give potential are less likely to take the advice.
Additionally, I wonder whether the orgs you surveyed understood ten additional applications as ten additional average applications or ten additional applications from EA’s (more educated and valued-aligned than the general population) who were dedicated enough to actually follow through on earning to give.
My point was that presumably the org thinks they’re better if they decide to hire them as opposed to the next best person.
Bob sees little reason to reconsider the trade-off, especially since ChatGPT seems to have vindicated 80,000 hours’ prior belief that AI was going to be a big deal
ChatGPT is just the tip of the iceberg here.
GPT4 is significantly more powerful than 3.5. Google now has a multi-modal model that can take in sound, images and video and a context window of up to a million tokens. Sora can generate amazing realistic videos. And everyone is waiting to see what GPT5 can do.
Further, the Center for AI Safety open letter has demonstrated that it isn’t just our little community that is worried about these things, but a large number of AI experts.Their ‘AI is going to be a big thing’ bet seems to have been a wise call, at least at the current point in time. Of course, I’m doing AI Safety movement building, so I’m a bit biased here, and maybe we’ll think differently down the line, but right now they’re clearly ahead.
For 1: A lot of global health and development is much less talent-hungry than animal welfare work or x-risk work. Take for example the Against Malaria Foundation. They receive hundreds of millions of dollars, but they only have a core team of 13. Sure you need a bunch of people to hand out bed nets, but the requirements for that aren’t that tight and sure you need some managers, but lots of people are capable of handling this kind of logistics, you don’t really have to headhunt them. I suppose this could change if there was more of a pivot into policy where talent really matters. However, in that case, you would probably want people from the country whose policy you want to influence, moreso then thinking about cost.
For 5: It’s not clear to me that the way you’re thinking about this makes sense to me. If you’re asking about the trade-off between direct work and donations, it seems as though we should ask about $5k from the job vs. a new candidate who is better than your current candidate as, in a lot of circumstances, they will have the option of doing earn to give so long as they don’t take an EA job (I suppose there is the additional factor of how much chasing EA jobs detracts from chasing earn to give jobs).
Very interesting report. It provided a lot of visibility into how these funders think.
Geographically, India may be a standout opportunity for getting talent to do research/direct work in a counterfactually cheap way.
I would have embraced that more in the past, but I’m a lot more skeptical of this these days and I think that if it were possible then it would have worked out. For many tasks, EA wants the best talent that is available. Top talent is best able to access overseas opportunities and so the price is largely independent of current location.
In terms of age prioritization, it is suboptimal that EA focuses more on
outreach to university students or young professionals as opposed to mid-career
people with greater expertise and experience.I agree that there is a lot of alpha in reaching out to mid-career professionals—if you are able to successfully achieve this. This work is a lot more challenging—mid-career professionals are often harder to reach, less able to pivot and have less time available to upskill. Less people are able to do this kind of outreach because these professionals may take current students or recent grads less seriously. So for a lot of potential movement builders, focusing on students or young professionals is a solid play because it is the best combination of impact and personal fit.
The report writes “80,000 Hours in particular, has been singled out for its work not having a clear positive impact, despite the enormous sums they have spent.
As an on-the-ground community builder, I’m skeptical of your take. So many people that I’ve talked to became interested in EA or AI safety through 80,000 Hours. Regarding: “misleading the community that it is cause neutral while being almost exclusively focused on AI risk”, I was more concerned about this in the past, but I feel that they’re handling this pretty well these days. Would be quite interested to hear if you had specific ways you thought they should change. Regarding, causing early career EA’s to choose suboptimal early career jobs that mess up their CVs, I’d love to hear more detail on this if you can. Has anyone written up a post on this?
On funding Rethink Priorities specifically – a view floated is that there is value in RP as a check on OP, and yet if OP doesn’t think RP is worth funding beyond a certain point, it’s hard to gainsay them
Rethink Priorities seems to be a fairly big org—especially taking into account that it operates on the meta-level—so I understand why OpenPhil might be reluctant to spend even more money there. I suspect that there’s a feeling that even if they are doing good work, that meta work should only be a certain portion of their budge. I wouldn’t take that as a strong signal.
In practice, it’s unclear if the community building actually translates to money/talent moved, as opposed to getting people into EA socially.
As someone on the ground, I can say that community building has translated to talent moved at the very least. Money moved is less visible to me because a lot of people aren’t shouting about the pledges from the rooftops, plus the fact that donations are very heavy-tailed. Would love to hear some thoughtful discussion of how this could be better measured.
What would interest OP might be a project about getting in more people who are doing the best things in GHW (e.g. Buddy Shah, Eirik Mofoss).
Very curious about what impressed Open Philanthropy about these people since I’m not familiar with their work. Would be keen to learn more though!
At least from an AI risk perspective, it’s not at all clear to me that this would improve things as it would lead to a further dispersion of this knowledge outward.
For anyone wondering about the definition of macrostrategy, the EA forum defines it as follows:
Macrostrategy is the study of how present-day actions may influence the long-term future of humanity.[1]
Macrostrategy as a field of research was pioneered by Nick Bostrom, and it is a core focus area of the Future of Humanity Institute.[2] Some authors distinguish between “foundational” and “applied” global priorities research.[3] On this distinction, macrostrategy may be regarded as closely related to the former. It is concerned with the assessment of general hypotheses such as the hinge of history hypothesis, the vulnerable world hypothesis and the technological completion conjecture; the development of conceptual tools such as the concepts of existential risk, of a crucial consideration and of differential progress; and the analysis of the impacts and capabilities of future technologies such as artificial general intelligence, whole brain emulation and atomically precise manufacturing, but considered at a higher level of abstraction than is generally the case in cause prioritization research.
If EA is trying to do the most good, letting people like Ives post their misinformed stuff here seems like a clear mistake.
Disagree because it is at −36.
Happy to consider your points on the merits if you have an example of an objectionable post with positive upvotes.
That said: part of me feels that Effective Altruism shouldn’t be afraid of controversial discussion, whilst another part of me wants to shift it to Less Wrong. I suppose I’d have to have a concrete example in front of me to figure out how to balance these views.
I didn’t vote, but maybe people are worried about the EA forum being filled up with a bunch of logistics questions?
Thank you for your service.
This post makes some interesting points about EA’s approach to philanthropy, but I certainly have mixed feelings on “please support at least one charity run by someone in the global south that just so happens to be my own”.
Might be more useful if you explain why the arguments weren’t persuasive to you
So my position is that most of your arguments are worth some “debate points” but that mitigating potential x-risks outweigh this.Our interest is in a system of liability that can meet AI safety goals and at the same time have a good chance of success in the real world
I’ve personally made the mistake of thinking that the Overton Window is narrower than it actually was in the past. So even though such laws may not seem viable now, my strong expectation is that it will quickly change. At the same time, my intuition is that if we’re going to pursue the liability route, at least strict liability has the advantage of keeping the developer focused on preventing the issue from occurring rather than taking actions to avoid legal responsibility. Those actions won’t help, so they need to focus on preventing the issue from occurring.
I know that I wrote above:
In any case my main worry about strong liability laws is that we may create a situation where AI developers end up thinking primarily about dodging liability more than actually making the AI safe.
and that this is in tension with what I’m writing now. I guess upon reflection I now feel that my concerns about strong liability laws only apply to strong fault-based liability laws, not to strict liability laws, so in retrospect I wouldn’t have included this sentence.
Regarding your discussion in point 1 - apologies for not addressing this in my initial reply—I just don’t buy that courts being able to handle chainsaws or medical or actuary evidence means that they’re equipped to handle transformative AI given how fast the situation is changing and how disputed many of the key questions are. Plus the stakes involved play a role in me not wanting to take a risk here/make an unnecessary bet on the capabilities of the courts. Even if there was a 90% chance that the courts would be fine, I’d prefer to avoid the 10% probability that they aren’t.
I find the idea of a reverse burden of proof interesting, but tbh I wasn’t really persuaded by the rest of your arguments. I guess the easiest way to respond to most of them would be “Sure, but human extinction kind of outweighs it” and then you’d reraise how these risks are abstract/speculative and then I’d respond that putting risks in two boxes, speculative and non-speculative, hinders clear thinking more than it helps. Anyway, that’s just how I see the argument play out.
In any case my main worry about strong liability laws is that we may create a situation where AI developers end up thinking primarily about dodging liability more than actually making the AI safe.
I have very mixed views on Richard Hannania.
On one hand, some of his past views were pretty terrible (even though I believe that you’ve exaggerated the extent of these views).On the other hand, he is also one of the best critics of conservatives. Take for example, this article where he tells conservatives to stop being idiots who believe random conspiracy theories and another where he tells them to stop scamming everyone. These are amazing, brilliant articles with great chutzpah. As someone quite far to the right, he’s able to make these points far more credibly than a moderate or liberal ever could.
So I guess I feel he’s kind of a necessary voice, at least at this particular point in time when there are few alternatives.
Yeah, it’s possible I’m taking a narrow view of what a professional organisation is. I don’t have a good sense of the landscape here.
I guess I’m a bit skeptical of this proposal.
I don’t think we’d have much credibility as a professional organisation. We could require people to do the intro and perhaps even the advanced fellowship, but that’s hardly rigorous training.
I’m worried that trying to market ourselves as a professional organisation might backfire if people end up seeing us as just a faux one.
I suspect that this kind of association might be more viable for specific cause areas than for EA as a whole, but there might not be enough people except in a couple of countries.
I’m confused. Don’t you already have a second building? Is that dedicated towards events or towards more guests?