I currently lead EA funds.
Before that, I worked on improving epistemics in the EA community at CEA (as a contractor), as a research assistant at the Global Priorities Institute, on community building, and Global Health Policy.
Unless explicitly stated otherwise, opinions are my own, not my employer’s.
You can give me positive and negative feedback here.
One idea I’ve had to try and resolve this issue for donors is to have all private grants audited by a trusted animal welfare person who doesn’t work on the fund (e.g. Lewis Bollard) and commit to publishing their comments in payout reports. I think they’d be able to say things like “I agree that the private grants should be kept private and on average they were about as cost-effective as the public grants”.
I’ll take <agree> <disagree> votes to indicate how compelling this would be to readers.
Firstly, I’m sorry that you feel inadequate compared to people on the EA Forum or at EAGs. I think EA is a pretty weird community and it’s totally reasonable for people to not feel like it’s for them and instead try and do an ambitious amount of good outside the community.
I think this is somewhat orthogonal to feelings of rejection or the broader point that you are making about the higher impact potential of larger communities but I’ve personally felt that whilst EA seems to “care more” about people who are particularly smart, hardworking, and altruistic, it does a good job of giving people from various backgrounds an opportunity to participate—even if it’s differentially easier if you went to a top university.
For example, I think if someone with little or no reputation were to post a few top 10% of rethink priorities quality articles on important topics in fish welfare on the EA Forum they’d gain a lot of career capital and would almost overnight be on various organisation’s radars as someone to consider hiring (or at least be competitive in various application processes). I think that story is probably more true for AI safety. Contrast this with hiring for various hedge funds and consultancies which can be really hard to break into if you didn’t go to a small set of universities.
Thanks. Should now be fixed!
Thanks for the flag, we have had some turnover recently—will ask our dev to update the site!
The main difference in actions so far is that the ARM Fund has focussed on active grantmaking (e.g. in AI x information security fieldbuilding). In contrast, the LTFF has a more democratic and passive grantmaking focus. I also don’t think that ARM Fund has reached product market fit yet, it’s done a few things reasonably well but I don’t think it has a scalable product (unless we decide to do a lot more active grantmaking but so far that has been more opportunistic).
This fund was spun out of the Long-Term Future Fund (LTFF), which makes grants aiming to reduce existential risk. Over the last five years, the LTFF has made hundreds of grants, specifically in AI risk mitigation, totalling over $20 million. Our team includes AI safety researchers, expert forecasters, policy researchers, and experienced grantmakers. We are advised by staff from frontier labs, AI safety nonprofits, leading think tanks, and others.
More recently ARM Fund has been doing active grantmaking in AIS areas, we’ll likely write more about this soon. I expect the funds to become much more differentiated in staff in the next few months (though that’s not a commitment). Longer term, I’d like them to be pretty separate entities but for now they share roughly the same staff.
If you perceive any sort of downside from it, you can always remove it again.
Aren’t most of the downsides and upsides to norms hard to reverse (almost by definition)? Maybe you don’t think the upside is in getting other people to also participate in using the signal—but my read of the OP thinks that this is mostly about creating norms.
I’ve only skimmed this post, but I think this I agree with all of the main points. I would prefer EA meta orgs that provide benefits to people with money charged for some of their services. I do think the situation is significantly more complicated with orgs that receive substantial institutional funding so I think the original post applies a bit less to orgs like CEA, and more to specific EA groups or small-scale projects (including projects that the EAIF funds).
I suggested to various regional EA groups that they should try and cover some fraction of their costs from members, but there was quite a lot of negative push back (e.g. fundraising distracting them from their main jobs).[1]
EAIF is most interested in funding projects that shouldn’t be funded via regular markets, or might not be noticed as being especially valuable (e.g. many public goods in the nonexcludable and nonrivalrous sense).
@Harri Besceli feel free to push back on any of this if it conflicts with your impression of how the EAIF does/should work.
I still think some version of this is workable but it’s not a priority for the EAIF to figure out right now.
I think focussing on pledges of future income (if you are targeting students) seems great, most students don’t have much money and are also used to living on a much lower amount than they will in a few years after graduating (particularly people in engineering, CS, and math).
I’m aware of at least two efforts to run table top exercises on AI takeoff with decision makers so I don’t think this is particularly neglected, but I do think it’s valuable.
AW seems clearly more neglected in terms of funding inside the effective giving space and on net.
There is plenty of room for funding in the AW space, I would be surprised if $100M couldn’t be spent down over the next 10 years in the AW space at > 50% of the cost-effectiveness of the current marginal dollar (on average).
Most of my uncertainty comes from some credence that human lives are vastly more important than non-human animal lives, or medium-term growth being accelerated by GH interventions which could make GH work much more leveraged.
This is great! Thank you very much for writing this up. I’d be extremely excited for more local groups to self-fund retreats like this. I have seen similar events have large impacts on people’s goals/career choices/etc. and they seem pretty viable to do without a huge amount of planning/money.
A few things that come to mind that I appreciate in people’s applications:
apply to several funders where possible
try and point to a concrete plan (even if it’s basic)
talk about any tests you’ve done for your plan already (e.g. have you spent some time trying to upskill outside of a grant)
talk about why a grant is better than applying to a program/internship/job (or it could be that it’s worse, but you aren’t ready to do those alternatives yet)
try to talk about an end-to-end theory of change for your work—this is mostly about showing you’ve thought about how this project fits into a larger plan and you’re thinking strategically about your career
To be clear you don’t need to do any of these things to get funding, but I often find that applications are improved after people consider some of these bullet points.
simply become one of the most successful and influential ML researchers 🤷♂️
Idk, many of the people they are directing would just do something kinda random which an 80k rec easily beats. I’d guess the number of people for whom 80k makes their plans worse in an absolute sense is kind of low and those people are likely to course correct.
Otoh, I do think people/orgs in general should consider doing more strategy/cause prio research, and if 80k were like “we want to triple the size of our research team to work out the ideal marginal talent allocation across longtermist interventions” that seems extremely exciting to me. But I don’t think 80k are currently being irresponsible (not that you explicitly said that, for some reason I got a bit of that vibe from your post).
I think it’s worth noting that the two papers linked (which I agree are flawed and not that useful from an x-risk viewpoint)
I haven’t read the papers but I am surprised that you don’t think they are useful from an x-risk perspective. The second paper “A Model for Estimating the Economic Costs of Computer Vision Systems that use Deep Learning” seems highly relevant to forecasting AI progress which imo is one of the most useful AIS interventions.
The OP’s claim
This paper has many limitations (as acknowledged by the author), and from an x-risks point of view, it seems irrelevant.
Seems overstated and I’d guess that many people working on AI safety would disagree with them.
Great post—I really enjoyed reading this.
I would have thought the standard way to resolve some of the questions above would be to use a large agent-based model, simulating disease transmission among millions of agents and then observing how successful some testing scheme is within the model (you might be able to backtest the model against well-documented outbreaks).
I’m not sure how much you’d trust these models over your intuitions, but I’d guess they’d have quite a lot of mileage.
I’ve only skimmed these papers, but these seem promising and illustrative of the direction to me:
The first two bullets don’t seem like small UI changes to me; the second, in particular, seems too adversarial imo.
Fwiw, I don’t think that being on the 80k podcast is much of an endorsement of the work that people are doing. I think the signal is much more like “we think this person is impressive and interesting”, which is consistent with other “interview podcasts” (and I suspect that it’s especially true of podcasts that are popular amongst 80k listeners).
I also think having OpenAI employees discuss their views publicly with smart and altruistic people like Rob is generally pretty great, and I would personally be excited for 80k to have more OpenAI employees (particularly if they are willing to talk about why they do/don’t think AIS is important and talk about their AI worldview).
Having a line at the start of the podcast making it clear that they don’t necessarily endorse the org the guest works for would mitigate most concerns—though I don’t think it’s particularly necessary.
If the survey had framed the same questions in multiple ways for higher reliability or had some kind of consistency checking* I would trust that respondents endorsed their numbers more. Not necessarily saying this is a good trade to make as it would increase the length of the survey.
*e.g., asking separately in different parts of the survey about the impact of: • Animal welfare $ / Global health $ • Global health $ / AI $ • Animal welfare $ / AI $
…and then checking if the responses are consistent across all sections.