Yep! Something like this is probably unavoidable, and itâs what all of my examples below do (BERI, ACE, and MIRI).
sawyerđ¸
Karma: 1,120
There are many examples of organizations with high funding transparency, including BERI (which I run), ACE, and MIRI (transparency page and top contributors page).
I think this dynamic is generally overstated, at least in the existential risk space that I work in. Iâve personally asked all of our medium and large funders for permission, and the vast majority of them have given permission. Most of the funding comes from Open Philanthropy and SFF, both of which publicly announce all of their grantsâwhen recipients decided not to list those funders, itâs not because the funders donât want them to. There are many examples of organizations with high funding transparency, including BERI (which I run), ACE, and MIRI (transparency page and top contributors page).
Nonprofit organizations should make their sources of funding really obvious and clear: How much money you got from which grantmakers, and approximately when. Any time I go on some orgâs website and canât find information about their major funders, itâs a big red flag. At a bare minimum you should have a list of funders, and Iâm confused why more orgs donât do this.
I think people would say that the dog was stronger and faster than all previous dog breeds, not that it was âmore capableâ. Itâs in fact significantly less capable at not attacking its owner, which is an important dog capability. I just think the language of âcapabilityâ is somewhat idiosyncratic to AI research and industry, and Iâm arguing that itâs not particularly useful or clarifying language.
More to my point (though probably orthogonal to your point), I donât think many people would buy this dog, because most people care more about not getting attacked than they do about speed and strength.
As a side note, I donât see why preferences and goals change any of this. Iâm constantly hearing AI (safety) researchers talk about âcapabilities researchâ on todayâs AI systems, but I donât think most of them think those systems have their own preferences and goals. At least not in the sense that a dog has preferences or goals. I just think itâs a word that AI [safety?] researchers use, and I think itâs unclear and unhelpful language.
#taboocapabilities
What is âcapabilitiesâ? What is âsafetyâ? People often talk about the alignment tax: the magnitude of capabilities/âtime/âcost a developer loses by implementing an aligned/âsafe system. But why should we consider an unaligned/âunsafe system âcapableâ at all? If someone developed a commercial airplane that went faster than anything else on the market, but it exploded on 1% of flights, no one would call that a capable airplane.
This idea overlaps with safety culture and safety engineering and is not new. But alongside recent criticism of the terms âsafetyâ and âalignmentâ, Iâm starting to think that the term âcapabilitiesâ is unhelpful, capturing different things for different people.
I played the paperclips game 6-12 months before reading Superintelligence (which is what convinced me to prioritize AI x-risk), and I think the game made these ideas easier for me to understand and internalize.
This is truly crushing news. I met Marisa at a CFAR workshop in 2020. She was open, kind, and grateful to everyone, and it was joyful to be around her. I worked with her a bit revitalizing the EA Operations Slack Workspace in 2020, and had only had a few conversations with her since then, here and there at EA events. Marisa (like many young EAs) made me excited for a future that would benefit from her work, ambition, and positivity. Now sheâs gone. She was a good person, Iâm glad she was alive, and I am so sad sheâs gone.
Good reasoning, well written. Reading this post convinced me to join the next NYC protest. Unfortunately I missed the one literally two days ago because I waited too long to read this. But I plan to be there in September.
One thing I think is often missing from these sorts of conversations is that âalignment with EAâ and âalignment with my organizationâs missionâ are not the same thing! Itâs a mistake to assume that the only people who understand and believe in your organizationâs mission are members of the effective altruism community. EA ideas donât have to come in a complete package. People can believe that one organizationâs mission is really valuable and important, for different reasons, coming from totally different values, and without also believing that a bunch of other EA organizations are similarly valuable.
For âcore EAâ orgs like the Centre for Effective Altruism[1], thereâs probably near-total overlap between these two things. But for lots of other organizations the overlap is only incidental, and what you should really be looking for is âalignment with my organizationâs missionâ. Perceived EA Alignment is an unpredictable measure of that, while also being correlated with a bunch of other things like culture, thinking style, network, and socioeconomic status, each of which you either donât care about or which you donât want to be selecting for in the first place.
Within EA, work on x-risk is very siloed by type of threat: There are the AI people, the bio people, etc. Is this bad, or good?
Which of these is the correct analogy?
âBiology is to science as AI safety is to x-risk,â or
âImmunology is to biology as AI safety is to x-riskâ
EAs seem to implicitly think analogy 1 is correct: some interdisciplinary work is nice (biophysics) but most biologists can just be biologists (i.e. most AI x-risk people can just do AI).
The âexistential risk studiesâ model (popular with CSER, SERI, and lots of other non-EA academics) seems to think that analogy 2 is correct, and that interdisciplinary work is totally criticalâimmunologists alone cannot achieve a useful understanding of the entire system theyâre trying to study, and they need to exchange ideas with other subfields of medicine/âbiology in order to have an impact, i.e. AI x-risk workers are missing critical pieces of the puzzle when they neglect broader x-risk studies.
I agree with your last sentence, and I think in some versions of this itâs the vast majority of people. A lot of charity advertising seems to encourage a false sense of confidence, e.g. âFeed this child for $1,â or âadopt this manateeâ. I think this makes use of a near-universal human bias which probably has a name but which I am not recalling at the moment. For a less deceptive version of this, note how much effort AMF and GiveDirectly seem to have put in into tracking the concrete impact of your specific donation.
Building off of Jasonâs comment: Another way to express this is that comparing directly to the $5,500 Givewell bar is only fair for risk-neutral donors (I think?). Most potential donors are not really risk neutral, and would rather spend $5,001 to definitely save one life than $5,000 to have a 10% chance of saving 10 lives. Risk neutrality is a totally defensible position, but so is non-neutrality. Itâs good to have the option of paying a âpremiumâ for a higher confidence (but lower risk-neutral EV).
Leaving math mode...I love this post. It made me emotional and also made me think, and it feels like a really central example of what EA should be about. Iâm very impressed by your resolve here in following through with this plan, and Iâm really glad to have people like you in this community.
BERI ReÂturns All FTX Funds
BERIâs 2024 Goals and Predictions
BERI is seekÂing new trial collaborators
Very nice post. âAnarchists have no idolsâ strikes me as very similar to the popular anarchist slogan, âNo gods, no masters.â Perhaps the person who said it to you was riffing on that?
I think a simpler explanation for his bizarre actions is that he is probably the most stressed-out person on the face of the earth right now. Or heâs not seeing the situation clearly, or some combination of the two. Also probably sleep-deprived, struggling to get good advice from people around him, etc.
(This is not meant to excuse any of his actions or words, I think heâs 100% responsible for everything he says and does.)
From everything Iâve seen, GWWC has totally transformed under your leadership. And I think this transformation has been one of the best things thatâs happened in EA during that time. Iâm so thankful for everything youâve done for this important organization.