Hi!
I’m currently (Aug 2023) a Software Developer at Giving What We Can, helping make giving significantly and effectively a social norm.
I’m also a forum mod, which, shamelessly stealing from Edo, “mostly means that I care about this forum and about you! So let me know if there’s anything I can do to help.”
Please have a very low bar for reaching out!
I won the 2022 donor lottery, happy to chat about that as well
As a side point, I’m confused when you say:
I don’t think they suggest that, depending on your definition of “strong”. Just above the sceenshotted quote, the article mentions that many early investors were at the time linked to EA.
That was said by the author of the article who was trying to make the point that there is a link between Anthropic and EA. So I don’t see this as evidence of Anthropic being forthcoming.
I think in the context of the article, their quotes (44 words in total) make more sense:
In that context, the quotes clarify that Anthropic is not an “EA company”, and give a more accurate understanding of the relationship to the reader.
A more in-depth analysis of the historical affiliations, separations, agreements, and disagreements of Anthropic’s funders, founders, and employees with various parts of EA over the past 15 years would take far more than two paragraphs.
If I didn’t know anything about Anthropic and I read the words “I definitely have met people here who are effective altruists, but it’s not a theme of the organization or anything”, I might think Anthropic is like Google where you may occasionally meet people in the cafeteria who happen to be effective altruists but EA really has nothing to do with the organisation.
You wouldn’t think that in the context of the article, though.
I would not get the impression that many of the employees are EAs who work at Anthropic or work on AI safety for EA reasons. And that the three members of the trust they’ve given veto power over the company to have been heavily involved in EA.
I don’t know what percentage of Anthropic employees consider themselves part of the EA community. Also, I don’t agree that it’s clear that Evidence Action’s CEO is part of the effective altruism community because evidence action received money from GiveWell.
https://www.linkedin.com/in/kanika-bahl-091a936/details/experience/ She was working in global health since before effective altruism was a thing, and many/most people funded by OpenPhilanthropy don’t consider themselves part of the community. In the same way that charities funded by Catholic donors are not necessarily Catholic. It does seem that OpenPhilanthropy was their main source of funding for many years though, which makes the link stronger than I originally thought.
I think the people in the article you quote are being honest about not identifying with the EA social community, and the EA community on X is being weird about this.
I think the confusion might stem from interpreting EA as “self-identifying with a specific social community” (which they claim they don’t, at least not anymore) vs EA as “wanting to do good and caring about others” (which they claim they do, and always did)
Going point by point:
Dario, Anthropic’s CEO, was the 43rd signatory of the Giving What We Can pledge and wrote a guest post for the GiveWell blog. He also lived in a group house with Holden Karnofsky and Paul Christiano at a time when Paul and Dario were technical advisors to Open Philanthropy.
This was more than 10 years ago. EA was a very different concept / community at the time, and this is consistent with Daniela Amodei saying that she considers it an “outdated term”
Amanda Askell was the 67th signatory of the GWWC pledge.
This was also more than 10 years ago, and giving to charity is not unique to EA. Many early pledgers don’t consider themselves EA (e.g. signatory #46 claims it got too stupid for him years ago)
Many early and senior employees identify as effective altruists and/or previously worked for EA organisations
Amanda Askell explicitly says “I definitely have met people here who are effective altruists” in the article you quote, so I don’t think this contradicts it in any way
https://x.com/AmandaAskell/status/1905995851547148659
Anthropic has hired a “model welfare lead” and seems to be the company most concerned about AI sentience, an issue that’s discussed little outside of EA circles.
That’s false: https://en.wikipedia.org/wiki/Artificial_consciousness
On the Future of Life podcast, Daniela said, “I think since we [Dario and her] were very, very small, we’ve always had this special bond around really wanting to make the world better or wanting to help people” and “he [Dario] was actually a very early GiveWell fan I think in 2007 or 2008.”
The Anthropic co-founders have apparently made a pledge to donate 80% of their Anthropic equity (mentioned in passing during a conversation between them here and discussed more here)Their first company value states, “We strive to make decisions that maximize positive outcomes for humanity in the long run.”
Wanting to make the world better, wanting to help people, and giving significantly to charity are not prerogatives of the EA community.
It’s perfectly fine if Daniela and Dario choose not to personally identify with EA (despite having lots of associations) and I’m not suggesting that Anthropic needs to brand itself as an EA organisation
I think that’s exactly what they are doing in the quotes in the article: “I don’t identify with that terminology” and “it’s not a theme of the organization or anything”
But I think it’s dishonest to suggest there aren’t strong ties between Anthropic and the EA community.
I don’t think they suggest that, depending on your definition of “strong”. Just above the sceenshotted quote, the article mentions that many early investors were at the time linked to EA.
I think it’s a bad look to be so evasive about things that can be easily verified (as evidenced by the twitter response).
I don’t think X responses are a good metric of honesty, and those seem to be mostly from people in the EA community.
In general, I think it’s bad for the EA community that everyone who interacts with it has to worry about being liable for life for anything the EA community might do in the future.
I don’t see why it can’t let people decide if they want to consider themselves part of it or not.
As an example, imagine if I were Catholic, founded a company to do good, raised funding from some Catholic investors, and some of the people I hired were Catholic. If 10 years later I weren’t Catholic anymore, it wouldn’t be dishonest for me to say “I don’t identify with the term, and this is not a Catholic company, although some of our employees are Catholic”. And giving to charity or wanting to do good wouldn’t be gotchas that I’m secretly still Catholic and hiding the truth for PR reasons. And this is not even about being a part of a specific social community.
GiveWell posts a lot of interesting stuff on their blog and on their website, but in the past year they only reposted hiring announcements on the EA Forum.
E.g. I don’t think that USAID Funding Cuts: Our Response and Ways to Help from 10 days ago was cross-posted here, but I think many readers would have found it interesting
Neil Buddy Shah also serves on Anthropic’s Long-Term Benefit Trust (see mentions of CHAI on this page)
Importantly, it seems that GiveWell only funds specific programs from CHAI, not CHAI as a whole. It could very well be the case that CHAI as a whole is inefficient and not particularly good at what they do, but GiveWell thinks those specific programs are cost-effective.
Disclaimer: this is only from looking at GiveWell’s website and searching for “CHAI”, I don’t have any insider information
I think that is extremely unlikely, they have a lot to lose as soon as it’s confirmed that the archived data is not manipulated.
Also, from the page you cite:
we emphasize that these attacks can in most cases be launched only by the owners of particular domains.
So they would need to claim that you took control of a relevant domain as well.
But even if something like that happened, you could show that the archive has not been tampered (e.g. by linking the exact resource containing the information, or mentioning the “about this capture” tool that was added by the web archive to mitigate this)
I strongly agree that the benefits of sharing the evaluation greatly outweigh the risks, but I’m not sure if sharing the it relatively early is best
There is a risk of starting a draining back-and-forth which could block or massively delay publication. See e.g. Research Deprioritizing External Communication which was delayed by one year
It would cost more time for the org to review a very early draft and point out mistakes that would be fixed anyway
It could cause the org to take the evaluation less seriously, and be less likely to take action based on the feedback
I think the minimal version proposed by @Jason of just sending an advance copy a week or two in advance is an extremely low-cost policy that mitigates most of the risks and provides most of the benefits (but some limited back-and-forth would be ideal)
The original information is still archived, my understanding is that those attacks just inject other data that changes what is shown to the user, but as they mention it’s easily detectable and the original information can still be recovered.
A bigger risk would be that the organization asks the archive to delete their data, but that would look very suspicious, and you could use multiple archives (e.g. https://archive.is/ )
As was mentioned by several commenters on your last article, I think it would be valuable to share your article with ACE or Sinergia Animal before publishing it here.
Sharing evaluations with the evaluated org before publishing would likely make your analyses both more useful and more accurate, I’m curious to know why you decided against this.
I agree that US policy is obviously very important, but
The 2024 US Election doesn’t seem to have been close (unlike e.g. the 2000 one). I’m skeptical that EAs could have changed things that much, given that they couldn’t cause relatively easier policy changes (e.g. SB 1047)
As you mentioned, Moskovitz & Tuna already were the largest democratic donors
They were a minority, but several EAs favoured Trump over Harris, so their efforts would partially cancel out
Politicizing EAs, or EA organizations, would have made them less effective after Trump’s win and the whole “vibe shift”, when collaborating with the current administration
generic US policy, especially focusing on long-term issues (like US governance, or US decisions on questions like Nuclear/bio/AI) might be a good use of EA funds.
I think it always has been? My sense is that lots of EA funds are already spent on US policy things, e.g. https://www.nti.org/analysis/ and https://www.governance.ai/research
Update: the lottery has been drawn and the results are in! An anonymous donor won the right to recommend how to allocate $200k
Congratulations to the winner and thanks to the 21 donors who collectively donated $108,577.65 this year
From here it seems that indeed «he focuses on the design of the company’s Responsible Scaling Policy and other aspects of preparing for the possibility of highly advanced AI systems in the future.»
It seems that lots of people with all sorts of roles at AI companies have the formal role “member of technical staff”
I would take an even-odds bet that the total amount donated to charity out of Anthropic equity, excluding matches, is >$400m in 4 years time.
If Anthropic doesn’t lose >85% of its valuation (which can definitely happen) I would expect way more.
As mentioned above, each of its seven cofounders is likely to become worth >$500m, and I would expect many of them to donate significantly.
Anthropic is the go to example of “founded by EAs”
I find these kind of statements a bit weird. My sense is that it used to be true, but they don’t necessarily identify themselves with the EA movement anymore: it’s never mentioned in interviews, and when asked by journalists they explicitly deny it.
I would be surprised if the 3:1 match applied to founders as well. Also, I think 20% of employees donating 20% of their equity within the next 4 years is very optimistic.
My guess is that donations from Antrhopic/OpenAI will depend largely on what the founders decide to do with their money. Forbes estimates Altman and Daniela Amodei at ~$1B each, and Altman signed the Giving Pledge.
See also this article from Jan 8:
At Anthropic’s new valuation, each of its seven founders — [...] — are set to become billionaires. Forbes estimates that each cofounder will continue to hold more than 2% of Anthropic’s equity each, meaning their net worths are at least $1.2 billion.
I don’t think Forbes numbers are particularly reliable, and I think that there’s a significant chance that Anthropic and/or OpenAI equity goes to 0; but in general, I expect founders to both have much more money than employees and be more inclined to donate significant parts of it (partly because of diminishing marginal returns of wealth)
OWID says that ~45% of the population in Uganda has access to electricity, and that it more than doubled in the past 10 years. Does this match your experience?
I used to think that the exact philosophical axiologies and the handling of corner cases were really important to guide altruistic action, but I now think that many good things are robustly good under most reasonable moral frameworks.
these practical and intuitive methods are ultimately grounded in Singer’s deeply counterintuitive moral premises.
I don’t think this is necessarily true. Many (I would argue most) other moral premises can lead you to value preventing child deaths or stunting, limiting the suffering of animals in factory farms, or ensuring future generations live positive, meaningful lives.
@WillieG mentioned Christianity, and indeed, EA for Christians has many Christians who care deeply about helping others and come from a very different moral background. (I think sometimes they mention this parable)
within the EA community, beyond working on their own projects, do people have the tendency to remind & suggest to others “what they could have done but didn’t?”
I don’t have an answer to this question, but you might like these posts: Invisible impact loss (and why we can be too error-averse) and Uncertain Optimizing and Opportunity Costs
I think people regularly do encourage themselves and others to consider opportunity costs and counterfactuals, but I don’t think it’s specific to the EA community.
The principle becomes more challenging to accept when Singer extends it to a particular edge case.
I think this is the nature of edge cases. I don’t think you need to agree with Singer on edge cases to value helping others. This vaguely reminded me of this Q&A answer from Derek Parfit where he very briefly talks about borderline cases and normative truths.
I do think things get trickier for e.g. shrimp welfare and digital sentience, and in those cases philosophical considerations are really important. But in my opinion the majority of EA work is not particularly sensitive to one’s stance on utilitarianism.
Note that the hold-out set doesn’t exist yet. https://x.com/ElliotGlazer/status/1880812021966602665
What does this mean for OpenAI’s 25% score on the benchmark?
Note that only some of FrontierMath’s problems are actually frontier, while others are relatively easier (i.e. IMO level, and Deepmind was already one point from gold on IMO level problems) https://x.com/ElliotGlazer/status/1870235655714025817
You might also be interested in this post: Measuring Good Better, as a very high level summary of different organizations views on measuring ‘good’ (apparently nobody uses DALYs!)
After reading the recent https://www.thenation.com/article/society/progressive-left-philanthropy-strategy/ and many similar articles, my understanding is that proponents of “system level” changes are sceptical of a neoliberal/market-driven approach, and want a more centrally planned economy, where opportunities and outcomes are guaranteed to be more equal, or at least everyone is guaranteed a basic amount of wealth.
My understanding is that they care primarily about things like increased inequality, homelessness and unemployment in the United States, and they believe that main causes for those issues are the greed of the top 0.01% and market regulations (or lack thereof) which favour the richest at the expense of the poorest.
So I would imagine that reading things like:
AGOA gives Sub-Saharan African countries duty-free access to the American market for a range of product categories — in particular, apparel, which has historically been a key stepping stone for countries pursuing export-led manufacturing growth. [...] With Chinese labor costs increasing and general protectionist pressures growing, there may be a window of opportunity for African manufacturing industries to grow before automation in high-income countries potentially leads to significant re-shoring — provided that AGOA does not expire beforehand. Advocating for a strong AGOA renewal bill could improve the odds for an African industrial transformation.
They would expect an AGOA renewal to increase inequality and unemployment in the US by replacing American jobs with sweatshops in countries with lower minimum wage/worker rights, enriching capitalists who would profit from exploiting less protected workers.
But this is definitely a position I struggle to understand, so it’s likely that I’m misrepresenting it and would welcome other guesses/corrections.
Do they mention effective giving or collaborate with Doneer Effectief/the Tien Procent Club?
You know much more than I do, but I’m surprised by this take. My sense is that Anthropic is giving a lot back:
My understanding is that all early investors in Anthropic made a ton of money, it’s plausible that Moskovitz made as much money by investing in Anthropic as by founding Asana. (Of course this is all paper money for now, but I think they could sell it for billions).
As mentioned in this post, co-founders also pledged to donate 80% of their equity, which seems to imply they’ll give much more funding than they got. (Of course in EV, it could still go to zero)
I don’t see why hiring people is more “taking” than “giving”, especially if the hires get to work on things that they believe are better for the world than any other role they could work on
My sense is that (even ignoring funding mentioned above) they are giving a ton back in terms of research on alignment, interpretability, model welfare, and general AI Safety work
To be clear, I don’t know if Anthropic is net-positive for the world, but it seems to me that its trades with EA institutions have been largely mutually beneficial. You could make an argument that Anthropic could be “giving back” even more to EA, but I’m skeptical that it would be the most cost-effective use of their resources (including time and brand value)