Hi!
Iâm currently (Aug 2023) a Software Developer at Giving What We Can, helping make giving significantly and effectively a social norm.
Iâm also a forum mod, which, shamelessly stealing from Edo, âmostly means that I care about this forum and about you! So let me know if thereâs anything I can do to help.â
Please have a very low bar for reaching out!
I won the 2022 donor lottery, happy to chat about that as well
you have a history of apologetics for Anthropic, Lorenzo.
Iâm very surprised by this and by the many strong votes and agree votes, I donât know much about Anthropic and I donât write much about Anthropic. Is it possible that youâre confusing me with someone else? You can see all my EA Forum comments and tweets mentioning âAnthropicâ here and here.
I was also confused by that paragraph, as someone who read the handbook in ~2022. I just randomly came across this and this, and apparently this was an issue 7 years ago. I think itâs likely that several people who have been around longer than us havenât noticed that the handbooks and CEA staff changed a lot.
Edit: the comment above has been edited, the below was a reply to a previous version and it makes less sense now, leaving it for posterity
You know much more than I do, but Iâm surprised by this take. My sense is that Anthropic is giving a lot back:
funding
My understanding is that all early investors in Anthropic made a ton of money, itâs plausible that Moskovitz made as much money by investing in Anthropic as by founding Asana. (Of course this is all paper money for now, but I think they could sell it for billions).
As mentioned in this post, co-founders also pledged to donate 80% of their equity, which seems to imply theyâll give much more funding than they got. (Of course in EV, it could still go to zero)
staff
I donât see why hiring people is more âtakingâ than âgivingâ, especially if the hires get to work on things that they believe are better for the world than any other role they could work on
and doesnât contribute anything back
My sense is that (even ignoring funding mentioned above) they are giving a ton back in terms of research on alignment, interpretability, model welfare, and general AI Safety work
To be clear, I donât know if Anthropic is net-positive for the world, but it seems to me that its trades with EA institutions have been largely mutually beneficial. You could make an argument that Anthropic could be âgiving backâ even more to EA, but Iâm skeptical that it would be the most cost-effective use of their resources (including time and brand value)
As a side point, Iâm confused when you say:
I donât think they suggest that, depending on your definition of âstrongâ. Just above the sceenshotted quote, the article mentions that many early investors were at the time linked to EA.
That was said by the author of the article who was trying to make the point that there is a link between Anthropic and EA. So I donât see this as evidence of Anthropic being forthcoming.
I think in the context of the article, their quotes (44 words in total) make more sense:
In that context, the quotes clarify that Anthropic is not an âEA companyâ, and give a more accurate understanding of the relationship to the reader.
A more in-depth analysis of the historical affiliations, separations, agreements, and disagreements of Anthropicâs funders, founders, and employees with various parts of EA over the past 15 years would take far more than two paragraphs.
If I didnât know anything about Anthropic and I read the words âI definitely have met people here who are effective altruists, but itâs not a theme of the organization or anythingâ, I might think Anthropic is like Google where you may occasionally meet people in the cafeteria who happen to be effective altruists but EA really has nothing to do with the organisation.
You wouldnât think that in the context of the article, though.
I would not get the impression that many of the employees are EAs who work at Anthropic or work on AI safety for EA reasons. And that the three members of the trust theyâve given veto power over the company to have been heavily involved in EA.
I donât know what percentage of Anthropic employees consider themselves part of the EA community. Also, I donât agree that itâs clear that Evidence Actionâs CEO is part of the effective altruism community because evidence action received money from GiveWell.
https://ââwww.linkedin.com/ââin/ââkanika-bahl-091a936/ââdetails/ââexperience/ââ She was working in global health since before effective altruism was a thing, and many/ââmost people funded by OpenPhilanthropy donât consider themselves part of the community. In the same way that charities funded by Catholic donors are not necessarily Catholic. It does seem that OpenPhilanthropy was their main source of funding for many years though, which makes the link stronger than I originally thought.
I think the people in the article you quote are being honest about not identifying with the EA social community, and the EA community on X is being weird about this.
I think the confusion might stem from interpreting EA as âself-identifying with a specific social communityâ (which they claim they donât, at least not anymore) vs EA as âwanting to do good and caring about othersâ (which they claim they do, and always did)
Going point by point:
Dario, Anthropicâs CEO, was the 43rd signatory of the Giving What We Can pledge and wrote a guest post for the GiveWell blog. He also lived in a group house with Holden Karnofsky and Paul Christiano at a time when Paul and Dario were technical advisors to Open Philanthropy.
This was more than 10 years ago. EA was a very different concept /â community at the time, and this is consistent with Daniela Amodei saying that she considers it an âoutdated termâ
Amanda Askell was the 67th signatory of the GWWC pledge.
This was also more than 10 years ago, and giving to charity is not unique to EA. Many early pledgers donât consider themselves EA (e.g. signatory #46 claims it got too stupid for him years ago)
Many early and senior employees identify as effective altruists and/âor previously worked for EA organisations
Amanda Askell explicitly says âI definitely have met people here who are effective altruistsâ in the article you quote, so I donât think this contradicts it in any way
https://ââx.com/ââAmandaAskell/ââstatus/ââ1905995851547148659
Anthropic has hired a âmodel welfare leadâ and seems to be the company most concerned about AI sentience, an issue thatâs discussed little outside of EA circles.
Thatâs false: https://ââen.wikipedia.org/ââwiki/ââArtificial_consciousness
On the Future of Life podcast, Daniela said, âI think since we [Dario and her] were very, very small, weâve always had this special bond around really wanting to make the world better or wanting to help peopleâ and âhe [Dario] was actually a very early GiveWell fan I think in 2007 or 2008.â
The Anthropic co-founders have apparently made a pledge to donate 80% of their Anthropic equity (mentioned in passing during a conversation between them here and discussed more here)Their first company value states, âWe strive to make decisions that maximize positive outcomes for humanity in the long run.â
Wanting to make the world better, wanting to help people, and giving significantly to charity are not prerogatives of the EA community.
Itâs perfectly fine if Daniela and Dario choose not to personally identify with EA (despite having lots of associations) and Iâm not suggesting that Anthropic needs to brand itself as an EA organisation
I think thatâs exactly what they are doing in the quotes in the article: âI donât identify with that terminologyâ and âitâs not a theme of the organization or anythingâ
But I think itâs dishonest to suggest there arenât strong ties between Anthropic and the EA community.
I donât think they suggest that, depending on your definition of âstrongâ. Just above the sceenshotted quote, the article mentions that many early investors were at the time linked to EA.
I think itâs a bad look to be so evasive about things that can be easily verified (as evidenced by the twitter response).
I donât think X responses are a good metric of honesty, and those seem to be mostly from people in the EA community.
In general, I think itâs bad for the EA community that everyone who interacts with it has to worry about being liable for life for anything the EA community might do in the future.
I donât see why it canât let people decide if they want to consider themselves part of it or not.
As an example, imagine if I were Catholic, founded a company to do good, raised funding from some Catholic investors, and some of the people I hired were Catholic. If 10 years later I werenât Catholic anymore, it wouldnât be dishonest for me to say âI donât identify with the term, and this is not a Catholic company, although some of our employees are Catholicâ. And giving to charity or wanting to do good wouldnât be gotchas that Iâm secretly still Catholic and hiding the truth for PR reasons. And this is not even about being a part of a specific social community.
GiveWell posts a lot of interesting stuff on their blog and on their website, but in the past year they only reposted hiring announcements on the EA Forum.
E.g. I donât think that USAID Funding Cuts: Our Response and Ways to Help from 10 days ago was cross-posted here, but I think many readers would have found it interesting
Neil Buddy Shah also serves on Anthropicâs Long-Term Benefit Trust (see mentions of CHAI on this page)
Importantly, it seems that GiveWell only funds specific programs from CHAI, not CHAI as a whole. It could very well be the case that CHAI as a whole is inefficient and not particularly good at what they do, but GiveWell thinks those specific programs are cost-effective.
Disclaimer: this is only from looking at GiveWellâs website and searching for âCHAIâ, I donât have any insider information
I think that is extremely unlikely, they have a lot to lose as soon as itâs confirmed that the archived data is not manipulated.
Also, from the page you cite:
we emphasize that these attacks can in most cases be launched only by the owners of particular domains.
So they would need to claim that you took control of a relevant domain as well.
But even if something like that happened, you could show that the archive has not been tampered (e.g. by linking the exact resource containing the information, or mentioning the âabout this captureâ tool that was added by the web archive to mitigate this)
I strongly agree that the benefits of sharing the evaluation greatly outweigh the risks, but Iâm not sure if sharing the it relatively early is best
There is a risk of starting a draining back-and-forth which could block or massively delay publication. See e.g. Research Deprioritizing External Communication which was delayed by one year
It would cost more time for the org to review a very early draft and point out mistakes that would be fixed anyway
It could cause the org to take the evaluation less seriously, and be less likely to take action based on the feedback
I think the minimal version proposed by @Jason of just sending an advance copy a week or two in advance is an extremely low-cost policy that mitigates most of the risks and provides most of the benefits (but some limited back-and-forth would be ideal)
The original information is still archived, my understanding is that those attacks just inject other data that changes what is shown to the user, but as they mention itâs easily detectable and the original information can still be recovered.
A bigger risk would be that the organization asks the archive to delete their data, but that would look very suspicious, and you could use multiple archives (e.g. https://ââarchive.is/ââ )
As was mentioned by several commenters on your last article, I think it would be valuable to share your article with ACE or Sinergia Animal before publishing it here.
Sharing evaluations with the evaluated org before publishing would likely make your analyses both more useful and more accurate, Iâm curious to know why you decided against this.
I agree that US policy is obviously very important, but
The 2024 US Election doesnât seem to have been close (unlike e.g. the 2000 one). Iâm skeptical that EAs could have changed things that much, given that they couldnât cause relatively easier policy changes (e.g. SB 1047)
As you mentioned, Moskovitz & Tuna already were the largest democratic donors
They were a minority, but several EAs favoured Trump over Harris, so their efforts would partially cancel out
Politicizing EAs, or EA organizations, would have made them less effective after Trumpâs win and the whole âvibe shiftâ, when collaborating with the current administration
generic US policy, especially focusing on long-term issues (like US governance, or US decisions on questions like Nuclear/âbio/âAI) might be a good use of EA funds.
I think it always has been? My sense is that lots of EA funds are already spent on US policy things, e.g. https://ââwww.nti.org/ââanalysis/ââ and https://ââwww.governance.ai/ââresearch
Update: the lottery has been drawn and the results are in! An anonymous donor won the right to recommend how to allocate $200k
Congratulations to the winner and thanks to the 21 donors who collectively donated $108,577.65 this year
From here it seems that indeed ÂŤhe focuses on the design of the companyâs Responsible Scaling Policy and other aspects of preparing for the possibility of highly advanced AI systems in the future.Âť
It seems that lots of people with all sorts of roles at AI companies have the formal role âmember of technical staffâ
I would take an even-odds bet that the total amount donated to charity out of Anthropic equity, excluding matches, is >$400m in 4 years time.
If Anthropic doesnât lose >85% of its valuation (which can definitely happen) I would expect way more.
As mentioned above, each of its seven cofounders is likely to become worth >$500m, and I would expect many of them to donate significantly.
Anthropic is the go to example of âfounded by EAsâ
I find these kind of statements a bit weird. My sense is that it used to be true, but they donât necessarily identify themselves with the EA movement anymore: itâs never mentioned in interviews, and when asked by journalists they explicitly deny it.
I would be surprised if the 3:1 match applied to founders as well. Also, I think 20% of employees donating 20% of their equity within the next 4 years is very optimistic.
My guess is that donations from Antrhopic/âOpenAI will depend largely on what the founders decide to do with their money. Forbes estimates Altman and Daniela Amodei at ~$1B each, and Altman signed the Giving Pledge.
See also this article from Jan 8:
At Anthropicâs new valuation, each of its seven founders â [...] â are set to become billionaires. Forbes estimates that each cofounder will continue to hold more than 2% of Anthropicâs equity each, meaning their net worths are at least $1.2 billion.
I donât think Forbes numbers are particularly reliable, and I think that thereâs a significant chance that Anthropic and/âor OpenAI equity goes to 0; but in general, I expect founders to both have much more money than employees and be more inclined to donate significant parts of it (partly because of diminishing marginal returns of wealth)
OWID says that ~45% of the population in Uganda has access to electricity, and that it more than doubled in the past 10 years. Does this match your experience?
I used to think that the exact philosophical axiologies and the handling of corner cases were really important to guide altruistic action, but I now think that many good things are robustly good under most reasonable moral frameworks.
these practical and intuitive methods are ultimately grounded in Singerâs deeply counterintuitive moral premises.
I donât think this is necessarily true. Many (I would argue most) other moral premises can lead you to value preventing child deaths or stunting, limiting the suffering of animals in factory farms, or ensuring future generations live positive, meaningful lives.
@WillieG mentioned Christianity, and indeed, EA for Christians has many Christians who care deeply about helping others and come from a very different moral background. (I think sometimes they mention this parable)
within the EA community, beyond working on their own projects, do people have the tendency to remind & suggest to others âwhat they could have done but didnât?â
I donât have an answer to this question, but you might like these posts: Invisible impact loss (and why we can be too error-averse) and Uncertain Optimizing and Opportunity Costs
I think people regularly do encourage themselves and others to consider opportunity costs and counterfactuals, but I donât think itâs specific to the EA community.
The principle becomes more challenging to accept when Singer extends it to a particular edge case.
I think this is the nature of edge cases. I donât think you need to agree with Singer on edge cases to value helping others. This vaguely reminded me of this Q&A answer from Derek Parfit where he very briefly talks about borderline cases and normative truths.
I do think things get trickier for e.g. shrimp welfare and digital sentience, and in those cases philosophical considerations are really important. But in my opinion the majority of EA work is not particularly sensitive to oneâs stance on utilitarianism.
Note that the hold-out set doesnât exist yet. https://ââx.com/ââElliotGlazer/ââstatus/ââ1880812021966602665
What does this mean for OpenAIâs 25% score on the benchmark?
Note that only some of FrontierMathâs problems are actually frontier, while others are relatively easier (i.e. IMO level, and Deepmind was already one point from gold on IMO level problems) https://ââx.com/ââElliotGlazer/ââstatus/ââ1870235655714025817
This whole thing is about two quotes with 44 words in total, which really donât seem to me to be false or misleading. Based on the little information I have, in the context of the article, I think they make the representation of the relationship closer to what I think is true, and consistent with previous messaging from them.
Of course I could be wrong and Askell and Amodei could secretly mostly have EA friends, go to EA parties, secretly self-identify with the community, etc. but I donât think thereâs public evidence to suggest that.