USAID took down its archive of evaluation reports, but a high school student happened to have downloaded them for a ML project, and they are now available online again:
Thoughts on the Centre for Effective Altruism, weakly held, n = 1, but feeling like things someone should say:
EAGs seem good
Community Health seems better than replacement. It’s a hard job, not perfect, but I am glad someone is doing it.
the Forum seems like not a place I want to be despite having previously been here a lot. Hard to make sense of what’s important, somehow
I rarely see comms work I think is good. Maybe it’s in places I don’t see, but on twitter, is does EA have a reputation that is helped by CEA? The odd thread I guess?
Vision? I dunno. Is there meant to be some kind of movement-wide vision I am meant to understand and get behind? ‘Do AI safety/policy work’, I guess?
Functionalness? Has there been any EA org as mired in dysfunction as CEA (other than FTX)? Has that changed recently? Base rates aren’t good.
Uni teams, no idea.
I imagine that there are lots of good, hard-working people at CEA and my intention isn’t to hurt. But I think we should see the world as it is and perhaps this is information to some. It is so easy never to say anything and then wonder why nothing changes. If this seems false to you, downvote it. But if it is upvoted when you see it, update on it. CEA in some ways speaks for EA, or has been seen to. What is going on?
People are welcome to DM me to chat about this though I prefer twitter DMs or emails to nathanpmyoung@gmail.com
Thanks for sharing! I mostly agree. If you read my posts from the past 2 years you can see part of the story for why the Forum is the way it is. Broadly, the Online Team has been more focused on supporting the rest of CEA and doing growth-related work, and we’ve been putting less and less FTE toward the Forum over time (~1.5 FTE when I left).
I feel sad that we didn’t push as much on quality and community as I had originally wanted. I found it challenging to try to convince leadership of the impact of doing cultivation work on the Forum, partly because the impact is actually quite nebulous and risks being just entertainment for EAs. Ultimately I still believe that there are tipping points that would cause the Forum to lose the majority of its value, and community building on the Forum can have significant impact by preventing that.
Thanks both! Relatedly, the Online Team is now more empowered to follow a strategy where growth is not the key aim of the Forum (currently our best guess is measuring ‘mind changes’, but stay tuned for precise metrics).
I’m very open to suggestions for how to make the EA Forum more of a place where crucial conversations are being had, and decisions are being made. This includes critiques. I’d love to hear more from people on how they think they Forum is falling short, here, in DM, or over a call.
My take: have you spoken with key decision-makers about incentivising them to write more on the forum instead of discussing things internally in Google Docs or whatever?
Yes, but there is always more of this to do. I.e. some great posts come from me nudging organisations or small conferences to post more. Bottleneck is more time than anything.
Editing to add—Forum readers can help here. If you read something that you think should be on the Forum, nudge the author to post it. If you don’t have time, ping me to do so. Sometimes I offer light editing or accountability support as well to get things over the line.
You’re great Toby, but I do worry that it will be hard to make progress without getting you more capacity for the Forum. I’m glad that we were given more freedom this year, and I hope that means CEA will invest more than the current 1.5 FTE into the Forum, at least in 2027.
With your limited time, I wonder if you should focus more on proactive community building, in the sense we discussed when I first took over the team. I think people’s perceptions of the space matter a great deal to how they engage with it, as you can see from some of Nathan’s comments under the parent quick take. We can chat about it when we meet tomorrow. :)
...this would be alleviated by another good hire. We are getting into the hiring queue now, so hopefully we’ll be hiring another content manager in the near(ish) future.
When I was a Christian, I led a lot of Bible study groups. Christians care a lot about Truth. In theory any question was askable but in practise it was clear that some questions were not. If somebody asked them I maybe allowed some kind of response, and then moved the discussion on. Discussions about the historicity of the Bible often created a mess so you didn’t really want to have them. In practise I recall having them very rarely, if at all.
To me the EA forum often has the smell of the same kind of thing where many important questions are just ‘not the done thing’. Below, James links CEA’s growth discusison. Do I really want to engage with that? Not really. I think I’ll feel like someone who is grumpy at a wedding. But also, from a quick scroll, do I think that CEA getting it’s engagement numbers up is the central thing it should be tracking? Not really. How many career changes are there? What is some notion of total impact?
And, sure, these are hard questions and I am not sure there are good answers. But when I came on here 7 years ago, this felt like a place to have those discussions. And now.. it doesn’t.
IIRC, 80k stopped using their DIPY(?) metric several years ago.
If you look through their recent reports, they mostly share metrics related to engagement with their programmes/services. And then occasionally there is an additional survey, e.g., the job board survey in their H2 2025 report.
From what I understand, CG likes grantees to report individual narrative case studies, and then these are coded. At least, that’s how it works for the CEA’s CBG programme. So maybe that’s what 80k does as well? And then this is supplemented with engagement metrics.
If that’s the case, it’s pretty similar to CEA’s system.
I’d be interested to hear more about why you didn’t want to engage with posts like CEA’s, and what you mean by “many important questions are just ‘not the done thing’”. Not sure if it’s related, but I think people who push back on CEA tend to get lots of karma still. :)
Honestly you’d have a great time at EA in the Lakes and should come to the next one I run. More generally, you want to get yourself into an EA group space of people who make donations (of money and/or time) to EA infrastructure, rather than being in receipt of resources from EA infrastructure. The vibe is totally different: the questions are freer, and we help each other clarify the answers to being more truthful and more loving and more actionable because we all want our donations to be used well.
I am pretty happy with my EA circles, for what it’s worth. I have come here to say somethings about this place and to some extent about the powerful organisation that runs it.
But I’d hear more about your EA in the Lakes thing. Which lakes?
They said growth was their focus, set goals, and, by their own account, did well.
However, the only place I’ve ‘felt’ their growth (beyond their reports) was with EAGxAmsterdam (attendance was up 36%). But I don’t know how much of that was due to things we did vs what CEA did.
My guess is there really is growth, but it’s not ‘felt’ by people like you and me because of where that growth is happening.
And yeah, they’ve struggled to hire for comms. They’ve admitted this. Last I heard, their hiring team was at full capacity hiring for other functions, so I guess they’re prioritising sorting their core programmes before going hard on comms.
I’ve said it before, but I really do think a General Assembly responsible for holding CEA’s board accountable would help keep everyone up to date with what CEA’s doing.
I think possibly the issue is that there’s a lot of effort being put in to manage reputational risks, possibly without quite understanding exactly what the forward-looking reputational risks are (because you’re reacting to bad things that have happened). That’s got to be really tricky.
To my eye, CEA has repeatedly focused on having good PR, meanwhile Yudkowsky wears a shiny gold hat and talks to Bernie Sanders and has a NYT best seller. Is all of that comms effort working?
Thanks for sharing that post, I thought it provided good insight into how you’re thinking about reputational risk. I did come away from the post wondering why the proposed reputational risk management method doesn’t appear more widespread across competitive firms if it has an edge on the traditional approach.
I don’t know if I understand/agree with your comparison of CEA to Yudkowsky. CEA, as the steward of much of the EA community’s infrastructure, has a lot of authority and responsibility for this community and its people. In order to maintain peoples’ trust and confidence to do that well, I suspect it’s necessary for it to observe a quite conservative form of message discipline that’s more consistent with traditional PR than the other strategy that the post you shared mentions. I can think of a couple examples where I don’t agree with a communications decision I’ve heard of them making, but I don’t think Yudkowsky’s approach that you gesture to would be the right direction to move. Maybe looking to the ways organizations like universities and churches succeed and fail with communications would provide more useful comparisons.
For individuals or organizations that don’t have responsibility for community stewardship, I agree with the praise you give of Yudkowsky’s approach.
I agree the comms is currently not as good as it could be.
I think the massive push to expand effective giving through a large variety of (non-CEA) methods might well take over EA external comms in the next few years. And I’m hoping the effects of that will eventually bounce back and fix CEA. There will just be a lot more comms-understanding EA-committed people in the EA hiring pool.
I’m also very hopeful about NEST, which seems to have understood that things like “hold a Nerdfighters or Kursgesagt meetup” is a thing a local EA group should be doing to attract the people who might become an EA, and is not cringe-bad-wrong-unprofessional.
Just a small idea I had: is anyone aware of artists who make “sleep music” on Spotify or any other streaming service who verifiably donate their streaming revenue?
It seems like a low-effort way to have a pretty sizeable impact. Some of these sleep artists are racking up millions of plays. Could maybe get some new people learning about EA as well.
Hank Green has done this on YouTube, and has a video discussing the economics (about $1000/month! although already having an audience presumably helped).
Mental health is one of the most widespread, impactable, yet neglected global health issues of our time. Even knowing mental health conditions are generally under-reported and/ or misrepresented, evidence still shows widespread effects of mental wellbeing on physical health, life satisfaction, and productivity, however it has still not taken off as a major cause area in many EA organizations I have talked to. Why is this?
I should mention there is an argument that many mental health interventions are in fact harmful, mainly via a priming mechanism where when people start to consciously preoccupy themselves with their own mental health, that itself causes psychological suffering. It is a sort of Buddhist argument, I guess.
Abigail Shrier’s Bad Therapy (2024) has probably most discussed version of that case, though I have not read the book, and I am extremely epistemically uncertain about all this, in a way that makes me reluctant to either support or oppose interventions in this area.
Interesting take @alesziegler ! I do believe it is true that too much of any one thing is usually bad, and also believe that people who are not trained/ underqualified taking on mental health is counterproductive. I think people take on these roles or use clinical language that should not be applied to what they are doing precisely because they cannot get the credible help they need and are trying to fill that void. Mental health is extremely subjective and therefore interventions may look like community connection or involvement in sports rather than clinical therapy.
I am also highly skeptical of the legitimacy of Schrier’s claims and her choice in very carefully selected stastistics as evidence. For ecample, contrary to some of Shier’s claims, evidence-based school-based emotional programs have had profound impact, especially on marginalized populations (https://www.sciencedirect.com/science/article/pii/S2773233924000032). Maybe my thoughts on this is best summed up by saying, “Don’t throw the baby out with the bath water”?
In addition to @huw’s great comment, there’s a branch of EA which focuses on well-being, and with that focus mental health interventions often look really effective. Check out the happier lives institute Huw mentioned, times the “WELLBY” and @MichaelPlant
Thank you @NickLaing ! I have done a lot of reading on the Wellby and that is one of the reasons I feel mental health is currently under-represented as a global health issue. It is fascinating to hear how people globally weigh the importance of wellbeing and I’m curious how many other aspects of their lives would be more easily improved if wellbeing increased! @MichaelPlant would love to connect on your work!
G’day Madeline, I run an EA mental health org in India. The reason for this is simply that existing mental health interventions do not compare with GiveWell’s grants on a DALYs/$ basis. In my opinion, the reasons are:
DALY moral weights may be biased against depression
The moral weights of different diseases in the Global Burden of Disease study, which informs DALY estimates, are determined by asking the general public whether they’d prefer to have one disease against another. When you do this with depression, people who haven’t experienced it tend to prefer to have it to many other conditions. However, when you ask people who have experienced it, they choose many very painful conditions over depression. This is one of the widest gaps in the moral weight data. See Pyne et al. 2009 and this post.
Psychotherapy is usually modelled as a short-term effect
Psychotherapy is typically modelled as a treatment, and not a ‘skill’. What I mean by this is that a dose of psychotherapy is assumed to only have effects that decay over a period of time and zero out after that in most CEAs, including those from the Happier Lives Institute. However, many psychotherapy patients will tell you that they learned skills that were useful long after the therapy ended, and there is some limited evidence that psychotherapy’s effects may last decades, or potentially never zero out. If this were true, the effects could be very long-lasting and therefore it would be much more valuable to treat a case of depression.
Suicide prevention isn’t cost-effective if it’s only a short-term effect
Consider that for most of GiveWell’s top interventions, the bulk of the DALYs averted come from ‘saving’ a life—i.e., preventing a death from a disease in a way that allows the person to then go on to live a healthy life, such as preventing a malaria case in an under-5 (which they might die from), even if they go on to catch it after 5 years old.
As a short-term effect, psychotherapy can only postpone a suicide by the length of the treatment effect. But if it were a skill and had some durable long-term effect, it may genuinely prevent one, which would tremendously increase the value of suicide prevention interventions.
Existing interventions haven’t been cheap enough yet
With the exception of some incredible policy work in, for example, reducing toxicity of pesticides commonly used for suicide, existing interventions are still quite expensive. The Happier Lives Institute’s top charities cost ~$40 to treat a single person, while a bednet costs $7. I’m fudging the numbers a bit here, but if we stick with DALYs, psychotherapy is still about an order of magnitude more expensive than it needs to be to look great for EAs.
However, there is work being done to improve that! My charity, Kaya Guides, treated people for $20 each in April, at what we estimate is a similar effect size to the best charities, and we’re confident we can get below $10. We’re using a technique called guided self-help that allows us to dramatically reduce contact hours per participant (and being all-digital helps a lot, too).
Conclusion
Orgs like the Happier Lives Institute have done a lot of advocacy work too, to raise the profile of mental health within EA, and there are plenty of funders that take mental health seriously (in a way that apparently wasn’t true a decade ago). It is, after all, still a nascent space.
Thank you @huw and apologies for my delay- still getting the hang of the forum! I really appreciate these insights and would love to learn more about the work you are doing at Kaya Guides. I have been thinking a lot about what innovation would look like in the mental health space and how the personal nature of mental health makes widespread interventions challenging (for example, medication can not necessarily cure mental illness the way it can cure a physical ailment like an infection).
I have this game theory thing I’ve been working on that involves modifying the Iterated Prisoner’s Dilemma to include death, asymmetric power, and aggressor reputation. Agents’ points are their “power” that dynamically impacts their payoff matrix.
The basic takeaway is that this simple simulation seems to make a case for cooperating with weaker agents, by showing how the cooperative strategies outcompete the aggressive ones in the long run. I think, before I can make a proper post about it, I’ll need to run some analysis to graph out how, for instance, having a higher percentage of cooperative agents increases the odds of survival, which implies a kind of Veil of Ignorance logic towards being cooperative.
Note that I mean cooperative in the sense that you don’t defect first except against aggressors that have defected first against non-aggressors.
With default settings, the most common result of any given run is that a significant number of the cooperative strategies survive and almost all of the aggressive ones die out. Very occasionally, particularly if you adjust the settings are certain way, a single “Opportunist” strategy, that Tit-For-Tats against stronger agents and Defects against weaker ones, will be the only survivor. This seems to imply, at least, to me, that being a cooperative strategy significantly increases your odds of survival, as the alternative is to hope to win a “Highlander” scenario.
I think this is relevant to AI alignment as a variation on Anthropic Capture, the “Hail Mary” approach that Bostrom mentions in Superintelligence. It could work as part of a defence-in-depth, a kind of “infoblessing” that could persuade some AGI to spare us as a kind of Superrational Signalling. While you might assume this only works if aliens are probable, it also functions in a multi-agent scenario where there are several AGI at near peer levels of power. It also potentially could be a way to align a previously unaligned AGI even after it is deployed. If enough AGIs are aligned in this way, their alliance could defeat the unaligned AGIs.
I realize that a very obvious critique of this work is that the simulation is probably too simple. I intentionally tried to keep it an MVP in its first iteration. I also should, as mentioned earlier, complete a more thorough and rigorous analysis of the apparent results. I’m also keenly aware that it seems like this is a “neglected” path towards alignment, and I’m uncertain whether this is because the idea is a bad one that’s already been discarded by others who are more competent. I know that there are related ideas around Decision Theory, Acausal Trade, and Superrationality, but I’ve never seen this particular kind of effort, which confuses me, because it seems obvious and trivial to try.
My main question to ask is simply, does this seem like something worth pursuing and expanding further, or am I wasting my time on a foolish endeavour?
Is the main idea here that if “have you ever defected against a non-agressor” is the agreed way to decide who to not cooperate with, then cooperative strategies win and then stay cooperative when power imbalances are large? Is there a reason would agents choose to use that instead of any other decision criteria? Also, I wouldn’t say that the obvious critique is that your model is too simple, I would say it’s that your model can easily be unrealistic if you don’t try very hard to check it. Good game theory models are sometimes very simple.
That is sorta the idea yes. Agents would choose this decision criteria mostly because it vastly increases their odds of survival, which allows them to further whatever goals they have. I would hope that this result is obvious enough that many agents will be able to converge on it, increasing the proportion using it, and thus increasing the overall survival rate.
The other takeaway is that, given that humans will be weaker than AGI/ASI, any game theoretic reason for such entities to still cooperate with us can potentially help reduce the existential risk.
I agree that the model requires further scrutiny to determine if it is realistic enough to matter.
The EA Forum Digest might be biased toward posts made on certain days. Had an LLM gather dates for the posts inside Digest #298 through #302, and compare their weekday frequencies against EA Forum posts made within the same window. Probably not enough data (n=50 for Recommended; n=86 for all Digest Posts) to be meaningful.
(Got the idea to check because I was imagining if I was responsible for writing the Digest and thought, “hmmm I feel like if I were sending out emails every Wed, I would be biased toward posts made on certain days because of the availability heuristic and/or the configuration of my weekly reading/writing schedule”)
Interesting! Kind of odd to see the peak being Wednesday (the day I write and send the digest) because posts posted on Wednesday will necessarily have the lowest karma relative to their final amount, and I start off with a list of all posts sorted by karma. Recommended matters most because the rest of the digest is for announcements etc… which are featured when relevant regardless of karma.
Introducing mental tool for rumination that helps me :)
Thinking Loop Breaker
Part 1: overall mindfullness (focus on the now / movement / gratitude practice) Part 2: in case of persistent thought: 5m timer to arrive at specific decision (including decision to do nothing), offloading to notes with outputs (for every offloading note: next action / decision / no action is required at this time)
P.S.
This it my first EA Forum post! If anyone finds this relevant, let me know! Or if this is not the place — also please let me know
Random idea: what if peopl animal advocates try to establish a norm that you don’t eat meat in space? It currently only requires persuading a small amount of people to take a quite low cost action, food in space is already weird, and the norm has a chance of persisting as space faring scales up. Could eventually be established as law, and hopefully expanded to veganism and no insect farming if that becomes less stigmatized.
oh okay I think the misunderstanding was that by “in space” you mean something like “everywhere/on planets that aren’t earth” and not just “in space stations/on spacecraft”
I don’t have a clear idea here! Something like: most astronauts don’t eat meat --> stuff like insect farming in space never takes off --> instead, an animal friendly protein production zero G technique develops into technological maturity --> hard to compete with later
Or
Most astronauts don’t eat meat --> as more and more people live off Earth, they adopt the norm --> counterfactually much less animal consumption in the future.
That makes sense, thanks for responding. I can see where you’re coming from.
I think early spacefaring will probably be a quite competitive space, since there seems to be a lot of diverse commercial and government interest right now. Because of that, it seems like it may be difficult to create any sort of durable norms around something like food sourcing unless there’s a clear winner on efficiency, which I presume would be arrived at independently of whether it involves animals. This might be weak argument if you expect early spacefaring to be dominated by an individual entity like NASA and if you expect government contracts for things like food in space to not be very competitive.
Maybe something that would make this more concrete is looking at whether there are examples of the lock-in of practices like this in the past, and whether we have reason to believe that additional dynamics would make early spacefaring harder to draw historical comparison to. What do you think?
epistemic status: quickly written mini-memo, not Claude-generated
I found lots of success with adopting this “Coaching” mindset framing going into my mentorship calls, that can strike the “firm but patient” balance:
this is possibly a (good) remnant from my past weightlifting-coaching backgrounds
accountability
so much value of paid coaching clients is from the social accountability of showing up to the scary arena together
where you’re coming from as a mentor...
from a place of “i’m rooting for you and i want you to succeed which is why i’m giving you this feedback” and “i believe you can win the Olympic gold medal if you fix xyz”
give concrete pointers
as often as possible, don’t be nebulous
eg. say less “go to gym”; say more “show up every Monday at 10am to 11am and do these five workouts, three sets of 10 reps each in this specific order.”
points out errors with immediacy;
build a relationship where error-correction is the norm
eg. “[mid-screen share] … wait wait go back, i think you should remove the picture of Terminator on the poster, because-”
leave mentees feeling like they worked with (an actual) coach and not someone they just answer to ;)
So researchers have used AI to make new viruses and broadcast the achievement to the world!
This could be fine, but given the commentaries alongside it seems there is some disagreement at least.
“But concerns have already been raised that the same technology could be used maliciously to create new disease
“In a commentary accompanying the publication in the journal Science, Dr Thomas Inglesby and Dr Moritz Hanke from the Center for Health Security at Johns Hopkins University wrote that the findings raise “urgent biosafety and biosecurity questions”.
They said it was no longer a question of “whether generative viral genome design will exist” but whether it can be used without “enabling serious harm”.
For example, they said new viruses with the potential to cause disease “should not be pursued”.
The researchers themselves took steps to maximise safety. They excluded viruses that could infect complex organisms from the training database, performed the research on phage rather than viruses that infect people and it all took place in a secure laboratory.”
I’m interested in what biosec people here think. Is making virus’s and publicising the achievement genuinely OK and safe at this point with the safeguards, or could it be net bad?
It seems to me that much of the evaluations/criticisms/defenses of EA conflate the empirical phenomena (whatever you see on with the EA forum, whatever is happening at your local university group, CEA, EAGs, or whatever) with some broader idea of effective altruism (the philosophy stuff / possibly the normative stuff if you believe in that). I am uncertain how to define this “broader idea”, maybe I can call it some theoretical core, maybe I can call it Weber’s idealized type (a deliberately purified theoretical construct that no empirical instance fully matches), maybe I can say something about Wittgenstein and family resemblance (his refute to the demand that a concept must have a common essence), maybe we can each reference our own favorite theoretical definition of EA for the purpose of this point)
But I think never the less it is pretty important to distinguish the empirical manifestation of EA (the type of stuff you can do sociology on), and the theoretical core (to the extent that we can define some semi-coherent notion of such).
We used to call this upper case (ea community/movement) vs lower case (the (meta) philosophy). I think meta normative framework/philsophy is the way I think about it; that is, you can apply EA to most normative frameworks, esp consequentalisty ones (though to many people in the community, the EA framework is actually just applied total utilitarianism). FWIW, and I say this as someone who has many issues with the EA community, the vast majority of EA criticisms from outside the community I see are just highly inaccurate and not worth engaging with from an intellectual POV (but maybe still worth engaging with for movement reputation).
Situational Awareness (the hedge unhedged fund) has imploded. It seems much of their gains were from overleveraging their trades, which has then caused them to, in some readings, undergo the largest absolute short-term fund loss in history (over $10B in a few weeks).
This is totally relevant to EA because it’s another case of youthful hubris vs veteran experience. EA is a place an elite graduate can meet in a bar with some other cool kids and do a napkin pitch for their first charity with zero field experience and soon bring in a million and where a 43 year old veteran with 20 years in the field in his cause area is virtually ignored because he doesn’t have the youthful EA vibe.
FTX was all EA in the beginning including Aschenbrenner himself who must have still been using a practice razor when he worked there.
The point is EA needs to create a culture that draws in more experienced people who bring their common sense which is informed by real life experience not just classes they had in college. To do this requires thoughtful planning and execution, which should have happened post FTX and maybe this is a reminder. And it’s not just experience, there’s a host of human variety of perspectives that need to all be here so the table benefits from many viewpoints and ways of thinking.
One of the studies on where intelligent people go wrong is that they all follow each other to the same fail point, and they had nobody saying hey, there’s another possible point over here. They lacked perspective. It’s called “Groups of diverse problem solvers can outperform groups of high-ability problem solvers,” which is just common sense to people who love collective intelligence. And love is actions.
I’m an 82-year-old collective intelligence pioneer, with my first book chapter, “Quest for Collective Intelligence,” published in 1995. Recently, my support offered to young CI “geniuses” was overlooked a couple of times because I didn’t have their youthful vibe. I understand that, as every new generation totally legit need to make their own mistake.
Hi George, thrilled to meet you. Yes, I wish I was as chill as you and what you say is true and also every generation shall try to save the next from repeating mistakes.
When I was only a boy I figured out that adults were really far smarter than me and sought to learn from them. Later I worked with the elderly for a while and learned it was a mixed bag, some were old and bitter obviously having made bad choices and failed to forgive. But others were clearly life experience geniuses and so you just had to be discerning in who you listened to. So probably because of this and my eventual passion for CI I just find the failure to seek the value of all perspectives as just leaving so much value on the table, to use a popular EA term. I’m going to DM you, can’t bypass a CI pioneer!
Their gains were from leveraged directional bets and their losses were from leveraged directional bets. Seems they were margin called, so their liquidity was squeezed—poor risk management. It seems that they have learned from this and are now using “fully paid for” derivatives, so presumably call options whose premium they can afford. Hedge funds are not necessarily hedged, but eg long/short equity hedge funds are (while macro hedge funds can have all kinds of unhedged macro bets on.)
Sadly it appears that adjacency to EA correlates with poor risk management, I’m wondering whether this is somewhat structural (beyond eg the age group…)
They certainly lost a lot of money in July and by being forced to sell, but per the WSJ Aschenbrenner wrote that the fund is still up 80% on the year.
The fund doesn’t seem to be closing, and it looks like they weren’t forced to sell their stake in Anthropic, so it’s not clear to me how much of a failure or a success the project as a whole is so far.
The article ends by saying: “It still holds a big chunk of harder-to-sell positions in privately held companies such as Anthropic. It remains to be seen if those companies will live up to their lofty valuations if and when they hit the public markets.”
I wonder if this means that the fund did not get wiped out completely—otherwise Citadel (who bought all of their publicly traded stock positions) would have also taken their Anthropic shares?
To this community’s credit, much of the discussion of his paper and career plans was skeptical. I think Aschenbrenner is a useful template for the kind of person that we need to have our guards up around, for a future where large sums of money enter the movement through the Third Wave.
I don’t think he was working in bad faith, and probably didn’t have bad intentions. However, it is objectively nuts to put a 24-year-old with no experience in charge of a $50B hedge fund. This is very different from giving a 24-year-old a $100k/year charity or EA Funds project. We should not assume that clear skills and intelligence in one domain should ever cleanly map to others.
We should also have learned the lesson from FTX that an inexperienced person speaking with confidence, or dazzling people with technical prowess, is much more likely to simply be inexperienced than a child prodigy. You are not immune to the Dunning-Kruger effect.
If I may speculate, both of these seem to have roots in some of the broader psychological flaws underpinning the ratsphere and the valley more broadly—namely, that intelligence is a fixed, general attribute, and that precociousness is overlooked only because traditional power structures are trying to protect themselves. These are not necessarily or always untrue, but some parts of the community who notice that these are not generally accepted ideas overreact, and overindex on them.
There will be more Aschenbrenners approaching this community in the future, and we should welcome them—but we shouldn’t hand them the controls right away just because they dazzle us.
(P.S. I notice that Leopold’s CV also includes ‘Fund Manager, FTX Future Fund’)
it is objectively nuts to put a 24-year-old with no experience in charge of a $50B hedge fund
That would indeed be nuts, but I think that’s also clearly not what happened. My understanding is that he founded the fund himself, and he initially raised $100-200 million from a bunch of non-EA investors. See https://www.forourposterity.com/
Thanks to their spectacular returns they had a lot more assets under management, but my sense is that this is from normal investors who buy into the “AI is going to be big” thesis and were trying to make a profit.
I also found this article interesting, as well as the fact that Bloom Energy, IREN, Nebius, SanDisk, Micron, CoreWeave, and so on are up 10% to 30% since Wednesday
I don’t think the difference between $100m and $50B is that substantial for my point. Even though they are two orders of magnitude apart, I think the limit on how much money you should trust with someone who has no prior experience managing money is lower than that.
They seem to have made a lot of money by investing in his fund, even after this week, so I find it hard to criticize them for it
Aschenbrenner didn’t launch the fund alone, his cofounder Carl Shulman had previous experience managing money
We don’t know the rest of their portfolio and how situational awareness fits into that (e.g. maybe it’s partially a small hedge against AI making Stripe worthless, or who knows)
However, it is objectively nuts to put a 24-year-old with no experience in charge of a $50B hedge fund.
? It is his fund. He started it.
I feel with the FTX and rat culture reflection, you are reading into the situation too much.
The losses, as you pointed out, happened due to highly leveraged bets. He didn’t expect memory stocks to bleed as much as they did. Besides, it is possible that the fund will survive because his core thesis has paid off exceptionally well so far. He also has a rumored $5B equity stake in a certain large AI company.
Tough time for Situational Awareness LP, but I hope they make it!
His creditors supplied the money—it is their judgement I’m questioning.
The losses, as you pointed out, happened due to highly leveraged bets. He didn’t expect memory stocks to bleed as much as they did.
This is disqualifying if you are running a hedge fund. It’s literally in the name—you are supposed to hedge your positions in order to prevent an unexpected situation from tanking the whole fund.
Besides, it is possible that the fund will survive because his core thesis has paid off exceptionally well so far
We don’t judge funds by whether they don’t go bankrupt, we judge them by their performance against a market index over a long period of time. Even if the positions are net up by 2× or so, this is not particularly impressive in and of itself over a short period, because of survivorship bias. If you make a bunch of stupidly leveraged bets on different sectors, one of them is likely to pay off very well, but not for long, and not through downturns.
The AI sector has monotonically gone up since the release of ChatGPT—any overleveraged investor in this space would be likely to produce incredible gains. If one’s fund gets obliterated at the first market downturn because one was overleveraged, all this proves is that you managed your fund badly, not that you’re some kind of savant genius market whisperer.
(c.f. anything written about Cathie Wood in 2022—a lot of it has aged very poorly)
It seems prudent that banks were quick to margin call him, they probably tightened this kind of stuff after archegos. They still had net assets, so this was early enough.
you are supposed to hedge your positions in order to prevent an unexpected situation from tanking the whole fund
It’s my understanding that this hasn’t been the definition of a hedge fund for some time (ChatGPT agrees). I’m assume that the investors knew that Situational Awareness was not a hedge fund in the sense you’re describing one.
Of course, it might still be possible that investors weren’t aware of the amount of risks the fund was taking on.
Apparently, even $1 invested into Situational Awareness LP 6 months ago is worth $1.78 now. Totally possible he is not a savant whisperer and had enough knowledge and connections to make great bets, but the bets are still paying off. Time will tell if their hedging strategy is actually sound or just lucky.
But I don’t think this is comparable to FTX or contains any lessons for the EA community. Is Situational Awareness LP an EA venture? I could be wrong but I don’t think so. I have always thought of it as “oh I guess Aschenbrenner is doing his own thing, cool.”
I keep seeing people say pangram is a really good tool and will improve AI safety. I feel decently confident that the end state of AI writing detection is that AI’s can ~perfectly replicate human writing (when they want to) and that the wide scale deployment of this tool is analogous to spamming antibiotics on factory farms. I made a 100$ charitable bet at 1:1 odds that within 3 years AI writing detection will be ~useless. would be very eager to be pointed to any strong arguments for why I’m wrong.
Hey all—our experimental microgranting app no hotdog is now accepting submissions for our first $250 grant. The theme is Solarpunk, meant to include anything related to nature, technology, and animals.
Potentially a good opportunity for student groups and young people for whom $250 might make a bigger difference (and who might be excited about the app concept). Please share with anyone who might be interested! The project budget need not be limited to the $250 you receive from us, and both new and ongoing projects are welcome.
As a new person here, i am curious to learn more. I think that Longtermism strongly implies growth-oriented policies, and that might even be the existential risk-minimizing strategy. I did an entire substack post on my thoughts, and would be interested to hear where you might think i am wrong (I probably am, you may have thought a lot more on it than I have, or just be smarter). Also what is the general longtermist take on AI-regulation, and the larger statement made here—We Must Act Now. It seems to me that carte blanche calls to action are not really helpful, but would be open to changing my mind. :-)
I’m curious how the Effective Altruism community addresses fiscal substitution in the “global health and development” cause area? Do any of the top GiveWell recommended charities find ways to track how their impact applies to government spending in their host countries?
“While donor-driven programs undoubtedly saved millions of lives, they also created unintended distortions in national health priorities.
Many governments in sub-Saharan Africa and parts of Asia actually scaled back domestic health investments, as donor program filled key gaps in HIV/AIDS, maternal and child health, and infectious disease control.
In some cases, domestic health budgets shrank in real terms, even as external funding increased. This phenomenon, often referred to as “fiscal substitution,” led to national health systems that were heavily donor-dependent, externally managed, and vulnerable to funding shocks.
Notably, this reliance emerged despite countries pledged to allocate at least 15% of their national budgets to health.
More than two decades later, only a handful has met this target.” (this is from Redefining Global Health in the 21st Century by Michael John Alastair Reid and Eric Paul Goosby).
GiveWell accounts for government spending through quantitative adjustments for ‘fungibility’ and ‘leverage’, estimating both the probability that their funding alters government budgets and the comparative value of those displaced funds. These adjustments are explicitly included in their cost-effectiveness models. You can go through their spreadsheets and check out their inputs and assumptions.
Fungibility and Displacement: GiveWell evaluates whether philanthropic funding frees up a beneficiary government’s domestic budget to spend elsewhere, or conversely, whether the government would have funded the intervention anyway (the counterfactual).
Counterfactual Value: They assign a rough numerical value to what a government would have done with the money otherwise—estimating, for instance, that general counterfactual government spending is worth a fraction (such as roughly 75% or lower, depending on the specific model and sector) of direct cash transfers.
Leverage Effects: If a grant successfully persuades or helps a government scale up a highly cost-effective program using its own domestic resources, GiveWell attempts to credit that amplified impact to the initial philanthropic investment.
What about new charities?
Charities incubated by the main EA global health incubator, Ambitious Impact (AIM), view government adoption not just as a bonus, but as their primary strategy for massive, cost-effective scaling.
Unlike established organisations that often run parallel to public systems, AIM’s incubation model heavily favours non-profits designed to systematically plug ‘execution gaps’ in public health. Their goal is to build, prove, and hand over the playbook to state and national governments.
Lafiya might be a good example. They distribute modern contraceptives in northern Nigeria through a network of community health workers called “Lafiya Sisters”. They do not view themselves as a permanent, independent health provider; their explicitly stated scaling strategy is to “work with governments to execute, pay for, and take ownership of the model”.
Disclaimer: I knew this info but was feeling lazy so got AI to write it up.
Thank you for taking the time to reply! I’m curious what your prompt was? Before posting here I asked claude the same question and got something that was more confusing to me than your answer.
I suspect that this is directly resulting from the $1m of Coefficient funding that the Guardian recently received for FAW journalism. Seem likely they would be therefore be open to EA-aligned applications. Could be a great opportunity to have more aligned FAW journalism. In particular, having someone within the community in the role could be really helpful as a route for existing orgs to get coverage on their work!
One week left—apply to be a Teacher for TARA Round 2, 2026 🧑🏫
• Part-time role ($80 AUD/hr, ~13 hours/week). • Can work in Australian, NZ or Indian time zones. • Lead Saturday sessions and provide remote support. • Must have strong ML skills and completed most or all of the ARENA curriculum. • Applications close this Saturday 8 August 2026, reviewing candidates on rolling basis. • Check out the role description and apply.
USAID took down its archive of evaluation reports, but a high school student happened to have downloaded them for a ML project, and they are now available online again:
https://www.aiddata.org/blog/before-reimagining-development-data-remember-what-weve-learned
Thoughts on the Centre for Effective Altruism, weakly held, n = 1, but feeling like things someone should say:
EAGs seem good
Community Health seems better than replacement. It’s a hard job, not perfect, but I am glad someone is doing it.
the Forum seems like not a place I want to be despite having previously been here a lot. Hard to make sense of what’s important, somehow
I rarely see comms work I think is good. Maybe it’s in places I don’t see, but on twitter, is does EA have a reputation that is helped by CEA? The odd thread I guess?
Vision? I dunno. Is there meant to be some kind of movement-wide vision I am meant to understand and get behind? ‘Do AI safety/policy work’, I guess?
Functionalness? Has there been any EA org as mired in dysfunction as CEA (other than FTX)? Has that changed recently? Base rates aren’t good.
Uni teams, no idea.
I imagine that there are lots of good, hard-working people at CEA and my intention isn’t to hurt. But I think we should see the world as it is and perhaps this is information to some. It is so easy never to say anything and then wonder why nothing changes. If this seems false to you, downvote it. But if it is upvoted when you see it, update on it. CEA in some ways speaks for EA, or has been seen to. What is going on?
People are welcome to DM me to chat about this though I prefer twitter DMs or emails to nathanpmyoung@gmail.com
Thanks for sharing! I mostly agree. If you read my posts from the past 2 years you can see part of the story for why the Forum is the way it is. Broadly, the Online Team has been more focused on supporting the rest of CEA and doing growth-related work, and we’ve been putting less and less FTE toward the Forum over time (~1.5 FTE when I left).
I feel sad that we didn’t push as much on quality and community as I had originally wanted. I found it challenging to try to convince leadership of the impact of doing cultivation work on the Forum, partly because the impact is actually quite nebulous and risks being just entertainment for EAs. Ultimately I still believe that there are tipping points that would cause the Forum to lose the majority of its value, and community building on the Forum can have significant impact by preventing that.
Thanks both! Relatedly, the Online Team is now more empowered to follow a strategy where growth is not the key aim of the Forum (currently our best guess is measuring ‘mind changes’, but stay tuned for precise metrics).
I’m very open to suggestions for how to make the EA Forum more of a place where crucial conversations are being had, and decisions are being made. This includes critiques. I’d love to hear more from people on how they think they Forum is falling short, here, in DM, or over a call.
My take: have you spoken with key decision-makers about incentivising them to write more on the forum instead of discussing things internally in Google Docs or whatever?
Yes, but there is always more of this to do. I.e. some great posts come from me nudging organisations or small conferences to post more. Bottleneck is more time than anything.
Editing to add—Forum readers can help here. If you read something that you think should be on the Forum, nudge the author to post it. If you don’t have time, ping me to do so. Sometimes I offer light editing or accountability support as well to get things over the line.
You’re great Toby, but I do worry that it will be hard to make progress without getting you more capacity for the Forum. I’m glad that we were given more freedom this year, and I hope that means CEA will invest more than the current 1.5 FTE into the Forum, at least in 2027.
With your limited time, I wonder if you should focus more on proactive community building, in the sense we discussed when I first took over the team. I think people’s perceptions of the space matter a great deal to how they engage with it, as you can see from some of Nathan’s comments under the parent quick take. We can chat about it when we meet tomorrow. :)
Keen to hear that case.
And I agree,
...this would be alleviated by another good hire. We are getting into the hiring queue now, so hopefully we’ll be hiring another content manager in the near(ish) future.
A personal reflection:
When I was a Christian, I led a lot of Bible study groups. Christians care a lot about Truth. In theory any question was askable but in practise it was clear that some questions were not. If somebody asked them I maybe allowed some kind of response, and then moved the discussion on. Discussions about the historicity of the Bible often created a mess so you didn’t really want to have them. In practise I recall having them very rarely, if at all.
To me the EA forum often has the smell of the same kind of thing where many important questions are just ‘not the done thing’. Below, James links CEA’s growth discusison. Do I really want to engage with that? Not really. I think I’ll feel like someone who is grumpy at a wedding. But also, from a quick scroll, do I think that CEA getting it’s engagement numbers up is the central thing it should be tracking? Not really. How many career changes are there? What is some notion of total impact?
And, sure, these are hard questions and I am not sure there are good answers. But when I came on here 7 years ago, this felt like a place to have those discussions. And now.. it doesn’t.
Be grumpy! I think the forum lacks spice.
Tier 5 is about career changes. That seems like a sensible thing to be tracking, no?
But it’s a lagging indicator so to me it also makes sense to look at engagement at other points in the funnel.
Do you know how this compares to 80ks similar metric? I dunno, seems a bit weak (only counts careers, not how they differ, no attempt at attribution)
IIRC, 80k stopped using their DIPY(?) metric several years ago.
If you look through their recent reports, they mostly share metrics related to engagement with their programmes/services. And then occasionally there is an additional survey, e.g., the job board survey in their H2 2025 report.
From what I understand, CG likes grantees to report individual narrative case studies, and then these are coded. At least, that’s how it works for the CEA’s CBG programme. So maybe that’s what 80k does as well? And then this is supplemented with engagement metrics.
If that’s the case, it’s pretty similar to CEA’s system.
I’d be interested to hear more about why you didn’t want to engage with posts like CEA’s, and what you mean by “many important questions are just ‘not the done thing’”. Not sure if it’s related, but I think people who push back on CEA tend to get lots of karma still. :)
The top six posts by karma of 2026 are all criticisms of EA, two of them of CEA explicitly
Yeah that’s a fair point. I don’t come on here much. So maybe I am wrong on how receptive people would be. But this doesn’t sound great.
Honestly you’d have a great time at EA in the Lakes and should come to the next one I run. More generally, you want to get yourself into an EA group space of people who make donations (of money and/or time) to EA infrastructure, rather than being in receipt of resources from EA infrastructure. The vibe is totally different: the questions are freer, and we help each other clarify the answers to being more truthful and more loving and more actionable because we all want our donations to be used well.
I am pretty happy with my EA circles, for what it’s worth. I have come here to say somethings about this place and to some extent about the powerful organisation that runs it.
But I’d hear more about your EA in the Lakes thing. Which lakes?
There are some good write-ups on her profile on EA in the Lakes, like this one.
They said growth was their focus, set goals, and, by their own account, did well.
However, the only place I’ve ‘felt’ their growth (beyond their reports) was with EAGxAmsterdam (attendance was up 36%). But I don’t know how much of that was due to things we did vs what CEA did.
My guess is there really is growth, but it’s not ‘felt’ by people like you and me because of where that growth is happening.
And yeah, they’ve struggled to hire for comms. They’ve admitted this. Last I heard, their hiring team was at full capacity hiring for other functions, so I guess they’re prioritising sorting their core programmes before going hard on comms.
I’ve said it before, but I really do think a General Assembly responsible for holding CEA’s board accountable would help keep everyone up to date with what CEA’s doing.
I think possibly the issue is that there’s a lot of effort being put in to manage reputational risks, possibly without quite understanding exactly what the forward-looking reputational risks are (because you’re reacting to bad things that have happened). That’s got to be really tricky.
My standard position on reputational risk is as follows.
https://www.lesswrong.com/posts/SWxnP5LZeJzuT3ccd/pr-is-corrosive-reputation-is-not
To my eye, CEA has repeatedly focused on having good PR, meanwhile Yudkowsky wears a shiny gold hat and talks to Bernie Sanders and has a NYT best seller. Is all of that comms effort working?
Thanks for sharing that post, I thought it provided good insight into how you’re thinking about reputational risk. I did come away from the post wondering why the proposed reputational risk management method doesn’t appear more widespread across competitive firms if it has an edge on the traditional approach.
I don’t know if I understand/agree with your comparison of CEA to Yudkowsky. CEA, as the steward of much of the EA community’s infrastructure, has a lot of authority and responsibility for this community and its people. In order to maintain peoples’ trust and confidence to do that well, I suspect it’s necessary for it to observe a quite conservative form of message discipline that’s more consistent with traditional PR than the other strategy that the post you shared mentions. I can think of a couple examples where I don’t agree with a communications decision I’ve heard of them making, but I don’t think Yudkowsky’s approach that you gesture to would be the right direction to move. Maybe looking to the ways organizations like universities and churches succeed and fail with communications would provide more useful comparisons.
For individuals or organizations that don’t have responsibility for community stewardship, I agree with the praise you give of Yudkowsky’s approach.
I agree the comms is currently not as good as it could be.
I think the massive push to expand effective giving through a large variety of (non-CEA) methods might well take over EA external comms in the next few years. And I’m hoping the effects of that will eventually bounce back and fix CEA. There will just be a lot more comms-understanding EA-committed people in the EA hiring pool.
I’m also very hopeful about NEST, which seems to have understood that things like “hold a Nerdfighters or Kursgesagt meetup” is a thing a local EA group should be doing to attract the people who might become an EA, and is not cringe-bad-wrong-unprofessional.
Anybody in this community focused on international law? Human Rights law, etc.
Also would love to get pointed to any groups of EA-like folks donating to/raising money for international law non-profits (like HRW, etc).
Just a small idea I had: is anyone aware of artists who make “sleep music” on Spotify or any other streaming service who verifiably donate their streaming revenue?
It seems like a low-effort way to have a pretty sizeable impact. Some of these sleep artists are racking up millions of plays. Could maybe get some new people learning about EA as well.
Hank Green has done this on YouTube, and has a video discussing the economics (about $1000/month! although already having an audience presumably helped).
Mental health is one of the most widespread, impactable, yet neglected global health issues of our time. Even knowing mental health conditions are generally under-reported and/ or misrepresented, evidence still shows widespread effects of mental wellbeing on physical health, life satisfaction, and productivity, however it has still not taken off as a major cause area in many EA organizations I have talked to. Why is this?
I should mention there is an argument that many mental health interventions are in fact harmful, mainly via a priming mechanism where when people start to consciously preoccupy themselves with their own mental health, that itself causes psychological suffering. It is a sort of Buddhist argument, I guess.
Abigail Shrier’s Bad Therapy (2024) has probably most discussed version of that case, though I have not read the book, and I am extremely epistemically uncertain about all this, in a way that makes me reluctant to either support or oppose interventions in this area.
Interesting take @alesziegler ! I do believe it is true that too much of any one thing is usually bad, and also believe that people who are not trained/ underqualified taking on mental health is counterproductive. I think people take on these roles or use clinical language that should not be applied to what they are doing precisely because they cannot get the credible help they need and are trying to fill that void. Mental health is extremely subjective and therefore interventions may look like community connection or involvement in sports rather than clinical therapy.
I am also highly skeptical of the legitimacy of Schrier’s claims and her choice in very carefully selected stastistics as evidence. For ecample, contrary to some of Shier’s claims, evidence-based school-based emotional programs have had profound impact, especially on marginalized populations (https://www.sciencedirect.com/science/article/pii/S2773233924000032). Maybe my thoughts on this is best summed up by saying, “Don’t throw the baby out with the bath water”?
In addition to @huw’s great comment, there’s a branch of EA which focuses on well-being, and with that focus mental health interventions often look really effective. Check out the happier lives institute Huw mentioned, times the “WELLBY” and @MichaelPlant
Thank you @NickLaing ! I have done a lot of reading on the Wellby and that is one of the reasons I feel mental health is currently under-represented as a global health issue. It is fascinating to hear how people globally weigh the importance of wellbeing and I’m curious how many other aspects of their lives would be more easily improved if wellbeing increased! @MichaelPlant would love to connect on your work!
G’day Madeline, I run an EA mental health org in India. The reason for this is simply that existing mental health interventions do not compare with GiveWell’s grants on a DALYs/$ basis. In my opinion, the reasons are:
DALY moral weights may be biased against depression
The moral weights of different diseases in the Global Burden of Disease study, which informs DALY estimates, are determined by asking the general public whether they’d prefer to have one disease against another. When you do this with depression, people who haven’t experienced it tend to prefer to have it to many other conditions. However, when you ask people who have experienced it, they choose many very painful conditions over depression. This is one of the widest gaps in the moral weight data. See Pyne et al. 2009 and this post.
Psychotherapy is usually modelled as a short-term effect
Psychotherapy is typically modelled as a treatment, and not a ‘skill’. What I mean by this is that a dose of psychotherapy is assumed to only have effects that decay over a period of time and zero out after that in most CEAs, including those from the Happier Lives Institute. However, many psychotherapy patients will tell you that they learned skills that were useful long after the therapy ended, and there is some limited evidence that psychotherapy’s effects may last decades, or potentially never zero out. If this were true, the effects could be very long-lasting and therefore it would be much more valuable to treat a case of depression.
Suicide prevention isn’t cost-effective if it’s only a short-term effect
Consider that for most of GiveWell’s top interventions, the bulk of the DALYs averted come from ‘saving’ a life—i.e., preventing a death from a disease in a way that allows the person to then go on to live a healthy life, such as preventing a malaria case in an under-5 (which they might die from), even if they go on to catch it after 5 years old.
As a short-term effect, psychotherapy can only postpone a suicide by the length of the treatment effect. But if it were a skill and had some durable long-term effect, it may genuinely prevent one, which would tremendously increase the value of suicide prevention interventions.
Existing interventions haven’t been cheap enough yet
With the exception of some incredible policy work in, for example, reducing toxicity of pesticides commonly used for suicide, existing interventions are still quite expensive. The Happier Lives Institute’s top charities cost ~$40 to treat a single person, while a bednet costs $7. I’m fudging the numbers a bit here, but if we stick with DALYs, psychotherapy is still about an order of magnitude more expensive than it needs to be to look great for EAs.
However, there is work being done to improve that! My charity, Kaya Guides, treated people for $20 each in April, at what we estimate is a similar effect size to the best charities, and we’re confident we can get below $10. We’re using a technique called guided self-help that allows us to dramatically reduce contact hours per participant (and being all-digital helps a lot, too).
Conclusion
Orgs like the Happier Lives Institute have done a lot of advocacy work too, to raise the profile of mental health within EA, and there are plenty of funders that take mental health seriously (in a way that apparently wasn’t true a decade ago). It is, after all, still a nascent space.
Thank you @huw and apologies for my delay- still getting the hang of the forum! I really appreciate these insights and would love to learn more about the work you are doing at Kaya Guides. I have been thinking a lot about what innovation would look like in the mental health space and how the personal nature of mental health makes widespread interventions challenging (for example, medication can not necessarily cure mental illness the way it can cure a physical ailment like an infection).
I have this game theory thing I’ve been working on that involves modifying the Iterated Prisoner’s Dilemma to include death, asymmetric power, and aggressor reputation. Agents’ points are their “power” that dynamically impacts their payoff matrix.
The basic takeaway is that this simple simulation seems to make a case for cooperating with weaker agents, by showing how the cooperative strategies outcompete the aggressive ones in the long run. I think, before I can make a proper post about it, I’ll need to run some analysis to graph out how, for instance, having a higher percentage of cooperative agents increases the odds of survival, which implies a kind of Veil of Ignorance logic towards being cooperative.
Note that I mean cooperative in the sense that you don’t defect first except against aggressors that have defected first against non-aggressors.
With default settings, the most common result of any given run is that a significant number of the cooperative strategies survive and almost all of the aggressive ones die out. Very occasionally, particularly if you adjust the settings are certain way, a single “Opportunist” strategy, that Tit-For-Tats against stronger agents and Defects against weaker ones, will be the only survivor. This seems to imply, at least, to me, that being a cooperative strategy significantly increases your odds of survival, as the alternative is to hope to win a “Highlander” scenario.
I think this is relevant to AI alignment as a variation on Anthropic Capture, the “Hail Mary” approach that Bostrom mentions in Superintelligence. It could work as part of a defence-in-depth, a kind of “infoblessing” that could persuade some AGI to spare us as a kind of Superrational Signalling. While you might assume this only works if aliens are probable, it also functions in a multi-agent scenario where there are several AGI at near peer levels of power. It also potentially could be a way to align a previously unaligned AGI even after it is deployed. If enough AGIs are aligned in this way, their alliance could defeat the unaligned AGIs.
You can run the simulation yourself here: https://paxscientia.com/power/
I have the code and initial analysis here: https://github.com/josephius/power
I realize that a very obvious critique of this work is that the simulation is probably too simple. I intentionally tried to keep it an MVP in its first iteration. I also should, as mentioned earlier, complete a more thorough and rigorous analysis of the apparent results. I’m also keenly aware that it seems like this is a “neglected” path towards alignment, and I’m uncertain whether this is because the idea is a bad one that’s already been discarded by others who are more competent. I know that there are related ideas around Decision Theory, Acausal Trade, and Superrationality, but I’ve never seen this particular kind of effort, which confuses me, because it seems obvious and trivial to try.
My main question to ask is simply, does this seem like something worth pursuing and expanding further, or am I wasting my time on a foolish endeavour?
Is the main idea here that if “have you ever defected against a non-agressor” is the agreed way to decide who to not cooperate with, then cooperative strategies win and then stay cooperative when power imbalances are large? Is there a reason would agents choose to use that instead of any other decision criteria? Also, I wouldn’t say that the obvious critique is that your model is too simple, I would say it’s that your model can easily be unrealistic if you don’t try very hard to check it. Good game theory models are sometimes very simple.
That is sorta the idea yes. Agents would choose this decision criteria mostly because it vastly increases their odds of survival, which allows them to further whatever goals they have. I would hope that this result is obvious enough that many agents will be able to converge on it, increasing the proportion using it, and thus increasing the overall survival rate.
The other takeaway is that, given that humans will be weaker than AGI/ASI, any game theoretic reason for such entities to still cooperate with us can potentially help reduce the existential risk.
I agree that the model requires further scrutiny to determine if it is realistic enough to matter.
The EA Forum Digest might be biased toward posts made on certain days. Had an LLM gather dates for the posts inside Digest #298 through #302, and compare their weekday frequencies against EA Forum posts made within the same window. Probably not enough data (n=50 for Recommended; n=86 for all Digest Posts) to be meaningful.
(Got the idea to check because I was imagining if I was responsible for writing the Digest and thought, “hmmm I feel like if I were sending out emails every Wed, I would be biased toward posts made on certain days because of the availability heuristic and/or the configuration of my weekly reading/writing schedule”)
Interesting! Kind of odd to see the peak being Wednesday (the day I write and send the digest) because posts posted on Wednesday will necessarily have the lowest karma relative to their final amount, and I start off with a list of all posts sorted by karma. Recommended matters most because the rest of the digest is for announcements etc… which are featured when relevant regardless of karma.
Introducing mental tool for rumination that helps me :)
Thinking Loop Breaker
Part 1: overall mindfullness (focus on the now / movement / gratitude practice)
Part 2: in case of persistent thought: 5m timer to arrive at specific decision (including decision to do nothing), offloading to notes with outputs (for every offloading note: next action / decision / no action is required at this time)
P.S.
This it my first EA Forum post! If anyone finds this relevant, let me know! Or if this is not the place — also please let me know
Random idea: what if peopl animal advocates try to establish a norm that you don’t eat meat in space? It currently only requires persuading a small amount of people to take a quite low cost action, food in space is already weird, and the norm has a chance of persisting as space faring scales up. Could eventually be established as law, and hopefully expanded to veganism and no insect farming if that becomes less stigmatized.
I’m probably missing something but why would this be worth doing ?
Because eventually a lot of people might live off Earth, and having a norm against meat eating could reduce animal consumption by a lot.
oh okay I think the misunderstanding was that by “in space” you mean something like “everywhere/on planets that aren’t earth” and not just “in space stations/on spacecraft”
Could you talk more about how you think the norm might persist?
I don’t have a clear idea here! Something like: most astronauts don’t eat meat --> stuff like insect farming in space never takes off --> instead, an animal friendly protein production zero G technique develops into technological maturity --> hard to compete with later
Or
Most astronauts don’t eat meat --> as more and more people live off Earth, they adopt the norm --> counterfactually much less animal consumption in the future.
That makes sense, thanks for responding. I can see where you’re coming from.
I think early spacefaring will probably be a quite competitive space, since there seems to be a lot of diverse commercial and government interest right now. Because of that, it seems like it may be difficult to create any sort of durable norms around something like food sourcing unless there’s a clear winner on efficiency, which I presume would be arrived at independently of whether it involves animals. This might be weak argument if you expect early spacefaring to be dominated by an individual entity like NASA and if you expect government contracts for things like food in space to not be very competitive.
Maybe something that would make this more concrete is looking at whether there are examples of the lock-in of practices like this in the past, and whether we have reason to believe that additional dynamics would make early spacefaring harder to draw historical comparison to. What do you think?
Who should read this: EA or AIS Groups Mentors
epistemic status: quickly written mini-memo, not Claude-generated
I found lots of success with adopting this “Coaching” mindset framing going into my mentorship calls, that can strike the “firm but patient” balance:
this is possibly a (good) remnant from my past weightlifting-coaching backgrounds
accountability
so much value of paid coaching clients is from the social accountability of showing up to the scary arena together
where you’re coming from as a mentor...
from a place of “i’m rooting for you and i want you to succeed which is why i’m giving you this feedback” and “i believe you can win the Olympic gold medal if you fix xyz”
give concrete pointers
as often as possible, don’t be nebulous
eg. say less “go to gym”; say more “show up every Monday at 10am to 11am and do these five workouts, three sets of 10 reps each in this specific order.”
points out errors with immediacy;
build a relationship where error-correction is the norm
eg. “[mid-screen share] … wait wait go back, i think you should remove the picture of Terminator on the poster, because-”
leave mentees feeling like they worked with (an actual) coach and not someone they just answer to ;)
So researchers have used AI to make new viruses and broadcast the achievement to the world!
This could be fine, but given the commentaries alongside it seems there is some disagreement at least.
“But concerns have already been raised that the same technology could be used maliciously to create new disease
“In a commentary accompanying the publication in the journal Science, Dr Thomas Inglesby and Dr Moritz Hanke from the Center for Health Security at Johns Hopkins University wrote that the findings raise “urgent biosafety and biosecurity questions”.
They said it was no longer a question of “whether generative viral genome design will exist” but whether it can be used without “enabling serious harm”.
For example, they said new viruses with the potential to cause disease “should not be pursued”.
The researchers themselves took steps to maximise safety. They excluded viruses that could infect complex organisms from the training database, performed the research on phage rather than viruses that infect people and it all took place in a secure laboratory.”
I’m interested in what biosec people here think. Is making virus’s and publicising the achievement genuinely OK and safe at this point with the safeguards, or could it be net bad?
https://www.bbc.com/news/articles/c5y3j3ngevmo
It seems to me that much of the evaluations/criticisms/defenses of EA conflate the empirical phenomena (whatever you see on with the EA forum, whatever is happening at your local university group, CEA, EAGs, or whatever) with some broader idea of effective altruism (the philosophy stuff / possibly the normative stuff if you believe in that). I am uncertain how to define this “broader idea”, maybe I can call it some theoretical core, maybe I can call it Weber’s idealized type (a deliberately purified theoretical construct that no empirical instance fully matches), maybe I can say something about Wittgenstein and family resemblance (his refute to the demand that a concept must have a common essence), maybe we can each reference our own favorite theoretical definition of EA for the purpose of this point)
But I think never the less it is pretty important to distinguish the empirical manifestation of EA (the type of stuff you can do sociology on), and the theoretical core (to the extent that we can define some semi-coherent notion of such).
We used to call this upper case (ea community/movement) vs lower case (the (meta) philosophy). I think meta normative framework/philsophy is the way I think about it; that is, you can apply EA to most normative frameworks, esp consequentalisty ones (though to many people in the community, the EA framework is actually just applied total utilitarianism). FWIW, and I say this as someone who has many issues with the EA community, the vast majority of EA criticisms from outside the community I see are just highly inaccurate and not worth engaging with from an intellectual POV (but maybe still worth engaging with for movement reputation).
Situational Awareness (the
hedgeunhedged fund) has imploded. It seems much of their gains were from overleveraging their trades, which has then caused them to, in some readings, undergo the largest absolute short-term fund loss in history (over $10B in a few weeks).This is totally relevant to EA because it’s another case of youthful hubris vs veteran experience. EA is a place an elite graduate can meet in a bar with some other cool kids and do a napkin pitch for their first charity with zero field experience and soon bring in a million and where a 43 year old veteran with 20 years in the field in his cause area is virtually ignored because he doesn’t have the youthful EA vibe.
FTX was all EA in the beginning including Aschenbrenner himself who must have still been using a practice razor when he worked there.
The point is EA needs to create a culture that draws in more experienced people who bring their common sense which is informed by real life experience not just classes they had in college. To do this requires thoughtful planning and execution, which should have happened post FTX and maybe this is a reminder. And it’s not just experience, there’s a host of human variety of perspectives that need to all be here so the table benefits from many viewpoints and ways of thinking.
One of the studies on where intelligent people go wrong is that they all follow each other to the same fail point, and they had nobody saying hey, there’s another possible point over here. They lacked perspective. It’s called “Groups of diverse problem solvers can outperform groups of high-ability problem solvers,” which is just common sense to people who love collective intelligence. And love is actions.
I’m an 82-year-old collective intelligence pioneer, with my first book chapter, “Quest for Collective Intelligence,” published in 1995. Recently, my support offered to young CI “geniuses” was overlooked a couple of times because I didn’t have their youthful vibe. I understand that, as every new generation totally legit need to make their own mistake.
Hi George, thrilled to meet you. Yes, I wish I was as chill as you and what you say is true and also every generation shall try to save the next from repeating mistakes.
When I was only a boy I figured out that adults were really far smarter than me and sought to learn from them. Later I worked with the elderly for a while and learned it was a mixed bag, some were old and bitter obviously having made bad choices and failed to forgive. But others were clearly life experience geniuses and so you just had to be discerning in who you listened to. So probably because of this and my eventual passion for CI I just find the failure to seek the value of all perspectives as just leaving so much value on the table, to use a popular EA term. I’m going to DM you, can’t bypass a CI pioneer!
Their gains were from leveraged directional bets and their losses were from leveraged directional bets. Seems they were margin called, so their liquidity was squeezed—poor risk management. It seems that they have learned from this and are now using “fully paid for” derivatives, so presumably call options whose premium they can afford.
Hedge funds are not necessarily hedged, but eg long/short equity hedge funds are (while macro hedge funds can have all kinds of unhedged macro bets on.)
Sadly it appears that adjacency to EA correlates with poor risk management, I’m wondering whether this is somewhat structural (beyond eg the age group…)
They certainly lost a lot of money in July and by being forced to sell, but per the WSJ Aschenbrenner wrote that the fund is still up 80% on the year.
The fund doesn’t seem to be closing, and it looks like they weren’t forced to sell their stake in Anthropic, so it’s not clear to me how much of a failure or a success the project as a whole is so far.
See also this from the Financial Times.
Edit: see also this tweet claiming to be the full letter Aschenbrenner sent to his LPs last night.
The article ends by saying: “It still holds a big chunk of harder-to-sell positions in privately held companies such as Anthropic. It remains to be seen if those companies will live up to their lofty valuations if and when they hit the public markets.” I wonder if this means that the fund did not get wiped out completely—otherwise Citadel (who bought all of their publicly traded stock positions) would have also taken their Anthropic shares?
To this community’s credit, much of the discussion of his paper and career plans was skeptical. I think Aschenbrenner is a useful template for the kind of person that we need to have our guards up around, for a future where large sums of money enter the movement through the Third Wave.
I don’t think he was working in bad faith, and probably didn’t have bad intentions. However, it is objectively nuts to put a 24-year-old with no experience in charge of a $50B hedge fund. This is very different from giving a 24-year-old a $100k/year charity or EA Funds project. We should not assume that clear skills and intelligence in one domain should ever cleanly map to others.
We should also have learned the lesson from FTX that an inexperienced person speaking with confidence, or dazzling people with technical prowess, is much more likely to simply be inexperienced than a child prodigy. You are not immune to the Dunning-Kruger effect.
If I may speculate, both of these seem to have roots in some of the broader psychological flaws underpinning the ratsphere and the valley more broadly—namely, that intelligence is a fixed, general attribute, and that precociousness is overlooked only because traditional power structures are trying to protect themselves. These are not necessarily or always untrue, but some parts of the community who notice that these are not generally accepted ideas overreact, and overindex on them.
There will be more Aschenbrenners approaching this community in the future, and we should welcome them—but we shouldn’t hand them the controls right away just because they dazzle us.
(P.S. I notice that Leopold’s CV also includes ‘Fund Manager, FTX Future Fund’)
That would indeed be nuts, but I think that’s also clearly not what happened. My understanding is that he founded the fund himself, and he initially raised $100-200 million from a bunch of non-EA investors. See https://www.forourposterity.com/
Thanks to their spectacular returns they had a lot more assets under management, but my sense is that this is from normal investors who buy into the “AI is going to be big” thesis and were trying to make a profit.
I also found this article interesting, as well as the fact that Bloom Energy, IREN, Nebius, SanDisk, Micron, CoreWeave, and so on are up 10% to 30% since Wednesday
I don’t think the difference between $100m and $50B is that substantial for my point. Even though they are two orders of magnitude apart, I think the limit on how much money you should trust with someone who has no prior experience managing money is lower than that.
They seem to have made a lot of money by investing in his fund, even after this week, so I find it hard to criticize them for it
Aschenbrenner didn’t launch the fund alone, his cofounder Carl Shulman had previous experience managing money
We don’t know the rest of their portfolio and how situational awareness fits into that (e.g. maybe it’s partially a small hedge against AI making Stripe worthless, or who knows)
? It is his fund. He started it.
I feel with the FTX and rat culture reflection, you are reading into the situation too much.
The losses, as you pointed out, happened due to highly leveraged bets. He didn’t expect memory stocks to bleed as much as they did. Besides, it is possible that the fund will survive because his core thesis has paid off exceptionally well so far. He also has a rumored $5B equity stake in a certain large AI company.
Tough time for Situational Awareness LP, but I hope they make it!
His creditors supplied the money—it is their judgement I’m questioning.
This is disqualifying if you are running a hedge fund. It’s literally in the name—you are supposed to hedge your positions in order to prevent an unexpected situation from tanking the whole fund.
We don’t judge funds by whether they don’t go bankrupt, we judge them by their performance against a market index over a long period of time. Even if the positions are net up by 2× or so, this is not particularly impressive in and of itself over a short period, because of survivorship bias. If you make a bunch of stupidly leveraged bets on different sectors, one of them is likely to pay off very well, but not for long, and not through downturns.
The AI sector has monotonically gone up since the release of ChatGPT—any overleveraged investor in this space would be likely to produce incredible gains. If one’s fund gets obliterated at the first market downturn because one was overleveraged, all this proves is that you managed your fund badly, not that you’re some kind of savant genius market whisperer.
(c.f. anything written about Cathie Wood in 2022—a lot of it has aged very poorly)
It seems prudent that banks were quick to margin call him, they probably tightened this kind of stuff after archegos. They still had net assets, so this was early enough.
It’s my understanding that this hasn’t been the definition of a hedge fund for some time (ChatGPT agrees). I’m assume that the investors knew that Situational Awareness was not a hedge fund in the sense you’re describing one.
Of course, it might still be possible that investors weren’t aware of the amount of risks the fund was taking on.
Yeah this has indeed never really been the definition of a hedge fund. Only a subset of hedge funds are approximately point-in-time market neutral.
I made claude do some quick maths:
Apparently, even $1 invested into Situational Awareness LP 6 months ago is worth $1.78 now. Totally possible he is not a savant whisperer and had enough knowledge and connections to make great bets, but the bets are still paying off. Time will tell if their hedging strategy is actually sound or just lucky.
But I don’t think this is comparable to FTX or contains any lessons for the EA community. Is Situational Awareness LP an EA venture? I could be wrong but I don’t think so. I have always thought of it as “oh I guess Aschenbrenner is doing his own thing, cool.”
I keep seeing people say pangram is a really good tool and will improve AI safety. I feel decently confident that the end state of AI writing detection is that AI’s can ~perfectly replicate human writing (when they want to) and that the wide scale deployment of this tool is analogous to spamming antibiotics on factory farms. I made a 100$ charitable bet at 1:1 odds that within 3 years AI writing detection will be ~useless. would be very eager to be pointed to any strong arguments for why I’m wrong.
Hey all—our experimental microgranting app no hotdog is now accepting submissions for our first $250 grant. The theme is Solarpunk, meant to include anything related to nature, technology, and animals.
Potentially a good opportunity for student groups and young people for whom $250 might make a bigger difference (and who might be excited about the app concept). Please share with anyone who might be interested! The project budget need not be limited to the $250 you receive from us, and both new and ongoing projects are welcome.
More details here: https://nohotdog.love/grant
As a new person here, i am curious to learn more. I think that Longtermism strongly implies growth-oriented policies, and that might even be the existential risk-minimizing strategy. I did an entire substack post on my thoughts, and would be interested to hear where you might think i am wrong (I probably am, you may have thought a lot more on it than I have, or just be smarter). Also what is the general longtermist take on AI-regulation, and the larger statement made here—We Must Act Now. It seems to me that carte blanche calls to action are not really helpful, but would be open to changing my mind. :-)
https://open.substack.com/pub/ulrikahm/p/longtermism-and-the-moral-necessity?r=4cnrou&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true
I’m curious how the Effective Altruism community addresses fiscal substitution in the “global health and development” cause area? Do any of the top GiveWell recommended charities find ways to track how their impact applies to government spending in their host countries?
I found this via Tyler Cowen’s Marginalrevolution.com -
“While donor-driven programs undoubtedly saved millions of lives, they also created unintended distortions in national health priorities.
Many governments in sub-Saharan Africa and parts of Asia actually scaled back domestic health investments, as donor program filled key gaps in HIV/AIDS, maternal and child health, and infectious disease control.
In some cases, domestic health budgets shrank in real terms, even as external funding increased. This phenomenon, often referred to as “fiscal substitution,” led to national health systems that were heavily donor-dependent, externally managed, and vulnerable to funding shocks.
Notably, this reliance emerged despite countries pledged to allocate at least 15% of their national budgets to health.
More than two decades later, only a handful has met this target.” (this is from Redefining Global Health in the 21st Century by Michael John Alastair Reid and Eric Paul Goosby).
GiveWell accounts for government spending through quantitative adjustments for ‘fungibility’ and ‘leverage’, estimating both the probability that their funding alters government budgets and the comparative value of those displaced funds. These adjustments are explicitly included in their cost-effectiveness models. You can go through their spreadsheets and check out their inputs and assumptions.
Fungibility and Displacement: GiveWell evaluates whether philanthropic funding frees up a beneficiary government’s domestic budget to spend elsewhere, or conversely, whether the government would have funded the intervention anyway (the counterfactual).
Counterfactual Value: They assign a rough numerical value to what a government would have done with the money otherwise—estimating, for instance, that general counterfactual government spending is worth a fraction (such as roughly 75% or lower, depending on the specific model and sector) of direct cash transfers.
Leverage Effects: If a grant successfully persuades or helps a government scale up a highly cost-effective program using its own domestic resources, GiveWell attempts to credit that amplified impact to the initial philanthropic investment.
What about new charities?
Charities incubated by the main EA global health incubator, Ambitious Impact (AIM), view government adoption not just as a bonus, but as their primary strategy for massive, cost-effective scaling.
Unlike established organisations that often run parallel to public systems, AIM’s incubation model heavily favours non-profits designed to systematically plug ‘execution gaps’ in public health. Their goal is to build, prove, and hand over the playbook to state and national governments.
Lafiya might be a good example. They distribute modern contraceptives in northern Nigeria through a network of community health workers called “Lafiya Sisters”. They do not view themselves as a permanent, independent health provider; their explicitly stated scaling strategy is to “work with governments to execute, pay for, and take ownership of the model”.
Disclaimer: I knew this info but was feeling lazy so got AI to write it up.
Thank you for taking the time to reply! I’m curious what your prompt was? Before posting here I asked claude the same question and got something that was more confusing to me than your answer.
I can’t remember, to be honest, nothing fancy...
The Guardian is hiring a farmed animal reporter: https://www.linkedin.com/jobs/view/4444587806/
I suspect that this is directly resulting from the $1m of Coefficient funding that the Guardian recently received for FAW journalism. Seem likely they would be therefore be open to EA-aligned applications. Could be a great opportunity to have more aligned FAW journalism. In particular, having someone within the community in the role could be really helpful as a route for existing orgs to get coverage on their work!
I want to see more EA blog/forum posts that don’t use AI images
One week left—apply to be a Teacher for TARA Round 2, 2026 🧑🏫
• Part-time role ($80 AUD/hr, ~13 hours/week).
• Can work in Australian, NZ or Indian time zones.
• Lead Saturday sessions and provide remote support.
• Must have strong ML skills and completed most or all of the ARENA curriculum.
• Applications close this Saturday 8 August 2026, reviewing candidates on rolling basis.
• Check out the role description and apply.