Minh Nguyen

Karma: 785

I proposed the Nonlinear Emergency Fund and Superlinear as Nonlinear Intern.^[1]

I co-founded Singapore’s Fridays For Future (featured on Al Jazeera and BBC). After arrests + 1 year of campaigning, Singapore adopted all our demands (Net Zero 2050, $80 Carbon Tax and fossil fuel divestment).

I developed a student forum with >300k active users and a study site with >25k users. I founded an education reform campaign with the Singapore Ministry of Education.

^
I proposed both ideas at the same time as the Nonlinear team, so we worked on these together.

Minh Nguyen 8 May 2024 13:27 UTC
4 points
0 ∶ 0
on: Key takeaways from our EA and alignment research surveys
What surprised me the most was that EAs and alignment researchers are more extroverted on average.
In hindsight, I suppose it makes sense. The average alignment researcher I’ve met interacts way more than the average person, certainly the average male, (conferences, communities, research groups, organising parties etc.).

It really contradicted my assumptions, because EA events are unusually introvert-friendly and online communications for EA/alignment are wayyyyy more text-heavy than most communities.
I wonder how much of that is a selection pressure? For me, I’m strongly introverted, but I probably could not have done AI Safety without actively socialising and reaching out to people very frequently.

Minh Nguyen 23 Apr 2024 8:56 UTC
5 points
2 ∶ 0
in reply to: Ben_West🔸’s comment on: On failing to get EA jobs: My experience and recommendations to EA orgs
Paul Graham had a nice take on this:
The more arbitrary college admissions criteria become, the more the students at elite universities will simply be those who were most determined to get in.
I think “actually really want to apply” is not enough of a correlation to base decisions on. The fact is that even qualified+motivated applicants would need to apply to a dozen+ places, and often EA application questions require a lot of thought anyway.
To give an example, lots of EAs are from top unis, and I’m pretty sure the meta-strategy for applying to selective unis is to not fall in love with any particular one and shotgun a lot of applications. This reduces the role of personal fit.
One thing I always thought was interesting would be to have an actual application limit like UK colleges do. I applied to 20 colleges, and I would’ve been fine applying to 5 with improved odds.

Minh Nguyen 11 Apr 2024 21:12 UTC
8 points
2 ∶ 0
on: Mediocre EAs: career paths and how do they engage with EA?
I think there’s 2 different questions at play here:

1. Whether you’re competitive for EA roles—As others have mentioned, EA roles are competitive. To be honest, it really is just a matter of why you want an EA role.

If it’s purely a matter of impact allocation, I think it’s good to just … try hard as you want and let hirers decide? Whenever I’m in an EA hiring round, I genuinely just want the most suitable person to get the job, and the EA hirers I’ve come across have been pretty good at figuring out what candidate they want. I have declined several role on this basis (it wasn’t unilateral, I explained, had a conversation with the hirers and recommended someone I thought was better-suited). The EA community is small and nice, so I’m really rooting for people to find the best role for them.

If it’s a sense of community, for me I’ve been lucky to find friends in EA who will still stick around even if I Don’t Have An EA Job. I think you can definitely find close friends in EA without getting a fulltime role, and our national chapter only has like a dozen regulars.

For me, I just try my best, have low expectations and vaguely tell myself it’ll work out. I also argue more people should look to start orgs, but that’s a separate issue.

2. Whether you feel imposter syndrome because EAs are generally high-achieving—Objectively, EAs are on average, abnormally high-achieving. IIRC, like 20% of EAs have attended to Top 25 university and 40% have attended a Top 100 university. Which is an absolutely bonkers percentage. I do think this kind of comparison is inherent in any social setting.

For me, I just talk to a lot of EAs, and realise they’re just nice nerds who like talking about ideas. The power distance feels a lot lower the more I work with them. If someone works at A Big Well-funded AI Lab and went to A Top 5 PhD Program, then yeah you’re gonna feel intimidated. But like, after a year I just started doing more research which involves pinging people dumb practical questions like “what settings on AWS do I use for a training run” or funny what-ifs like “do you think we can make Deep Learning into Deep Unlearning” (an actual thing btw). Then they’re always really nice and helpful, and I just kinda forget about the imposter syndrome.

There’s also the stoic approach, which is to accept imposter syndrome as motivation to do better without letting it demotivate you. Which is prolly good if you want to improve, but easier said than done.

Minh Nguyen 17 Feb 2024 10:11 UTC
2 points
0 ∶ 0
on: Minh Nguyen’s Quick takes
In AI Safety, it seems easy to feel alienated and demoralised.

The stakes feel vaguely existential, most normal people have no opinion on it, and it never quite feels like “saving a life”. Donating to global health or animal welfare feel more direct, if still distant.

I would imagine a young software developer discovers EA, and then AI Safety, hoping to do something meaningful. But the moments after that, feel much the same as it would a normal job.

Curious if others feel the same.

Minh Nguyen 11 Feb 2024 17:38 UTC
9 points
1 ∶ 0
in reply to: Jonny Spicer 🔸’s comment on: Jonny Spicer’s Shortform
I’ve found not many people bother to play arbitrage with prosocial outcomes.

You essentially need someone to care about prosocial outcomes, be very quantitative and strigent with calculations vs just going by concensus, and be sufficiently motivated to make life changes. In a way, being agreeable to care about others while being disagreeable enough to go against social concensus and gut feel.

Early adopters get to play a lot of arbitrage.

Minh Nguyen 2 Feb 2024 19:49 UTC
13 points
7 ∶ 0
on: Minh Nguyen’s Quick takes
Suggestion for EAGs/EAGx:
Please give more detail on your Swapcard about your research/projects. Even just the title of the research paper, or any links.
I always review the entire attendee list. I actively want to meet people, but if someone just puts “I am trying to get into AI alignment research”, I literally have nothing to go off, and can’t think of a reason to reach out with so few details.

This also saves time during 1-on-1s so you don’t re-introduce yourself/areas of interest every 30 minutes.

If you don’t have a project/experience, you can even put ideas you’re fascinated by, or groups you identify with. I literally Ctrl+F “ADHD”, “startup”, “Singapore” and other keywords related to my research interests.

Minh Nguyen 15 Jan 2024 5:28 UTC
6 points
6 ∶ 3
in reply to: Elizabeth’s comment on: Practically A Book Review: Appendix to “Nonlinear’s Evidence: Debunking False and Misleading Claims”
I mean, the fact that I’ve dealt with a few similar agreements with similar/bigger platforms, and I myself manage a bigger platform than Deck’s, and am managing my own partnership deal, similar to Deck’s.
From my perspective, the events described really aren’t that weird. Like, at all. It’s very strange to me that the Deck story keeps coming up like some sort of smoking gun, when there seems to be nothing unusual about it in the social media marketing space.
The deal itself is normal.
The fact that Deck became inactive afterwards is unfortunate, but normal.
The fact that Deck and his parent sued doesn’t really imply anything about Emerson’s character. A teenager lost interest in an online venture, comes back to demand money when the venture made money. It happens all the time.
At least to me, it’s really weird that everyone keeps talking about it like some strong red flag.

Minh Nguyen 12 Jan 2024 3:17 UTC
3 points
4 ∶ 8
in reply to: Kelsey Piper’s comment on: Practically A Book Review: Appendix to “Nonlinear’s Evidence: Debunking False and Misleading Claims”
Again, not a lawyer so I’m willing to accept I’m wrong.

I think this very much depends on how “services requested by the company” is interpreted.
“render internet related services as requested by the Company from time to time
Failure by the Contractor to meet deadlines for performance of
services or failing to meet the standards required by the Company in the
performing of services
You could argue that access is what’s requested, yes. Personally, I don’t, because access by default is already assumed under a later clause that uses a different wording: “The Contractor agrees not to change any passwords for any Company-related projects without the consent of the Company”
Or you could argue that if Emerson reasonably demands active posting, he is entitled to it under the terms of the contract, and entitled to terminate the contract if this is not fulfilled. We could argue over what constitutes reasonable demands, but I would think it’s more than “almost nothing”.
Not specifying the exact timings didn’t stand out to me. In my experience, it’s not very common to specify exact posting schedules for a long-term brand partnership with a teenager. There’s a certain expectation that teenage influencers are a little bit inconsistent.
In any case, my point with this whole thread is that … the whole Deck thing is really not that much of a red flag? A fairly standard contract was signed with a minor and his parent, who tried to sue by invalidating the entire validity of a contract with a minor+guardian, and it seems Deck wasn’t actively involved the whole time.
I’m not seeing obvious predatory behaviour here. Maybe poorly defined contract terms, but I really don’t see why people keep bringing this up a decade later as evidence against Emerson’s character. Keep in mind, we haven’t seen how many such contracts Spartz Inc signed without any incident.
If this is the absolute worst example an investigation could dig up from Emerson’s 2-decade-long career, it just doesn’t seem very convincing, ya know?

Minh Nguyen 11 Jan 2024 4:20 UTC
18 points
7 ∶ 4
in reply to: Elizabeth’s comment on: Practically A Book Review: Appendix to “Nonlinear’s Evidence: Debunking False and Misleading Claims”
Hi! Sorry for the late response, I was actually handling my own contract for a partnership with a Generative AI startup to co-develop and deploy AI character voice chat onto our platforms w ~ 1 million users. Hopefully, history does not repeat itself.

There are the exact terms I’d found here (Sharing Information About Nonlinear) via Wayback Machine (archive.org).

The relevant parts:

The very first line under “Responsibilities of contractor”:
> The Contractor shall render internet related services as requested by the Company from time to time

> The Contractor agrees not to engage in any behavior that would cause hardship to the Company, such as instances of sabotage. The Contractor agrees not to change any passwords for any Company-related projects without the consent of the Company. Should the Contractor breach this Agreement, the Contractor will, at the Company’s edict, be required to rectify the breach, including but not limited to forfeiture of revenues and a requirement to reveal to the Company the current password of the OMG Facts Twitter account

> This Agreement shall commence on the “Effective Date” defined above and shall terminate one year thereafter, provided, however, that the Company may extend this Agreement for ten additional periods of one year each upon notice to Contractor given at least thirty days prior to the expiration initial expiration date or extended term. However, either party may terminate this contract in writing in the following instances:
(1) If either party is convicted of a criminal offense.
(2) Failure by the Contractor to meet deadlines for performance of
services or failing to meet the standards required by the Company in the
performing of services

Obviously, I am not a lawyer, but there’s indeed several key clauses in there mentioning an obligation to actively perform services, which in the case of OMGFacts would essentially mean making Twitter posts. I don’t think there’s anything there that’s particularly “predatory”?.

Regarding whether Deck performed this responsibility:^[1]
> Here is the untold side of the story: The @OMGFacts Twitter (and Website/YouTube) is produced by us, not him. Adorian has done almost nothing for the past 15 months. Almost every fact you have read from OMG Facts was researched, written, and edited by us.
I will note that I couldn’t find any examples of Deck disputing that he was inactive. Deck’s lawyer instead argued that contracts signed by Deck himself were invalid even if co-signed by his guardians.
Again, I’m not a lawyer, but if Emerson’s main argument was that Deck didn’t uphold his responsibility, you’d think the easiest rebuttal would be that … he did things as per the contract. Instead of arguing that Deck signed the contract without knowing what he was doing:^[2]
Deck’s complaint contends that the contract he signed with Spartz was predatory and designed to dupe the teen into turning over his rights to the OMG Facts trademark without realizing what he was doing. Deck’s attorney, Glenn Peterson of Roseville, argues that the intellectual property issues are moot in any case because California law allows individuals to disavow any contracts signed when they were minors.
Spartz counters that Deck’s mother co-signed the contract as his legal guardian, and that Deck is simply hoping to exploit Spartz Inc.’s hard work for a quick gain.
So even just using public sources, we can establish that there were multiple clauses referencing an obligation to actively post, that Emerson claims Deck did not fulfil this, and Deck never even attempts to deny this, instead trying to invalidate the contract altogether. I don’t see a particularly strong reason to “side with Deck” on this one, unless there’s any claims/evidence that he did perform his responsibility that he hasn’t publicly disclosed for 15 years.
1. ^
  Emerson Spartz, OMG Fact: There are two sides to every story. (archive.org)
2. ^
  California teen’s lawsuit over Twitter feed underscores power of a tweet (jacksonville.com)

Minh Nguyen 5 Jan 2024 13:54 UTC
21 points
13 ∶ 11
on: Practically A Book Review: Appendix to “Nonlinear’s Evidence: Debunking False and Misleading Claims”
I’m not going to lie, I disagreed with a lot of this, especially the conclusion. I don’t quite have the time to respond to all of it, but I’ll address Adorian Deck.
Deck sued for more money. Emerson knew that Deck’s dream was to become a YouTuber, so he threatened to get two top YouTubers to create videos slamming Deck. In response, Deck settled in what Emerson characterizes as “a deal he was satisfied with that gave him significant financial freedom.”
Here’s the timeline as I understand it (it’s been a while):
1. Deck builds a big social media account.
2. Emerson approaches Deck, offering to buy the brand and profit-share, under certain stipulations, including that he continue actively posting.
3. Deck no longer actively posts, essentially ending the contract terms and releasing Emerson’s obligation to pay him.
To me, this seems pretty fair and standard? It doesn’t seem particularly different from a brand acquisition or an influencer marketing program. Lots of founders get acquired and compensated with the stipulation that they keep performing their work under the new entity. Emerson is paying Deck to make posts in his own style. The alternative is to hire someone else to do it, and that’s not ideal if you want to preserve the appeal of the original.
But the OMGFacts brand itself was valuable: that’s why Emerson bought it instead of starting a competing YayFacts Twitter account. Emerson probably paid more to the lawyer who negotiated the contract than he did to Deck—the person who created the brand in the first place.
Social media brands that don’t post have no value.
And, in my experience managing platforms and accounts much bigger than Deck’s, brands are a multiplier, but have very little value on their own. The average person follows hundreds of accounts on multiple platforms. If a Fun Facts account isn’t posting for a year, no one’s gonna notice, and their posts won’t show up because their followers’ feed will be occupied by accounts that do post regularly.
A California law invalidates any contract with a minor that isn’t overseen by a judge—a crucial protection for child actors, who are often exploited by their parents. Emerson characterizes this law as a “bizarre loophole,” which strikes me as a very cavalier attitude towards children’s labor protections.
I mean, sorta? But of the entire influencer ecosystem, many are teenagers and very few have had their brand deals overseen by a judge. I doubt Mr Beast’s first sponsor check was overseen by a judge. You’re technically correct, but in practice it would be quite the demanding ask to have a judge oversee every contract with a minor influencer.

Minh Nguyen 21 Dec 2023 18:50 UTC
12 points
3 ∶ 1
on: Attention on AI X-Risk Likely Hasn’t Distracted from Current Harms from AI
Thank you for this! This is 1 less misconception to deal with.

I always get suspicious when someone treats societal issues like a zero-sum game. Yes, we can worry about more than 1 thing at a time, and it’ often not very productive to frame caring about things as oppositional to caring about another thing

Minh Nguyen 12 Dec 2023 14:30 UTC
45 points
25 ∶ 9
on: Nonlinear’s Evidence: Debunking False and Misleading Claims
How do we prevent the methodology of exclusively seeking and publishing negative information, without fact checking, from becoming an acceptable norm?
Re: Checking that claims are true
Adding on as former Nonlinear intern who was aware of a “falling out” between Alice and Nonlinear for almost a year now:
1. To my knowledge, Nonlinear was given very few/practically no opportunities to respond to the many claims made in “Sharing Information About Nonlinear” before they were posted, despite repeatedly communicating for several months that this counter-evidence was available to Ben and some CEA employees.
2. I understand that the power asymmetry, high-trust environment and ethical standards within EA makes this complicated to resolve. However, my issue is that the vast majority of the claims made were easily verifiable/falsifiable. Things like payment/lack of payment, delivery orders, messages, receipts, who stayed where etc. all have paper trails. If it’s so trivially easy to verify, there is a responsibility to verify!
I’m not against Ben and Alice choosing to post this. I believe we should normalise people exercising their option to speak out publicly. The alternative is being silenced by massive power asymmetry.
What I am against, is the way these allegations were made, which did not prioritise verifying allegations/claims when repeatedly presented with significant, factual counter-evidence.
Why was Nonlinear not given some chance to present counterevidence? It’s clear the initial investigation took months to gather; only a few days (two days, I think) before posting were Kat and Emerson presented with this, after reaching out to Ben several times! Even granting Nonlinear a day to submit an official refutation of the top 5-10 claims for review would have made a difference.^[1] And that’s before factoring in the asymmetry required to refute these allegations with evidence vs making the initial allegations.
I think the handling of this community issue was not healthy for EA/longtermism. Fewer people will read this post than the initial allegations, and Nonlinear’s reputation has definitely been harmed. At best, future whistleblowers are less likely to be believed. I don’t see this as a win for anyone.
Personal Story: How unverified allegations cause harm to real people
Throughout this discussion, there was this undertone that over-weighting Alice’s claims justified the increased reputational risk to Nonlinear, because Kat and Emerson are “better-off” than Alice, so harming them is a more “acceptable” risk because Kat and Emerson will still do fine, whereas Alice is new and less established in EA.
I’d like to say that these allegations don’t just affect Emerson and Kat. It affects the many independent AI Safety researchers Nonlinear helps fund.^[2] It also affects Nonlinear’s other employees. It has personally affected me. I am from Southeast Asia, where it’s much harder to find work in EA/longtermism than in EA hubs. Nonlinear was the first (and currently only) EA org I’ve interned at.
Nonlinear had formally stopped hiring interns when I applied, due to the incidents mentioned above. I contributed to the Superlinear bounty platform as a remote volunteer, without knowing it was owned by Nonlinear, or what Nonlinear was. I had spent so much time trying to contribute to EA part-time, that I wanted to make the experience easier for others.
When I was hired as an intern, I texted my friend “What’s Nonlinear? Are they … like, a big deal?”. My friend explained that having Nonlinear as a reference would help me gain admission to EA conferences, and be taken seriously for EA job applications.
Now that Nonlinear’s reputation within EA has been seriously harmed, I’ve been very concerned about how this affects my ability to contribute within EA. Should I add Nonlinear/Kat as references and risk very negative associations, or omit them and risk being overlooked in favour of other applicants who do have references from prominent EAs? It means a lot to me because, as a non-US/EU/UK citizen, I know I’m always applying at a significant disadvantage.^[3] I will always have fewer opportunities than an EA born in London who goes to a prestigious UK college with an active EA chapter and many EA internship options, who doesn’t have additional Visa requirements. And if I get rejected for a role, I often don’t get to know why.
I didn’t mention this before, because I cared about whether Alice was actually abused. I had a hunch they were making false claims, but I didn’t want to invalidate victims who might be telling the truth. As of now, this seems … less likely.
These allegations do cause harm: to me, to other Nonlinear employees trying to contribute to EA and the people Nonlinear helps through our work.
In the future, please verify these more seriously. Thank you.
1. ^
  The first time I asked Nonlinear about the allegations, it took me maybe 5-10 minutes to figure out there were multiple misleading statements, since I was shown message logs.
2. ^
  In fundraising, reputation matters. Serious, public allegations of abuse means funders are (rightfully) hesitant, and less funding goes to researchers.
3. ^
  If you are reading this and trying to get into AI Safety/longtermism from a non EA hub, do reach out and I’ll try to reply when I can! We gotta support each other >:)

Minh Nguyen 2 Dec 2023 5:16 UTC
5 points
5 ∶ 0
on: Philosophers Against Malaria Fundraiser
a very minor and inconsequential point:

I initially read this as “Philosophers Against Malaria Fundraisers” and thought it was gonna be an interesting essay about how AMF fundraisers are bad, actually.

Minh Nguyen 25 Nov 2023 4:46 UTC
3 points
0 ∶ 0
in reply to: VictorW’s comment on: VictorW’s Quick takes
I find that asking EAs (or anyone, really) for open-ended feedback tends not to yield novel insight by default. EAs tend to have high openness, and as long something passes the bar of “this has plausible theory of change, has no obviously huge downside and is positive EV enough to be worth exploring” , it isn’t subject to particularly intense scrutiny. Consider also that you have thought about the problem for days/weeks whereas they’ve only thought about it for maybe 10-20 minutes
Haven’t found a 100% perfect solution, but usually I express my 2-3 most pressing doubts and ask them whether there’s a possible solution to those doubts. It scopes the question towards more precise, detailed and actionable feedback.
Alternatively, if the person has attempted a similar project, I would ask them what goals/principles they found most important, and 2-3 things they wish they knew before they’d started.

Minh Nguyen 23 Nov 2023 7:55 UTC
7 points
3 ∶ 2
in reply to: NickLaing’s comment on: Elizabeth’s Quick takes
I will say I also never use the Drowning Child argument. For several reasons:
- I generally don’t think negative emotions like shame and guilt are a good first impression/initial reason to join EA. People tend to distance themselves from sources of guilt. It’s fine to mention the drowning child argument maybe 10-20 minutes in, but I prefer to lead with positive associations.
- I prefer to minimise use of thought experiments/hypotheticals in intros, and prefer to use examples relatable to the other person. IMO, thought experiments make the ethical stakes seem too trivial and distant.
What I often do is to figure out what cause areas the other person might relate to based on what they already care about, describe EA as fundamentally “doing good, better” in the sense of getting people to engage more thoughtfully with values they already hold.

Minh Nguyen 22 Nov 2023 2:14 UTC
3 points
0 ∶ 0
in reply to: James Herbert’s comment on: James Herbert’s Shortform
On CB, my views that are half informed by EA CBs and half personal opinions:
1. Very casual events—If you are holding no events for a long time and don’t have much capacity, just hold low-stakes casual events and follow-up with high-engaged people afterwards. Highly-engaged people tend to show up/follow up several times after learning about EA anyway. 80-90% of the time, I think having some casual events every few weeks is better than no casual events.
2. Bigger events—Try to direct highly-engaged people to bigger and/or more specialised events. The EA community is big and diverse, and letting people know other events exist lets them self-select better. When I first explored beyond EA Singapore, I spent 2 months straight learning about every EA org and resource in existence, individually reviewing all the Swapcard profiles at every EAG. That was absolutely worth the effort, IMO.^[1]
3. 1-on-1s are probably still important − 1-on-1s with someone of very similar interest areas or career trajectories are the most valuable experiences in EA, in my opinion. Only 10% of 1-on-1s are like this, but they more than make up for the 90% that don’t really go anywhere. As much as I try to optimise, this seems to be a numbers game of just finding and meeting a lot of potentially interesting people.^[2]
4. Online resources—For highly-engaged EAs, important information should be online-first. I’m of the opinion that highly-engaged/agentic new EAs tend to read a lot online, and can gain >80% of the same field-specific knowledge reading on their own. This especially holds true in AI Safety, which is like … code and research that’s all publicly available short of frontier models. I think events should be for casual socials, intentional networking and accountability+complex coordination (basically, coworkers).
1. ^
  If you want the ⁸⁰⁄₂₀ for AI Safety, check out aisafety.training, aisafety.world, check EA Forum, Lesswrong and Alignment Forum once a week (~1 hour/week), check 80k job board and EA Opportunities Board once a week (~20 minutes/week), review forum tags for things like prizes, job opportunities and research programs to see what programs were run last year that will be run again this year.
  It is possible to capture all open opportunities this way. The rest is just researching interesting orgs, seeing which ones you vibe with and engaging with them. This is just for AI Safety, for other cause areas I’d expect the same amount of time spent passively checking.
2. ^
  My personal view is people should slightly prioritise “potentially interesting” over “potentially useful”. The few times I’ve met EAs just because they’re high-ranking, the conversation is usually generic and could have been had by Googling and emailing/texting.

Minh Nguyen 21 Nov 2023 13:02 UTC
6 points
2 ∶ 0
in reply to: James Herbert’s comment on: James Herbert’s Shortform
I have heard many people argue against organising relatively simple events
I’m actually very surprised to hear this. What does the “common view” presume then?

Personally, I see 3 tiers of events: 1. Any casual, low-commitment, low stakes events 2. Big EA conferences that I find quite valuable for meeting lots of people intentionally and socially 3. Professionally-focused events (research fellowships, incubators etc)

I think “simple” events like 1 are great for socialising and meeting new people. While 2 and 3 get more done, I don’t think the community would feel as welcoming if the only events occurring were ones where you had to be fully professional.

Sometimes I still want to interact with EAs, but without the expectation of “meeting right” or “networking”. I suspect this applies especially to introverts and beginners. Even just going to a conference with the expectation of booking lots of 1-on-1s vs just chilling feels very different.

Minh Nguyen 21 Nov 2023 9:16 UTC
10 points
3 ∶ 0
in reply to: John G. Halstead’s comment on: CEA is fundraising, and funding constrained
Perhaps the thinking was that FTX is not directly relevant to this specific post?

But I agree, clearly FTX is on the minds of many people reading this, so avoiding the elephant in the room is not working.

Minh Nguyen 18 Nov 2023 2:17 UTC
52 points
9 ∶ 8
on: Sam Altman fired from OpenAI
Found this on Reddit: Anxious_Bandicoot126 comments on Sam Altman is leaving OpenAI (reddit.com)
I feel compelled as someone close to the situation to share additional context about Sam and company.
Engineers raised concerns about rushing tech to market without adequate safety reviews in the race to capitalize on ChatGPT hype. But Sam charged ahead. That’s just who he is. Wouldn’t listen to us.
His focus increasingly seemed to be fame and fortune, not upholding our principles as a responsible nonprofit. He made unilateral business decisions aimed at profits that diverged from our mission.
When he proposed the GPT store and revenue sharing, it crossed a line. This signaled our core values were at risk, so the board made the tough decision to remove him as CEO.
Greg also faced some accountability and stepped down from his role. He enabled much of Sam’s troubling direction.
Now our former CTO, Mira Murati, is stepping in as CEO. There is hope we can return to our engineering-driven mission of developing AI safely to benefit the world, and not shareholders.
Obviously just speculation for now, but seems plausible. The moment the GPT store was released I thought:
“wow that’s really good for business … wow that’s really bad for alignment”

Minh Nguyen 12 Nov 2023 6:44 UTC
11 points
2 ∶ 0
on: “Effective Altruism, Longtermism, and the Problem of Arbitrary Power” by Gwilym David Blunt
Honestly, I like that this essay tries to engage with and understand EA more than other critiques I’ve seen. Usually when I see an article about EA’s association with billionaires/elites, it tends to be a lot less substantive.
I do agree that EA decision making generally biases towards privilege. EA orgs pretty openly bias towards recruiting and hiring from prestigious universities, which are significantly overrepresented in EA demographics. I also recall data from Spencer Greenberg that essentially placed EAs as ideologically centrist, as oppose to more left-leaning social movements popular with similar demographics.
I think her critique here is actually quite sound:
Effective altruism, when it began, had ascetic norms designed to constrain power, but slowly and surely they have been eroded by arguments about ‘effectiveness’, and as they receded the absence of strong institutional constraints became apparent. This leaves effective altruism and longtermism exposed to what we may call ‘the despotism trap’.
It rejects constraints against these visionaries as the pettiness of small minds. They drape themselves in the regalia of liberty to challenge government, but their interest is in being able to exercise uncontrolled power. The excuse is that the ends will justify the means. We can see how these attitudes have influenced effective altruism when it overlaps with ‘philanthrocapitalism’, whereby people like Gates bring the mechanisms of the market into philanthropy on the grounds that is more effective.
I think this is a pretty accurate description. EA and EA-adjacent orgs are generally funded by rich philanthropists, and receive less support and collaboration with governmental bodies. A lot of EA-adjacent or recommended orgs exist in the private sector. I think at the very least, we have to acknowledge that funding sources do affect culture. EA happens to be disproportionately funded by tech billionaires, so EA is more likely to be sympathetic to the views and ideologies of tech billionaires.
I think where outside critics get it wrong, is the interpretation that because EA is funded by social elites, it actively sides/colludes with social elites. I’m not sure how to convey it, but outside coverage seems to imply that EA is essentially a front for social elites to gain more power without actually doing any good, when my take is that EA began in elite college campuses, and so the most accessible sources of support and funding just happened to be social elites. Most notably, coverage of EA often focuses more on ties with social elites than the actual work done by EA orgs in various cause areas. The average article I read contains maybe 1-2 brief sentences about the fairly complex work/research done by EA orgs, and the rest of the article is just speculating about its ties w rich elites.

Minh Nguyen

Suggestion for EAGs/​EAGx:

Please give more detail on your Swapcard about your research/​projects. Even just the title of the research paper, or any links.

Re: Checking that claims are true

Personal Story: How unverified allegations cause harm to real people

Suggestion for EAGs/EAGx:

Please give more detail on your Swapcard about your research/projects. Even just the title of the research paper, or any links.