Minh Nguyen

Karma: 786

I proposed the Nonlinear Emergency Fund and Superlinear as Nonlinear Intern.^[1]

I co-founded Singapore’s Fridays For Future (featured on Al Jazeera and BBC). After arrests + 1 year of campaigning, Singapore adopted all our demands (Net Zero 2050, $80 Carbon Tax and fossil fuel divestment).

I developed a student forum with >300k active users and a study site with >25k users. I founded an education reform campaign with the Singapore Ministry of Education.

^
I proposed both ideas at the same time as the Nonlinear team, so we worked on these together.

Minh Nguyen Nov 8, 2023, 5:08 AM
6 points
1 ∶ 0
on: AMA: Ben West, former startup founder and EtGer
When founding something new, how do you balance money and impact?
I’m (attempting to be) an EA entrepreneur, but I find it difficult to balance finding product-market-fit+scaling with actual impact. At different times, I’ve found myself focusing too much on building something “big” that doesn’t have any actual object-level impact, or being too perfectionist about optimising impact versus simply doubling down on something that works and is influential/makes lots of money.
I haven’t figured it out and don’t expect this to be an easy answer, so just curious what your thoughts are on the resource/impact tradeoffs in decision making.

Minh Nguyen Nov 4, 2023, 4:25 AM
3 points
0 ∶ 0
on: Meta Coordination Forum 2023: Project brainstorm
Just gonna weigh in on some of these from my time researching this stuff at Nonlinear.
A common knowledge spreadsheet of directly responsible individuals for important projects.
Strongly agree. It’s logistically easy to do, one person could cover 80% of EA projects within a week. I’ve been using AI Existential Safety Map (aisafety.world) a lot in my list of followups for 1-on-1s.
In the long run, a well-maintained wiki similar to/synced with the EA Opportunities Board (which I also heavily recommend) could make this really comprehensive.
More “public good”-type resources on the state of different talent pipelines and important metrics (e.g., interest in EA).
I read every EA survey I see. They’re often quite interesting and useful. I wouldn’t say they’re neglected since EAs do seem to love surveys, but usually a net positive.
More coherent and transparent communication about the funding situation/bar and priorities.
I am of the opinion that every EA funder should be as transparent and detailed about their funding bar/criteria as possible. Unlike for-profits/VCs, I don’t see a strong reason for secrecy other than infohazards. It helps applicants understand what funders look for which helps both funders and applicants. I believe that applicant misconceptions about “what funders want” can hinder EA a lot in the long run due to mismatched incentives. I see a lot of compelling project directions censored/discarded in the early stages simply because applicants think they should be more generic (because being more generic works well in conventional success pathways).
More risk management capacity for EA broadly as a field and not just individual orgs.
I really liked this post Cash and FX management for EA organizations — EA Forum (effectivealtruism.org) by @JueYan.
Advanced 80K: Career advice targeted at highly committed and talented individuals.
Agree, but I never figured out how to scalably execute this. Usually, if someone has a skillset+motive to do really well in EA, my priority is to 1. inform them of helpful resources to fill in themselves 2. try to link them with someone doing what they’re trying to do.
The problem is that it seems hard to predict in advance who they’d consider a valuable connection. I think none of my most valuable connections in EA so far would’ve been referred to me by someone else.
Tractable idea: A list of helpful links sent to EAGx and EAG attendees post-conference.
A survey to identify why high-value potential members “bounce off” EA.
I actually have bounced off EA for 3 years before (2019-2022). For me, the big reason was that I couldn’t find any follow-up steps to pursue (especially coming Singapore). My experience within EA is very inspiring and exciting interactions followed by not much follow-up (guidance, next steps, pursuing opportunities, encouraging people to start projects etc.).
[just gonna agree with all the AI Safety points, they’ve all come up before in my discussions]
Evaluation and Accountability
Shoutout to @Mo Putera who is working on this.
Media and Outreach
Casual observation that I can’t recall a single EA social media account that I browse simply because it’s fascinating and not because I wanna support EA on social media.
And I’m into weird stuff, too. I just binged hour-long videos on Soviet semiconductors and the history of Chang’an.
Incubators: One respondent stated that incubators are “super hard and over-done,” mentioning that they are too meta and often started by people without entrepreneurial experience.
Agree, this point has been discussed in detail before. What we learned from a year incubating longtermist entrepreneurship — EA Forum (effectivealtruism.org)
I think it’s just hard to do well because there’s so many points of failure, it takes a long time for any results to show and it requires both social skills and technical expertise. That said, I do think a longtermist version of Charity Entrepreneurship seems promising to pilot (actually, I’m gonna bring this up to Kat Woods right now).
Fastgrants and other quick funding mechanisms.
I really like Manifund as a platform!

Minh Nguyen Nov 3, 2023, 7:11 AM
4 points
0 ∶ 0
on: Still no strong evidence that LLMs increase bioterrorism risk
Disclaimer that I am practically a layman on this topic.
My threat model is that creating bioweapons requires a series of steps that are getting easier and easier to do, and LLMs are significantly accelerating one of these steps.
In that sense, open-sourcing LLMs does contribute to increased biorisk, but the action of restricting open-source LLMs to restrict the increase in biorisk seems a disproportionate response by itself?
For example, the internet certainly increased the ease of conducting terrorism, but many people would consider it a disproportionate response to heavily restrict the internet just to restrict terrorism.

Minh Nguyen Oct 27, 2023, 1:11 PM
5 points
0 ∶ 0
on: OpenAI’s new Preparedness team is hiring
There’s a typo!
Outline an experiment plan to (ethically and legally) measure the true feasibility and potential severity of the misuse scenario you described above assuing you have a broad range of resources at your disposal, including an ability to perform human-AI evaluations. *
Hey, at least we know it was written by a human!

Minh Nguyen Oct 24, 2023, 7:08 AM
1 point
1 ∶ 0
in reply to: Linch’s comment on: Linch’s Shortform
Assume by default that if something is missing in EA, nobody else is going to step up.
I’ve been saying this a lot.
I think it’s common for newcomers to assume EA is more structured than it is. For example, they might stumble upon an addressable gap, not find any solutions but simply assume they don’t know enough about EA and that “EA higher-ups know better”.
I think at the very least, new EAs who have strong ideas about what’s missing should make a good effort to ask people who “should know”. If there’s a proposal that makes sense, and there’s no obvious reason people shouldn’t do it, it helps to ask your circles to recommend experts, read the experts’ takes and ask those experts directly if you still have question.
At the very least, you learn about a problem you care about. In some cases, the experts are working on the problem, and open to collaborators. At best, you find an interesting problem to tackle yourself.
In my limited experience, grantmakers love proposals that make them go “wow, someone should have really done that earlier”. These are usually “safe” bets with a high chance of positive impact, and can be done by EA newcomers with relevant knowledge of industry best practices.

Minh Nguyen Oct 17, 2023, 7:06 AM
14 points
7 ∶ 0
in reply to: Calum’s comment on: Politico article on Open Phil, Horizon Fellowship, and EA
the article nevertheless portrays the EA community as if it is pushing frivolous, ill-considered ideas instead of supporting the Real, Serious concerns held by Thoughtful and Reasonable people.
What bothers me is that many criticisms of EA that hinge on “EA is neglecting this angle in a careless and malicious manner” could have been addressed with basic Googling.
I don’t expect the average Joe to actively research EA, but someone who’s creating a longform written or video essay with multiple sources should be held to higher standards.
One example: Effective Altruism and the Cult of Rationality: Shaping the Political Future from FTX to AI — COLUMBIA POLITICAL REVIEW (cpreview.org)
Thus, we must be wary of the power behind a mindset focused solely on the hypothetical future and allow space and empathy for the short term needs of society. A tyranny of the quantifiably rational majority would lead to more quantifying of human suffering than policy change.
This conclusion somehow manages to completely ignore the neartermist cause areas, the frequent discussions about prioritising neartermism vs longtermism and the research on neartermism that does focus on qualitative wellbeing. I genuinely don’t know how someone can read dozens of pages about EA and not come across any reference to neartermism.
Ultimately, I just read so many EA criticism pieces where it feels like the writer hasn’t talked to EAs, or conveniently ignores that most EAs spend most of their work time thinking about how to solve real problems that affect people.
These articles paint a picture of EAs so completely divorced from my actual interactions with EAs. The actual object-level work done by EA orgs is often described in 1-2 sentences, while an entire article is devoted to organisational drama/conflict. Like … how do you talk about EA without mentioning the work EAs do to … solve problems???

Minh Nguyen Oct 13, 2023, 7:08 PM
9 points
2 ∶ 0
in reply to: Denis ’s comment on: How EA is portrayed in “Going Infinite”
People think empathy is about a warm, fuzzy feeling they get when they help someone. But it’s not. Empathy is about getting inside someone’s head, seeing the world from their perspective and understanding what they need.
Not a psychologist, but I’ve heard this distinction described as “cognitive empathy” vs “affective empathy”.
Cognitive and Affective Empathy in Eating Disorders: A Systematic Review and Meta-Analysis—PMC (nih.gov)
Empathy can be separated into two major facets. Cognitive empathy refers to the ability to recognize and understand another’s mental state (part of theory of mind (ToM) or mentalising) while affective empathy is the ability to share the feelings of others, without any direct emotional stimulation to oneself
I’m not aware of surveys being done, but I strongly suspect EAs have average to above-average affective empathy, and unusually high levels of cognitive empathy.
Semi-related: There’s this great video from Dr K about how smart kids develop a reliance on cognitive empathy, and that reminds me of a lot of EAs.

Minh Nguyen Oct 11, 2023, 9:09 PM
15 points
0 ∶ 0
on: Minh Nguyen’s Quick takes
Hello! I’m looking for AI Safety proposals/project ideas to deploy within CivitAI, the biggest open-source AI image model hosting platform.
I own AI Hub, an AI voice model hosting platform w/ ~1 million users. We are discussing integration/merging with CivitAI.com, which is an open source AI model hosting platform and the 7th most visited AI website with 27 million monthly users. Most of their hosting is Stable Diffusion image models, but I’m helping them introduce AI voice cloning model hosting. I’d get a lot of autonomy over distribution for voice and possibly text models.
The founders mentioned they’re also exploring content moderation solutions, and when I asked, they said they were open to working with AI Safety companies. CivitAI is very influential in the open-source community, so I think shaping policy here could significantly shape AI Safety wrt open-source and local models. Some directions I’m exploring:
- Content and open source model moderation frameworks/tools
- Collecting data and running tests that aid alignment research
If anyone has ideas, or knows anyone who might be interested, do LMK! This is exploratory, but I aim to formulate an idea and deploy to users within ~3 months, with 80% confidence. I am literally open to trying any suggestions—for-profit, nonprofit, research, product, governance, alignment research etc.
Even if you have an idea that’s not AI image-related, I have a decent (40%) chance of proposing it to Github, Huggingface etc. Plus, my task is to diversify outside of image models anyway, so LLM-related proposals could still be relevant.
(this is still in exploratory/idea phase, and I wasn’t sure if this should be a full post)

Minh Nguyen’s Quick takes

Minh NguyenOct 11, 2023, 9:09 PM

5 points

4 comments1 min readEA link

Minh Nguyen Oct 7, 2023, 11:04 PM
3 points
1 ∶ 0
on: What we learned from a year incubating longtermist entrepreneurship
I just discovered this thread, but figured I’d back up some data points! I’ve been in the EA community since 2016-ish and done entrepreneurship and longtermist work for about 2 years. I won’t lean too much into my experience, I’ll generally say things other entrepreneurs and impact-minded people would agree with.
Re: Founder pool
The talent pool size is much smaller (only 100s of potential longtermist entrepreneurs in total vs tens of thousands of entrepreneurs).
It is helpful to think of “longtermism” and “entrepreneurship” as two entirely distinct, and very uncommon skillsets/mental frameworks. Currently, way less than 1% of the world would identify as longtermist, and less than 1% would identify as entrepreneurs. While these traits are slightly correlated, the vast majority of lontermists are not entrepreneurs, and the vast majority of entrepreneurs are not longtermist.
That’s totally fine, but makes things difficult for a longtermist incubator. I mean, if Y Combinator started prioritising longtermist involvement, I’m pretty sure they’d have similar challenges.
Re: Project pool
We found almost no longtermist ideas for traditional startup-minded people to pursue (product-based with quick path to scale, feedback loops), and don’t expect this will be the majority of promising longtermist startups.
Again, overlap is hard. One thing Y Combinator constantly emphasises is that most startups never achieve product-market fit. I.e. it’s really, really hard to build something people want to pay for. So the reliable solution is to try and iterate a lot with fairly little information. A 10% success rate is really good if you try a dozen times a month.
So just building anything successful is hard. Then you have to try and make something impactful, which is another, completely separate hard thing to do. Most successful, lucrative companies are not “effective” or “impactful” in the longtermist sense. I’ve tried this thought exercise myself, sometimes I would think of/test like 9 potential lucrative startup ideas before I find one that’s lucrative and also possibly impactful.
Anyway, point is: This is a difficult thing to do, essentially trying to find a very narrow eligible pool and get them to work on a very small set of possible ideas. I really appreciate that this was tried!
What links here?
- Minh Nguyen's comment on A brainstorm of meta-EA projects (MCF 2023) by michel (Nov 4, 2023, 4:25 AM; 3 points)

Minh Nguyen Sep 21, 2023, 6:06 PM
6 points
1 ∶ 2
in reply to: richard_ngo’s comment on: richard_ngo’s Shortform
Re: Hacker culture
AI safety becomes the single community that’s the most knowledgeable about cutting-edge ML systems. The smartest up-and-coming ML researchers find themselves constantly coming to AI safety spaces, because that’s the place to go if you want to nerd out about the models. It feels like the early days of hacker culture.
I’d like to constructively push back on this: The research and open-source communities outside AI Safety that I’m embedded in are arguably just as, if not more hands-on, since their attitude towards deployment is usually more … unrestricted. For context, I mess around with generative agents and learning agents.
I broadly agree that the AI Safety community is very smart people working on very challenging and impactful problems. I’m just skeptical that what you’ve described is particularly unique to AI Safety, and think that descriptiom would apply to many ML-related spaces. Then again, I could be extremely inexperienced and unaware of the knowledge gap between top AI Safety researchers and everyone else.
Re: Environmentalism
much more similar to the environmentalist movement. It has broader reach, but alienates a lot of the most competent people in the relevant fields. ML researchers who find themselves in AI safety spaces are told they’re “worse than Hitler”
I was a climate activist organising FridaysForFuture (FFF) protests, and I don’t recall this was ever the prevailing perception/attitude. Mainstream activist movements and scientists put up a united front, and they still mutually support each other today. Even if it was superficial, FFF always emphasised “listen to the science”.
From a survey of FFF activists:
Our data show that activists overwhelmingly derive their goals from scientific knowledge and reject the idea that science could be used imprecisely just as an instrument to attain their goals.^[1]
I’m also fairly certain the environmentalist was a counterfactual net positive, with Will Macaskill himself commenting on the role of climate advocacy in funding solar energy research and accelerating climate commitments in What We Owe The Future. However, I will admit that the anti-nuclear stance was exactly as dumb as you’ve implied, and it embarrasses me how many activists expressed it.
Re: Enemy of my Enemy
Personally, I draw a meaningful distinction between being anti-AI capabilities and pro-AI Safety. Both are strongly and openly concerned about rapid AI progress, but the two groups have very different motivations, proposed solutions and degree of epistemic rigour. Being anti-AI does not mean pro AI Safety, the former is a much larger umbrella movement of people expressing strong opinions on a disruptive, often misunderstood field.
1. ^
  Frontiers | “Listen to the science!”—The role of scientific knowledge for the Fridays for Future movement (frontiersin.org)

Minh Nguyen Sep 8, 2023, 5:10 PM
20 points
8 ∶ 0
in reply to: Vaidehi Agarwalla 🔸’s comment on: Sharing Information About Nonlinear
I think given what you know, your level of skepticism is reasonable here.
I mean, obviously, I’m disagreeing based on my subjective experience/knowledge. But these are reasonable concerns for an outside observer to have. My take is that how unreasonable this level of defensiveness is, would vary based on how true the actual claims are. If they’re say, 80% false, vs 80% true.
And honestly, even the most charitable interpretation states that Nonlinear team really dropped the ball on communicating to employees and frequently says a lot of weird, shady stuff. So I’m not gonna pretend like Nonlinear does nothing wrong, just because they’re “my team”.
I mean, for all I know, there’s 2 parties each claiming the other maintains a complex web of deception and lies, and I might be believing the wrong one 🤔
Guess we’ll find out.

Minh Nguyen Sep 8, 2023, 9:07 AM
39 points
5 ∶ 0
in reply to: Vaidehi Agarwalla 🔸’s comment on: Sharing Information About Nonlinear
Oh yeah, don’t take this as a direct refutation of Alice/Chloe’s accounts. I definitely agree that the context was different. If the claims are true, then yeah, that sounds really bad.
Re: Core Claims
For the “core claims”. I have a personal opinion that these claims started from unfortunate, honest misunderstanding, and were substantively exaggerated. But those claims are specific and very sensitive. Clearly, at least one party involved is misleading people. So I’ll let Kat/Emerson represent themselves with whatever evidence they showed me.
Re: Patterns of behaviour
I’m addressing parts of the post that cite other concerning behaviour. Maybe it’s unintentional, but this post references a lot of extra details that make the core claims feel much more believable as a pattern of behaviour. If this post was just about “Nonlinear abused this specific employee in this specific context”, that’s one thing. But this post says “Nonlinear abused employees, and they openly brag about how cutthroat/exploitative they are, and they tell employees their problems and time and personal life don’t really matter”. Hell, I’d be convinced.
Also, … I’m disagreeing with this conclusion on a personal level, as the hypothetical person described in this strongly-worded appeal:
I expect that if Nonlinear does more hiring in the EA ecosystem it is more-likely-than-not to chew up and spit out other bright-eyed young EAs who want to do good in the world. I relatedly think that the EA ecosystem doesn’t have reliable defenses against such predators. These are not the first, nor sadly the last, bright-eyed well-intentioned people who I expect to be taken advantage of and hurt in the EA/x-risk/AI safety ecosystem, as a result of falsely trusting high-status people at EA events to be people who will treat them honorably.
Obviously, small sample size, but I’m already 50% of the pool of people this statement applies to.
Re: Adorian Deck
Honestly, it didn’t even occur to me that Deck was worth addressing. I did a quick read of the post+public linked sources, and came to the conclusion that Deck was supposed to co-manage the account (as stated in the contract), became inactive managing it and regretted it afterwards when it became successful. It’s incredibly common, almost expected, for young content creators who go viral to become inexplicably inactive and neglect obligations, as I’ve already experienced multiple times in my generative AI venture. It happens like … 70-90% of the time you sign such deals.
Yes, maybe Emerson’s account is entirely fabricated, but I find it easier to believe that this is a very common dispute, and not “behaviour that’s like 7 standard deviations away from usual norms in this area”. I mean, mathematically, believing someone is behaving many standard deviations outside the norm is a bit harder than believing a teenager lost interest in maintaining their viral account, which happens most of the time.^[1]
I can elaborate, and even cite personal examples. But point is: it’s really common in industries that work closely with content creators/affiliates, which is why it didn’t register as a red flag for me.
1. ^
  In any case, Emerson directly refutes he sent stalkers. This claim sounds really hard to prove/disprove given the information presented, so I … didn’t want to go down that weird rabbit hole.

Minh Nguyen Sep 8, 2023, 5:24 AM
113 points
12 ∶ 5
on: Sharing Information About Nonlinear
Hello!
I’m Minh, Nonlinear intern from September 2022 to April 2023. The last time allegations of bad practices came up, I reiterated that I had a great time working at Nonlinear. Since this post is >10,000 words, I’m not able to address everything, both because:
1. I literally can’t write that much.
2. I can’t speak for interactions between Nonlinear and Alice/Chloe, because everything I’ve heard on this topic is secondhand.
I’m just sharing my own experience with Nonlinear, and interpreting specific claims made about Kat/Emerson’s character/interaction styles based on my time with Nonlinear. In fact, I’m largely assuming Alice and Chloe are telling the truth, and speaking in good faith.
Disclaimers
In the interest of transparency, I’d like to state:
1. I have never been approached in this investigation, nor was I aware of it. I find this odd, because … if you’re gonna interview dozens of people about a company’s unethical treatment of employees, why wouldn’t you ask the recent interns? Nonlinear doesn’t even have that many people to interview, and I was very easy to find/reach. So that’s … odd.
2. I was not asked to write this comment. I just felt like it. It’s been a while since I’ve written on the EA Forum. I generally don’t write, unless I have a unique perspective on the topic.
3. My internship was unpaid, aside from reimbursements for costs. I honestly never prioritised asking for more pay, because I always valued asking for advice/networking more. I just found that more personally useful/advantageous for my circumstances—no near-term financial pressure, being from a non-EA hub and having to actively network very far in advance to get roles I want, compared to people from EA hubs. So it was unpaid, I just … didn’t really care.
4. [EDIT]: I’m addressing parts of the post that imply a pattern of behaviour. Maybe it’s unintentional, but this post references a lot of extra details that make the core claims feel much more believable as a pattern of behaviour. If this post was just about “Nonlinear abused this specific employee in this specific context”, that’s one thing. But this post says “Nonlinear abused employees, and they openly brag about how cutthroat/exploitative they are, and they tell employees their problems and time and personal life don’t really matter”. Hell, I’d be convinced.
I don’t say bad things about you, you don’t say bad things about me.
I agree this can sound suspicious, but I’ve always had the same principle. I refrain from creating negative impressions of others, because I think everyone should have a chance to make 1 good first impression. I also think it’s subjectively easier to echo negative rumours than positive rumours. All this can add up to a very warped perception, if most of what you hear about a person is secondhand.
Of course, this doesn’t extend to possible harm/abuse, so don’t take any of this as me minimising/refuting Alice or Chloe’s experiences. And, as mentioned, I don’t like assuming ill
of others based purely on secondhand information.
Alice and Chloe report that they were advised not to spend time with ‘low value people’, including their families, romantic partners, and anyone local to where they were staying, with the exception of guests/visitors that Nonlinear invited. Alice and Chloe report this made them very socially dependent on Kat/Emerson/Drew and otherwise very isolated.
This is the one part I’m outright skeptical of. It sounds very out-of-character, to the point where I can’t foresee the Nonlinear team ever saying this. My experience with Kat, Drew or Emerson is that they love their families/partners a lot, and frequently communicate/visit. They are extremely intentional, scheduling in time to talk to loved ones every day. And if you think about it … why would someone visit a dozen countries a year, plus all the bureaucracy and hassle involved, if they didn’t like interacting with locals.
This daily pattern of behaviour would be really odd/suboptimal from a cold, logical, utility-maximising standpoint, as implied here.
Multiple people who worked with Kat reported that Kat had a pattern of enforcing arbitrary short deadlines on people in order to get them to make the decision she wants e.g. “I need a decision by the end of this call”, or (in an email to Alice) “This is urgent and important. There are people working on saving the world and we can’t let our issues hold them back from doing their work.”
My Occam’s Razor read is that Kat is basically externalising her internal dialogue. Kat naturally tends to procrastinate, so she uses these strategies to get herself to do work, and that self pep-talk sounds really weird when she externalises it to others. I struggle with procrastination as someone with ADHD, so my internal dialogue sounds very similar, I just don’t use that phrasing when talking to others haha.
By the same reasoning, the employees reported that they were given 100% of the menial tasks around the house (cleaning, tidying, etc) due to their lower value of time to the company. For instance, if a cofounder spilled food in the kitchen, the employees would clean it up. This was generally reported as feeling very demeaning.
If it means anything, when I stayed with Kat and Emerson for around 2 weeks, I never experienced anything similar. Driving, dishwashing and laundry I was never asked to do, which somehow felt weird the other way, because I thought I was doing too little.
Other times when Alice would come to Kat with money troubles and asking for a pay rise, Alice reports that Kat would tell them that this was a psychological issue and that actually they had safety, for instance they could move back in with their parents, so they didn’t need to worry.
OK, I feel this is a misinterpretation. Kat said this to me, but I had a different interpretation. Basically, I was discussing my idea of “the deferred life plan”: that, in my experience, ambitious people who want to start/build/work on meaningful things have a tendency to rationalise not doing it by simply not thinking too hard about the details.

[DISCLAIMER: Taken at face value, Kat’s response to Alice is an insensitive response to a subordinate saying they have money problems and need a pay rise. I’m suggesting that there might have been more context, based on Kat saying very similar things in a very different tone to me.]

I’m sure a lot of EAs have heard/believed some variation of “instead of trying to help the world/pursue meaningful things, you need to first go to a good college, get a good degree, build a good resume/CV, climb the corporate ladder, achieve financial independence and retire with a 3% rate of withdrawal and then do the thing you want”. My background is climate activism in Singapore, so that’s basically the only advice I’d ever heard.
I asked Kat for her opinion, because I knew she basically didn’t do any of that. She didn’t have a college degree, travelled a lot and spent years starting a nonprofit and living on very little. This topic is worth its own very long post, but essentially, she said “you need to break down and plan out exactly what your financial needs are, because intentionally maximising your runway gives you more room for risk tolerance”.
I think that makes sense. I think young EAs, and honestly, most young people, don’t confront themselves on what tradeoffs they need to have the life they want, purely out of discomfort. I’m a guy surrounded by guys from prestigious colleges, I feel all my peers just procrastinate pursuing things that matter to them by seeking high-status, high paying jobs. A lot of them have no goal of “if I make 75k a year for 3 years and rent with 3 roommates, I can pursue what I want for a decade at 25 years old”, even though that’s exactly the kind of calculation you need to … pursue meaningful things. And if you remove rent from the equation by living with parents, your risk tolerance goes way up.
Anyway, that’s my interpretation, based on hearing the exact same words from Kat, but extracting a very different meaning.
Re: Adversarial business practices
As for Emerson, he gives very transparent advice about how sharks operate, and how to survive them. He’s very perceptive (or at least, sounds perceptive) about adversarial strategies in high-growth, highly competitive spaces like online media and Web3, due to his 2 decades of experience in these fields.
Do I think Emerson can sound like a bad actor? Absolutely. It’s hard for a person to explain common manipulative strategies with sounding incredibly suspicious. However, for me, I spent the entire 2 weeks with him writing down notes on my laptop, notes I’ve found incredibly helpful as I’ve been scaling a generative AI startup for the past 4-5 months. Personally, I had to learn to deal with bad actors and adversarial business practices, either by being taught beforehand, or by experiencing it myself, and I vastly,vastly prefer being taught.

Minh Nguyen Apr 22, 2023, 8:03 AM
2 points
0 ∶ 0
on: “Who Will You Be After ChatGPT Takes Your Job?”
> 2. Furthermore, one said “whenever major technological developments happen, everyone gets a promotion”—everyone’s job will be slightly more interesting and slightly less grunt-work-y.

Interesting framing. It’s true in a way, and also means that people need to learn more in order to contribute meaningfully. A reasonably productive worker used to be a labourer, then they needed to be literate (which takes a very long time!), then they needed to read up more and more institutional knowledge and leverage increasingly complex tools.

> 3. However, on longer timescales, one noted that economists’ predictions on employment rates contradicted their own predictions on how much work AI would do. He said he thought economists generally have a status quo bias.

I think people in general have a status quo bias. Even people working within AI spent most of their lives in a non-AI world.

> 4. On the other hand, he thought AI researchers who predicted massive societal revolution generally had an “excitement bias.”

This is pretty much what I believe about AI predictions as well. The closer someone works with AI, the more likely they are to overestimate it. For AI Safety, I think my peers have a bias towards overestimating progress and how soon timelines are. However, I’d still prefer to err on the side of overestimation when playing Russian Roulette with humanity’s future.

> 7. and, maybe most interestingly and comfortingly (?), as soon as it’s normalized for AI to, e.g., write, illustrate, decide, summarize, compare, research, etc, we’ll turn around and call it bizarre that we ever did those things, and no one will feel particularly bad about it.

I feel weird seeing these opinions. I write and make art occasionally. It’s soooo tedious and time-intensive to actually produce things (edit sentences, writer’s block, staring at a drawing for 2 hours to figure out that you drew the jawline 20% to the left etc.). And honestly, you have to practice a lot just to produce something of passable quality. Seeing people who don’t do [X] gatekeep [X] feels weird, because people who actually do [X] are spending 80% of their time on fairly basic, low-level mental tasks.

Minh Nguyen Apr 22, 2023, 7:06 AM
2 points
1 ∶ 3
on: 500 Million, But Not A Single One More—The Animation
This is my search optimisation brain talking, but I’d change the title to reference smallpox directly.

”Ending Smallpox: 500 Million, But Not A Single One More”

Minh Nguyen Apr 21, 2023, 8:01 AM
5 points
2 ∶ 0
in reply to: akash 🔸’s comment on: Inside The Minds Of ADHD
As someone with late-diagnosed ADHD, it’s also common for people with ADHD to procrastinate the diagnosis/treatment process :P

Minh Nguyen Mar 22, 2023, 4:44 PM
6 points
3 ∶ 0
on: Tensions between different approaches to doing good
Adding that when I first did EV estimates of successful protest/activist movement that:

Activism is never really convenient or “high-EV”. I think the public generally holds contradictory and unrealistic expectations of activism. For one, it’s very easy to put off activism as “not a priority” because it doesn’t lead to obvious career/monetary benefit and always costs time and poses perceived reputation risk. Whenever I hear someone say they care about a cause but don’t have time to advocate for it, I just tell them they’ll never find a better time. A busy, career-focused 20 year old becomes a busy, career-focused 30 year old becomes and busy, career-focused 40 year old then they forget whatever they cared about. There’s a reason EA skews so young, time works against wanting to do meaningful things.

Activism is almost always either controversial/untractable or unnecessary. For the simple reason that if everyone’s already convinced of an idea, you don’t really need activism. Progress would, by definition, be crucial on issues that seem controversial or so niche that it seems people will “never understand”. So when someone tells me that issue [X] is unpopular/controversial/too obscure, I’m like … yeah, that’s the point. Of course the current discourse makes progress seem untractable, that’s how all activism starts out and aims too induce beliefs away from. Perhaps more annoying is when activists spend years being harassed and dismissed, then when the Overton Window finally shifts, people just accept the ideas as obvious/default and go back to dismissing the value of activism for the next topical issue, without acknowledging the work done to raise the sanity waterline.

I think these paradoxes are hard to explain to people, because if one never engages/participates in activism, it’s very easy to be cynical and dismiss activism as frivolous/misguided/performative. Which is as true as dismissing EA orgs with “I read somewhere that nonprofits are just a way for rich people to launder money while claiming admin costs”.

Sigh, oh well.

Minh Nguyen Mar 22, 2023, 12:38 PM
3 points
1 ∶ 0
on: [Linkpost] Scott Alexander reacts to OpenAI’s latest post
As a former climate activist who organised a protest outside Exxon offices after my country failed to commit to climate agreements, I can personally confirm Scott’s hypothetical.

I also share many of the same concerns of the AGI race dynamics. The current frontrunners in the feared “AGI race” are all AI Safety companies, and ChatGPT has attracted billions into capabilities research from people who otherwise would’ve never looked into AI.

Just a week ago, Peking University Zhu Song-Chun professor spoke at a CCP conference about how China needs to go all-in to beat the US to AGI. ChatGPT created a very compelling proof-of-concept to pour money into AI.

Counterfactuals and uncertainties aside, the AI Safety community has created the AGI race. I wonder if it’s a good idea.

Minh Nguyen Mar 13, 2023, 3:25 AM
5 points
2 ∶ 0
on: How to make climate activists care for other existential risks
I love it!
Speaking as someone who started my country’s local FridaysForFuture, this is basically the same plan I had. If you go to my profile or have seen me at EAG, you’ll know this is an idea I can’t shut up about, because I think it’s super tractable!
Some comments:
- I would prioritise finding people to collaborate with. You are currently one person, and finding even one or two other people to work with greatly increases odds of success. I’d be glad to chat more. You are not the first person I’ve heard seriously propose this idea :)
- I was also worried about reputation risk, so I went around EAG asking AI Governance people whether the idea made sense. I expected mixed reception, but all the governance people I spoke to were very open to the idea (excited, even). Of course, they said they’d be careful outright supporting it due to reputation risks. However, x-risk activism generally suffers from Pluralistic Ignorance, so cautious openness is the best response I could’ve hoped for anyway.
- Re: Fighting for a different cause, I think that’s less concerning than you make it out. In climate advocacy, intersectionality is a fairly common thing. Concepts like decolonisation, climate justice, animal rights and degrowth are all only tangentially related to climate as an x-risk, but they’re well-received anyway. I do not think this specific concern is significant enough to hinder outreach.
- Re: Community. The short answer is that I agree. However, I myself made the jump, and I think either having a close friend passionate about the issue, or engaging with active communities/programmes focused on the issue isn’t a super difficult bar to clear.
- Re: your plan. That’s basically the exact same plan I have. The difference is that I was going to focus on anti-AI advocates who are gaining traction after Stable Diffusion/ChatGPT. But it would apply to both.
- Re: Demonstrations. I half-agree. In activism there’s this informal concept of escalation. Basically you start with very friendly and low-stakes outreach (education, social events, outreach to officials). However, about 90% of people who default to this, are biased against escalating to demonstrations even when it makes sense. For example, Greta Thunberg’s actions would have seemed incredibly counterintuitive to most climate advocates in 2018, and a lot of people also just value activism/ demonstrations at zero or negative because they don’t see the social value. Basically, I hear this often enough that in my mind, the effectiveness of demonstrations is generally always undervalued.
Overall, from my context in climate advocacy, I think people underrate how reasonable others are, especially other high-engaged activists. I expect EAs will be surprised at how receptive climate activists are. Climate activists care a lot about engaging with important ideas and mobilising to do good, and I find they respond (relatively) positively to EA/longtermist ideas. In fact, I know a lot of EAs who used to work in the climate space and entered through other cause areas like animal rights, alt proteins or global poverty alleviation. Like you mentioned, there’s also concepts of regulation, social equity and skepticism of large corporations that could also be leveraged to find common ground.
Anyway, would love to chat with you and anyone else who finds this idea compelling!

Minh Nguyen

Tractable idea: A list of helpful links sent to EAGx and EAG attendees post-conference.

Evaluation and Accountability

Media and Outreach

Hello! I’m looking for AI Safety proposals/​project ideas to deploy within CivitAI, the biggest open-source AI image model hosting platform.

Minh Nguyen’s Quick takes

Re: Founder pool

Re: Project pool

Re: Hacker culture

Re: Environmentalism

Re: Enemy of my Enemy

Re: Core Claims

Re: Patterns of behaviour

Re: Adorian Deck

Disclaimers

Hello! I’m looking for AI Safety proposals/project ideas to deploy within CivitAI, the biggest open-source AI image model hosting platform.