trevor1

Karma: 233

trevor1 Mar 29, 2024, 12:43 AM
−1 points
0 ∶ 0
on: Recent and upcoming media related to EA
Important to note: I archived the Washington Post homepage here and it showed Robinson’s op-ed, but when I went to https://www.washingtonpost.com itself immediately after, at ~5:38 pm San Francisco time, it was nowhere to be found! (I was not signed in for either case).

trevor1 Mar 28, 2024, 3:25 PM
7 points
9 ∶ 4
on: [Linkpost] Leif Wenar’s The Deaths of Effective Altruism
This entire thing is just another manifestation of academic dysfunction
(philosophy professors using their skills and experience to think up justifications for their pre-existing lifestyle, instead of the epistemic pursuit that justified the emergence of professors in the first place).
It started with academia’s reaction to Peter Singer’s Famine, Affluence, Morality essay in 1972, and hasn’t changed much since. The status quo had already hardened, and the culture became so territorial that whenever someone has a big idea, everyone with power (who already optimized for social status) had an allergic reaction to the memetic spread rather than the epistemics behind the idea itself.

trevor1 Mar 27, 2024, 1:40 AM
3 points
0 ∶ 0
on: Crises reveal centralisation
The Dark Forest Problem implies that people centralizing power might face strong incentives to hide, act through proxies, and/or disguise their centralized power as decentralized power. The question is to what extent high-power systems are dark forests vs. the usual quid-pro-quo networks and stable factions.
Changing technology and applications for power, starting in the 1960s, implies that factions would not be stable and iterative trust is less reliable, and therefore a dark forest system was more likely to emerge.

trevor1 Mar 20, 2024, 9:33 PM
1 point
0 ∶ 0
in reply to: Ben Millwood🔸’s comment on: Can the AI afford to wait?
Yep, that’s the way it goes!
Also, figuring out what’s original and what’s memetically downstream, is an art. Even more so when it comes to dangerous technologies that haven’t been invented yet.

trevor1 Mar 17, 2024, 8:37 PM
1 point
0 ∶ 0
on: I’ve read the Effective Altruism Handbook. Here’s what I learned.
Ah, I didn’t know about the EA handbook and would not have found out if not for this post, thanks! It looks pretty good and along with the CFAR handbook, I wish I had known about it many years ago.

trevor1 Mar 15, 2024, 9:40 PM
2 points
1 ∶ 3
in reply to: calebp’s comment on: Unflattering aspects of Effective Altruism
Yeah, a lot of them are not openly advertised for good reasons. One example that’s probably fine to talk about is NunoSempere’s claim that EAforum is shifting towards catering to new or marginal users.
The direct consequence is reducing the net quality of content on EAforum, but it also allows it to steer people towards events as they get more interested in various EA topics, where they can talk more freely without worrying about saying things controversial, or get involved directly with people working on those areas via face-to-face interaction. And it doesn’t stop EAforum from remaining a great bulletin board for orgs to publish papers and updates and get feedback.
But at first glance, catering towards marginal users normally makes you think that they’re just trying to do classic user retention. That’s not what’s happening; this is not a normal forum and that’s the wrong way to think about it.

trevor1 Mar 15, 2024, 4:53 PM
1 point
1 ∶ 4
on: Unflattering aspects of Effective Altruism
My thinking about EAforum over the years has typically been “Jesus, why on earth would they people deliberately set things like that” and then maybe a couple months later, maybe a couple years later, I start to notice a possible explanation, and I’m like “oooooooooooohhhhhh, actually, that might make a lot of sense, I wish I had noticed that immediately”.
Large multi-human systems tend to be pretty complicated and counterintuitive, but it becomes way, way more so when most of the people are extremely thoughtful. Plus, the system changes in complicated and unprecedented ways as the world changes around it, or as someone here or there discovered a game-changing detail about the world, meaning that EAforum is entering uncharted territory and tearing down Schelling fences rather frequently.

trevor1 Mar 13, 2024, 11:07 PM
3 points
0 ∶ 0
on: China x AI Reference List
Sinocism is like Zvi’s blog, except for China Watchers instead of AI safety. It leans a little towards open source, but it’s free and the guy knows the space (though doesn’t know everything).

trevor1 Mar 10, 2024, 5:22 AM
5 points
0 ∶ 0
in reply to: Jason’s comment on: What should the EA/AI safety community change, in response to Sam Altman’s revealed priorities?
I would like to add that certain types of people might be predisposed towards power seeking (and succeeding at power seeking), rather than just being corrupted by power, status, money, or fame.
Social Dark Matter offers some interesting takes on this; it’s more nuanced than it appears e.g. neurotic people might even be more reputation-obsessed but also potentially more likely than the median human to internalize moral values (or, in the case of EA, commit to internalizing moral values in a lasting way). This is purely speculative food for thought to illustrate the complexity of this situation (empirically researching the psychology of different kinds of powerful people is difficult due to nonresponse bias making samples disproportionately stacked towards people who aren’t as powerful as they look).

trevor1 Jan 22, 2024, 6:41 PM
3 points
0 ∶ 0
in reply to: Evan_Gaensbauer’s comment on: Catalogue of POLITICO Reports and Other Cited Articles on Effective Altruism and AI Safety Connections in Washington, DC
Oh, sorry, by profiteers I was referring to people like forum lurkers and hostile open source researchers, not you at all.
My thinking was that this plan works fine with or without funding so long as someone (e.g. you) coordinates it, but it can’t be open-source on EAforum or Lesswrong because the bad guys (not journalists, the other bad guys) would get too much information out of it.

trevor1 Jan 21, 2024, 6:39 PM
4 points
0 ∶ 4
on: Catalogue of POLITICO Reports and Other Cited Articles on Effective Altruism and AI Safety Connections in Washington, DC
My current thinking about this is that EAforum and Lesswrong have confused, mentally ill, or profiteering people trying to do open source research and find ways to maximize damage to EA.
As a result, aggregating criticism in an open and decentralized way will boost the adversary’s epistemics in parallel, and is thus better done in an closed, in-person networked, and centralized way (I made the same mistake a couple years ago).

trevor1 Jan 14, 2024, 5:03 PM
1 point
0 ∶ 0
on: What’s a good intro to AI Safety?
Raemon, a moderator on Lesswrong, recommends Scott Alexander’s Superintelligence FAQ.

trevor1 Jan 11, 2024, 3:53 PM
1 point
0 ∶ 0
on: Call for Expressions of Interest: NYU Wild Animal Welfare Summit
I’m not a scholar, but is it alright if I ask what the best source is to explain wild animal welfare to laypeople? I’m looking for something similar to the Superintelligence FAQ, but selected based on success at explaining wild animal welfare instead of AGI. I know a couple scholars but haven’t introduced them yet and want to make sure I do it right. It’s plausibly a valuable thing to standardize too.
The only sources I’m aware of are the home page of wildanimalsuffering.org, the 80,000 hours page on the topic, and Dylan Matthew’s Vox article, and I have no idea which one has a higher success rate of explaining the concept in a way laypeople are able to take seriously. For example, the 80,000 hours page debunks the naturalistic fallacy quickly and efficiently, which indicates that the authors were serious about writing it well, but otherwise it’s kinda sparse (maybe the authors put a lot of effort into making it short so it’s easier to read and recommend?) and even tries to redirect people to farmed animal welfare instead.

trevor1 Jan 11, 2024, 3:37 PM
−3 points
0 ∶ 1
on: trevor1′s Quick takes
If cryopreservation becomes mainstream, then that’s literally it. Nobody dies, and all of humanity logrolls itself into raising the next generations to be friendly and create aligned AGI.
Even the total sociopaths participate to some degree (e.g. verbally support, often avoid obstruction if they are very powerful). If they don’t have preserved loved ones to protect, they still need a friendly long-term future for themselves to be unfrozen into. They’ll spend many more years alive in the future than the present anyway, because unfreezing a person is orders of magnitude harder than reversing aging or generating a new body for an unfrozen person.
Many other people have probably thought of this already. What am I missing?

trevor1 Jan 5, 2024, 8:52 PM
13 points
0 ∶ 0
in reply to: NickLaing’s comment on: MIRI 2024 Mission and Strategy Update
Oops! I’m off my groove today, sorry. I’m going to go read up on some of the conflict theory vs. mistake theory literature on my backlog in order to figure out what went wrong and how to prevent it (e.g. how human variation and inferential distance causes very strange mistakes due to miscommunication).

trevor1 Jan 5, 2024, 6:31 PM
−51 points
0 ∶ 19
in reply to: NickLaing’s comment on: MIRI 2024 Mission and Strategy Update
Strong downvoted. This isn’t a laughing matter.
I understand what it’s like to think of a really funny joke and not want to waste it. But this isn’t an appropriate environment to substitute charisma for substance.
If EA grows by, say, 30% per year, then that means at any given time there’s going to be a large number of people on the forum who will see this, think it’s normal, and upvote it (reinforcing that behavior). Even if professional norms hold strong, it will still make the onboarding process that much harder and more confusing for the new people, as they are misled into making serious social-status-damaging faux passes, and that reputation might follow them around in the community for years regardless of how talented or valuable they become.

trevor1 Jan 5, 2024, 4:11 PM
2 points
0 ∶ 0
in reply to: Karthik Tadepalli’s comment on: NicholasKross’s Quick takes
What is “reference class skepticism”? This is the first time I’ve heard that phrase, I googled it and didn’t find anything.

trevor1 Jan 5, 2024, 3:02 PM
0 points
0 ∶ 0
on: trevor1′s Quick takes
Planning to simply defy human nature doesn’t usually go very well.
—Robin Hanson

trevor1 Dec 26, 2023, 5:47 PM
4 points
4 ∶ 1
on: Altruism sharpens altruism
Crosspost to Lesswrong?

trevor1 Dec 3, 2023, 7:32 PM
5 points
3 ∶ 1
in reply to: Wei Dai’s comment on: Sam Altman / Open AI Discussion Thread
Two explanations come to my mind:
1. Past Sam Altman didn’t trust his future self, and wanted to use the OpenAI governance structure to constrain himself.
2. His status game / reward gradient changed (at least subjectively from his perspective). At the time it was higher status to give EA more power / appear more safety-conscious, and now it’s higher status to take it back / race faster for AGI. (I note there was internal OpenAI discussion about wanting to disassociate with EA after the FTX debacle.)
1. I think it’s reasonable to think that “Constraining Sam in the future” was obviously a highly pareto-efficient deal. EA had every reason to want Sam constrained in the future. Sam had every reason to make that trade, gaining needed power in the short-term, in exchange for more accountability and oversight in the future.
  This is clearly a sensible trade that actual good guys would make; not “Sam didn’t trust his future self” but rather “Sam had every reason to agree to sell off his future autonomy in exchange for cooperation and trust in the near term”.
2. I think “the world changing around Sam and EA, rather than Sam or EA changing” is worth more nuance. I think that, over the last 5 years, the world changed to make groups of humans vastly more vulnerable than before, due to new AI capabilities facilitating general-purpose human manipulation and the world’s power players investing in those capabilities.
  This dramatically increased the risk of outsider third parties creating or exploiting divisions in the AI safety community, to turn people against each other and use the chaos as a ladder. Given that this risk was escalating, then centralizing power was clearly the correct move in response.
  I’ve been warning about this during the months before the OpenAI conflict started, in the preceeding weeks (including the concept of an annual discount rate for each person, based on the risk of that person becoming cognitively compromised and weaponized against the AI safety community), and I even described the risk of one of the big tech companies hijacking Anthropic 5 days before Sam Altman was dismissed. I think it’s possible that Sam or people in EA also noticed the world rapidly becoming less safe for AI safety orgs, discovering the threat from a different angle than I did.