det

Karma: 374

det Mar 22, 2025, 2:21 PM
10 points
2 ∶ 1
in reply to: Greg_Colbourn ⏸️ ’s comment on: 80,000 Hours is shifting its strategic approach to focus more on AGI
The comment you replied to
- acknowledges the value of x-risk reduction in general from a non-longtermist perspective
- clarifies that it is making a point about the marginal altruistic value of x-risk vs AW or GHW work and points to a post making this argument in more detail
Your response merely reiterates that x-risk prevention has substantial altruistic (and non-altruistic) value. This isn’t responsive to the claim about whether, under non-longtermist assumptions, that value is greater on the margin than AW or GHW work.
So even though I actually agree with the claims in your comment, I downvoted it (along with this one complaining about the downvotes) for being off-topic and not embodying the type of discourse I think the EA Forum should strive for.

det Nov 15, 2024, 12:12 AM
1 point
0 ∶ 0
in reply to: Joe Rogero’s comment on: Flipping Out: The Cosmic Coinflip Thought Experiment Is Bad Philosophy
I take your claim in the post not to be “the fact that an offer is +EV is one strong reason to be in favor of it,” but rather “you ought to take the cosmic coin flip, risking the universe, just because it is +EV.” (Because being +EV definitionally means the good scenario is super unbelievably good, much better than most people considering the thought experiment are probably imagining.)

But even within the thought experiment, abstracting away all empirical uncertainties, I have enough philosophical uncertainty about EV maximization that I don’t want to take the bet.

det Nov 13, 2024, 12:48 PM
1 point
0 ∶ 0
in reply to: Holly Elmore ⏸️ 🔸’s comment on: Cutting AI Safety down to size
Again, just giving my impressions from interacting with AI safety people: it doesn’t seem to me like I get this impression by drawing a larger circle—I don’t recall hearing the types of arguments you allude to even from people I consider “core” to AI safety. I think it would help me understand if you were able to provide some examples? (Although like I said, I found examples either way hard to search for, so I understand if you don’t have any available.)
I still disagree about the Dial post: at the end Zvi says
Seeing highly intelligent thinkers who are otherwise natural partners and allies making a variety of obvious nonsense arguments, in ways that seem immune to correction, in ways that seem designed to prevent humanity from taking action to prevent its own extinction, is extremely frustrating. Even more frustrating is not knowing why it is happening, and responding in unproductive ways.
So my read is that he wants to explain and understand the position as well as possible, so that he can cooperate as effectively as possible with people who take the Dial position. He also agrees on lots of object-level points with the people he’s arguing against. But ultimately actually using the Dial as an argument is “obvious nonsense,” for the same reason the Technology Bucket Error is an error.

det Nov 13, 2024, 1:36 AM
2 points
1 ∶ 0
on: Flipping Out: The Cosmic Coinflip Thought Experiment Is Bad Philosophy
I agree that there’s been a phenomenon of people suddenly all agreeing that all of SBF’s opinions were wrong post-FTX collapse. So I appreciate the effort to make the case for taking the deal, and to portray the choice as not completely obvious.
To the extent that you’re hoping to save “maximizing utility via maximizing expected value,” I think it’s still an uphill battle. I like Beckstead and Thomas’s “A paradox for tiny probabilities and enormous values” on this, which runs essentially the same thought experiment as “flip the coin many times,” except with the coin weighted to 99.9% heads (and only your own life in play, not the universe). They point out that both positions, “timidity” and “recklessness”, have implausible conclusions.
I’m ultimately quite philosophically troubled by this “concentrating all the value into narrow regions of probability space” feature of EV maximization as a result (but I don’t have a better alternative on hand!). This makes me, in particular, not confident enough in EV-maximization to wager the universe on it. So while I’m more sympathetic than most to the position that the coin flip might be justifiable, I’m still pretty far from wanting to bite that bullet.

det Nov 12, 2024, 12:39 PM
22 points
3 ∶ 1
on: Cutting AI Safety down to size
I agree with the main ideas of this post. But I want to flag that, as someone who’s engaged with the AI safety community (outside of the Bay Area) for several years, I don’t recognize this depiction. In my experience, it’s very common to say “even a [1, 5, 10]% chance of AI x-risk justifies taking this very seriously.”
I don’t have time to track down many examples, but just to illustrate:
- Neel Nanda’s “Simplify EA Pitches to ‘Holy Shit, X-Risk’”: “If you believe the key claims of “there is a >=1% chance of AI causing x-risk and >=0.1% chance of bio causing x-risk in my lifetime” this is enough to justify the core action relevant points of EA.”
- Toby Ord estimated AI x-risk at 10% in The Precipice.
- Scott Alexander is sympathetic to libertarianism and put AI x-risk at 20-25% in 2023.
To address another claim: “The Dial of Progress” by Zvi, a core LessWrong contributor, makes the case that technology is not always good (similar to the “Technology Bucket Error” post) and the comments overwhelmingly seem to agree.
I’m sure someone has said that 10-25% x-risk would not be worth addressing due to libertarianism—but I don’t believe I’ve heard this argument, and wasn’t able to find someone making it after a few minutes of searching. (But it’s a hard thing to search for.)
I don’t doubt that Holly is accurately reporting her experiences, and she’s almost certainly engaged more widely than I have with people in AI safety. I wouldn’t be surprised if there are people saying these things in the Bay Area. But I don’t have the impression that they represent the mainstream of the AI safety community in any forum I’m familiar with (Twitter, LW, and definitely EA).

det Nov 1, 2024, 4:02 AM
33 points
9 ∶ 0
on: EA is bad for mental health (and I’m tired of pretending it’s not)
While I agree with many points in this post, I think it would be stronger if it engaged more with the existing discussion within EA on mental health, on the Forum and elsewhere.

For example:
- The “Self-care and wellbeing in the EA community” tag contains over 270 posts, including some of the most highly-upvoted posts on the site.
- 80,000 Hours has dedicated significant attention here:
  - Their most popular podcast episode (iirc) focuses on mental health in EA
  - They have an episode with a mental health practitioner about perfectionism and related challenges
  - The first three sections of the career guide’s “How to be successful in any job” chapter emphasize self-care, with the second section stating that dealing with poor mental health should be the top priority
- Several programs have popped up over the years offering to provide or connect EAs with mental health services. Currently Rethink Wellbeing is active and provides CBT and IFS-based peer-facilitated programs. (There may be others I’m forgetting.)
A few of these seem to me like the sort of thing the suggestions were asking for, e.g. “a few podcast episodes that could succinctly demonstrate the way therapy might explore common blind spots held by EAs, providing a rapid update with less resistance than other methods.”
I’ve personally experienced mental health challenges due to EA, so I’m certainly not saying the problems are all solved, or that the resources above cover everything. Publishing one podcast doesn’t solve a community-wide problem. But parts of this post read to me as suggesting these resources and discussions don’t exist, so I want to provide an alternate perspective.

det Oct 25, 2024, 1:35 AM
2 points
0 ∶ 0
in reply to: yanni kyriacos’s comment on: Yanni Kyriacos’s Shortform
What makes the agriculture transition stand out as the “point of no return”? Agriculture was independently invented several times, so I’d expect the hunter-gatherer → agriculture transition to be more convergent than agriculture → AGI.

det Oct 10, 2024, 11:54 AM
4 points
2 ∶ 0
in reply to: Henry Howard🔸’s comment on: Ways I see the global health → animal welfare shift backfiring
I didn’t say universal or 50% support.
Sorry you’re right, you didn’t say this—I misread that part of your comment.
I still think your framing misses something important: the logic “50% of people are women so I think women’s suffrage had a pretty strong support base” applies at all points in time, so it doesn’t explain why suffrage was so unpopular for so long. Or to put it another way, for some reason the popularity and political influence of the suffrage movement increased dramatically without the percentage of women increasing, so I’m not sure the percentage of people who are women is relevant in the way you’re implying.
The idea that you can go regulating without considering public support/resistance is silly
On the other hand I didn’t say this! The degree of public support is certainly relevant. But I’m not sure what your practical takeaway or recommendation is in the case of an unpopular movement.
For example you point out abolition as an example where resistance caused massive additional costs (including the Civil War in the US). I could see points 1, 3, 7, and possibly 8 all being part of a “Ways I see the Quaker shift to abolitionism backfiring” post. They could indeed be fair points that Quakers / other abolitionists should have considered, in some way—but I’m not sure what that post would have actually wanted abolitionists to do differently, and I’m not sure what your post wants EAs to do differently.
Maybe you just intend to be pointing out possible problems, without concluding one way or another whether the GH → AW shift is overall good or bad. But I get a strong sense from reading it that you think it’s overall bad, and if that’s the case I don’t know what the practical upshots are.

det Oct 10, 2024, 1:38 AM
3 points
1 ∶ 3
in reply to: Henry Howard🔸’s comment on: Ways I see the global health → animal welfare shift backfiring
I’m not super knowledgeable about women’s suffrage, but
- It was not universally supported by women (See e.g. here; I couldn’t quickly find stats but I’d be interested).
- Surely the relevant support base in this case is those who had political power, and the whole point is that women didn’t. So “50% support” seems misleading in that sense.
I could similarly say “>99.999% of animals are nonhumans, so nonhuman animal welfare has an extremely large support base.” But that’s not the relevant support base for the discussion at hand.

det Aug 29, 2024, 5:46 PM
5 points
0 ∶ 0
in reply to: Nithin Ravi🔸’s comment on: Do EAs feel bad for killing a mosquito? Should they feel bad?
if I assume that mosquitoes fall somewhere between black soldier flies and silkworms in their welfare range then killing 100-1000 mosquitoes a year (assuming this causes suffering) could be the moral equivalent to killing a human.
I don’t think this is a correct reading of the welfare range estimates. If I understand correctly, these numbers would mean that a mosquito can have hedonic states 0.1% − 1% as intense as humans. So 100-1000 days of mosquito suffering might be on par with one day of human suffering. (And of course this number is a wild guess based on other insects, whose numbers are already very uncertain.)
The harm of death is a different question that RP’s numbers don’t straightforwardly address. Even a purely hedonic account has to factor in lifespan (mosquitos live for about six weeks). And killing a human is bad for a whole host of additional reasons unrelated to preventing future happiness.
So while I think the welfare range estimates suggest huge moral updates, they’re not as huge as you say. It’s good to be able to take bold conclusions seriously, but it’s also worth taking seriously that there might be a good reason for a result to be extremely counterintuitive.

det Jul 11, 2024, 1:12 AM
16 points
0 ∶ 0
on: ‘Open to Debate’ Singer/Crary EA debate—July 10
If you missed this (I did), it looks like the recording will be released on August 9.

det Jun 18, 2024, 1:25 AM
35 points
15 ∶ 0
in reply to: Habryka [Deactivated]’s comment on: My experience at the controversial Manifest 2024
I’m surprised you say you have “no idea what people mean.” The Manifest / Summer Camp / LessOnline trio made Manifest seem closer to “project the LessWrong team is deeply involved with” than “some organization is renting out our space.”
Among the things that gave me this impression were Raemon’s post “some thoughts on LessOnline” and the less.online website, both of which integrate content about Manifest without clear differentiation.
Now that I’m looking at these with a more careful eye, I can see that they all say Manifest is independently operated with its own organizers, etc. I can understand how from the inside, it would be obvious that Manifest was run by completely different people and had (I’m now presuming) little direct LessWrong involvement. I just think it should be apparent that this is less clear from the outside, and it wouldn’t be hard for someone to be confused on this point.
Granted, I didn’t go to any of these, I’ve just seen some stuff about them online, so discount this take appropriately. But my impression is that if a friend had asked me “hey, I heard about Manifest, is that a Rationalist thing?” I think “yes” would have been a less misleading answer than “no.”

det Jun 8, 2024, 11:32 PM
1 point
0 ∶ 0
in reply to: Aditya P’s comment on: Why I’m doing PauseAI
Yes, based on the last month, “the number of safety-focused researchers is dropping rapidly” certainly seems true.
I’d guess “most” is still an overstatement; I doubt the number of people has actually dropped by >50%. But the departures, plus learning that Superalignment never got their promised compute, have caused me to revise my fuzzy sense of “how much core safety work OpenAI is doing” down by a lot, probably over 50%.

det Jun 4, 2024, 2:29 AM
12 points
0 ∶ 0
on: The “TESCREAL” Bungle
Relevant: Émile Torres posted a “TESCREAL FAQ” today (unrelated to this article I assume; they’d mentioned this was in the works for a while).
I’ve only skimmed it so far, but here’s one point that directly addresses a claim from the article.
Ozy:
However, Torres is rarely careful enough to make the distinction between people’s beliefs and the premises behind the conversations they’re having. They act like everyone who believes one of these ideas believes in all the rest. In reality, it’s not uncommon for, say, an effective altruist to be convinced of the arguments that we should worry about advanced artificial intelligence without accepting transhumanism or extropianism. All too often, Torres depicts TESCREALism as a monolithic ideology — one they characterize as “profoundly dangerous.”
TESCREAL FAQ:
5. I am an Effective Altruist, but I don’t identify with the TESCREAL movement. Are you saying that all EAs are TESCREALists?

[...] I wouldn’t say—nor have I ever claimed—that everyone who identifies with one or more letters in the TESCREAL acronym should be classified as “TESCREALists.” … There are some members of the EA community who do not care about AGI or longtermism; their focus is entirely on alleviating global poverty or improving animal welfare. In my view, such individuals would not count as TESCREALists.
Having followed Torres’s work for a while, I felt like Ozy’s characterization was accurate—I’ve shared the impression that many uses of TESCREAL have blurred the boundaries between the different movements / treated them like a single entity. (I don’t have time to go looking for quotes to substantiate this, however, so it’s possible my memory isn’t accurate—others are welcome to check this if they want.) Either way, it seems like Torres is now making an effort to avoid this (mis)use of the label.

det May 1, 2024, 4:46 PM
4 points
0 ∶ 0
on: Why I’m doing PauseAI
the number of safety focussed researchers employed by OpenAI is dropping rapidly
Is this true? The links only establish that two safety-focused researchers have recently left, in very different circumstances.
It seemed like OpenAI made a big push for more safety-focused researchers with the launch of Superalignment last July; I have no idea what the trajectory looks like more recently.
Do you have other information that shows that the number of safety-focused researchers at OpenAI is decreasing?

det Apr 27, 2024, 2:29 PM
23 points
6 ∶ 0
in reply to: Sanjay’s comment on: Announcing The New York Declaration on Animal Consciousness
As an outsider to the field, here are some impressions I have:
- NY declaration is very short and uses simple language, which makes it a useful tool for communicating with the public. Compare to this sentence from the Cambridge declaration:
The neural substrates of emotions do not appear to be confined to cortical structures. In fact, subcortical neural networks aroused during affective states in humans are also critically important for generating emotional behaviors in animals.
- The Cambridge declaration is over a decade old. Releasing a similar statement is another chance for media attention and indicates that the consensus hasn’t shifted in the opposite direction.
- The NY declaration places a clearer emphasis on “cephalopod mollusks, decapod crustaceans, and insects.”
  
  A footnote to the Cambridge declaration mentions “decapod crustaceans, cephalopod mollusks, and insects” in a somewhat confusing way: it says there’s “very strong evidence” to support that these animals also “possess the neurological substrates of consciousness,” but they aren’t mentioned in the main declaration because there was not any presentation on them at the particular conference where it was signed.
  
  Insofar as the NY declaration is meant to support shrimp or insect welfare, this seems like a plus.

det Apr 25, 2024, 3:49 AM
10 points
1 ∶ 0
on: Your feedback for Actually After Hours: the unscripted, informal 80k podcast
Feedback on third episode: Also really liked it! Felt different from the first two. Less free-wheeling, more clearly useful. (Still much more on the relaxed, informal side than main-feed 80k podcasts.)
Felt very useful to get an inside perspective on what 80k thinks its doing with career advising. I really appreciated Dwarkesh kicking the tires on the theory of change (“why not focus 100% on the tails?”), as well as the responses.
It wasn’t entirely an easy listen. I identify with the common EA tropes of: trying to push myself to be more ambitious, but this doesn’t come naturally so I end up often feeling bad about how non-agentic I am. Ex ante trying some things to see if I’m in the right tail of the distribution, figuring I’m probably not, and being kind of upset and adrift about it.
I personally appreciate that 80k thinks a lot about doing right by people like me. It was somewhat hard to hear Dwarkesh focus so intently people at the tails, as if the other 99% of us are a rounding error, but I see the case for it and I’m not sure it’s completely wrong. (I’m not supposed to be the primary beneficiary of 80k advising / other EA resources. If I voluntarily sign up to try being an ambitious altruist, and later feel bad about not (yet) succeeding, I’m not sure I get to blame anyone except myself.)

det Apr 25, 2024, 3:34 AM
3 points
2 ∶ 2
on: Your feedback for Actually After Hours: the unscripted, informal 80k podcast
Feedback on first two episodes: I really enjoyed them, and was instantly sold this series. I felt like I was sitting in on a conversation with fun people having great conversations. Wasn’t really sure what the impact case was for these, but they gave me a feeling I have at the best EA meetups: oh my gosh, these are my people. ^[1]
(Feedback on third episode in another comment)
1. ^
  I have some reservations about this: the cultural characteristics that sets off the “my people” sense don’t seem too strongly connected to doing the most good? So while I love finding “my people,” it’s strange that they are such a big fraction of EA, both at local meetups and apparently at 80k.

det Apr 17, 2024, 7:57 PM
10 points
0 ∶ 0
on: Future of Humanity Institute 2005-2024: Final Report
Nick Bostrom’s website now lists him as “Principal Researcher, Macrostrategy Research Initiative.”
Doesn’t seem like they have a website yet.

det Apr 17, 2024, 7:47 PM
4 points
0 ∶ 0
in reply to: harfe’s comment on: harfe’s Quick takes
Nick Bostrom’s website now lists him as “Principal Researcher, Macrostrategy Research Initiative.”
Doesn’t seem like they have a website yet.