LessWrong dev & admin as of July 5th, 2022.
RobertM
Karma: 941
Many things about this comment seem wrong to me.
Yudkowsky’s suggestions seem entirely appropriate if you truly believe, like him, that AI x-risk is probability ~100%.
These proposals would plausibly be correct (to within an order of magnitude) in terms of the appropriate degree of response with much lower probabilities of doom (i.e. 10-20%). I think you need to actually run the math to say that this doesn’t make sense.
unproven and unlikely assumptions, like that an AI could build nanofactories by ordering proteins to be mixed over email
This is a deeply distorted understanding of Eliezer’s threat model, which is not any specific story that he can tell, but the brute fact that something smarter than you (and him, and everyone else) will come up with something better than that.
In the actual world, where the probability of extinction is signficantly less than 100%, are these proposals valuable?
I do not think it is ever particularly useful to ask “is someone else’s conclusion valid given my premises, which are importantly different from theirs”, if you are attempting to argue against someone’s premises. Obviously “A ⇒ B” & “C” does not imply “B”, and it especially does not imply “~A”.
It seems like they will just get everyone else labelled luddites and fearmongerers, especially if years and decades go by with no apocalypse in sight.
This is an empirical claim about PR, which:
does not seem obviously correct to me
has little to say about the object-level arguments
falls into pattern of suggesting that people should optimize for how others perceive us, rather than optimizing for communicating our true beliefs about the world.
Separate from object-level disagreements, my crux that people can have inside-view models which “rule out” other people’s models (as well as outside-view considerations) in a way that leads to assigning very high likelihoods (i.e. 99%+) to certain outcomes.
The fact that they haven’t successfully communicated their models to you is certainly a reason for you to not update strongly in their direction, but it doesn’t mean much for their internal epistemic stance.
I don’t think you should be optimizing to avoid extreme views, but in favor of those with the most robust models, who can also communicate them effectively to the desired audience. I agree that if we’re going to be trying anything resembling public outreach it’d be good to have multiple voices for a variety of reasons.
On the first half of the criteria I’d feel good about Paul, Buck, and Luke. On the second half I think Luke’s blog is a point of evidence in favor. I haven’t read Paul’s blog, and I don’t think that LessWrong comments are sufficiently representative for me to have a strong opinion on either Paul or Buck.
They often organize or attend events, meetups, or workshops that are focused on or friendly to polyamory, such as EA Polyamory, Poly Speed Dating, or Rationalist Polyamory. They also often join or contribute to online groups, forums, or platforms that are focused on or friendly to polyamory, such as Polyamory EA, Polyamory LessWrong, or Polyamory Rationality.
Of the six events/groups/etc listed here, the only one that seems to actually exist is “Poly Speed Dating”, which has several events/groups/etc bearing the same name in multiple places. That also happens to be the only one unrelated to the EA/rationality communities.
I suspect this was written using a modern LLM, given the structure, verbosity, and apparent hallucinations of non-existent groups.
When they asked a different Bay Area rationality organiser, they were told that their talk on diversity may have been “epistemically weak” and “not truth-seeking” enough.
So, to clarify, a guess from an unrelated party about why this talk might have resulted in a lack of an invitation pattern-matched to language used by other people in a way that has no (obvious to me) relationship to blacklists...?
I’m not sure what this was intended to demonstrate.
I am curious how you would distinguish a blacklist from the normal functioning of an organization when making hiring decisions. I guess maybe “a list of names with no details as to why you want to avoid hiring them” passed around between organizations would qualify as the first but not the second? I obviously can’t say with surety that no such thing exists elsewhere, but I would be pretty surprised to learn about any major organizations using one.
I haven’t included any names in any of my comments.
I’m not discussing naming the accuser, but the accused.
I do not think we have an obligation to avoid discussing object-level details of sexual assault claims when those claims have already been made publicly, if it seems like discussing them would otherwise be useful.
I’m pretty confused about what’s going on here. The person who made this accusation made it on Twitter under their real name using an unlocked account, and the accusation remains public to date. Is the concern here that the accused did not previously know of the accusation against them, but would be made aware of it by this discussion?
(I’m not sure whether I’d want them named in absence of a request to the contrary, but I don’t understand the implied threat model and think other explanations for the request are plausible, given the whole “public tweet” thing.)
Basically what Jason said, yes. The process described sounds reasonable but my prior, even given this post, was that it would sound reasonable.
Just noting that this reply seems to be, to me, very close to content-free, in terms of addressing object-level concerns. I think you could compress it to “I did due diligence” without losing very much.
If you’re constrained in your ability to discuss things on the object-level, i.e. due to promises to keep certain information secret, or other considerations like “discussing policy work in advance of it being done tends to backfire”, I would appreciate that being said explicitly. As it is, I can’t update very much on it.
ETA: to be clear, I’m not sure I how I feel about the broader norm of requesting costly explanations when something looks vaguely off. My first instinct is “against”, but if I were to adopt a policy of not engaging with such requests (unless they actually managed to surface something I’d consider a mistake I didn’t realize I’d made), I’d make that policy explicit.
No, sorry, I meant that at the time the feature was released (a few months ago), it didn’t have any karma requirement.
Clarifying a couple of points:
Crossposting used to be totally unrestricted; it now requires a user to have 100 karma on both LW and the EA Forum (regardless of which side they’re crossposting from) to use the feature
While historically most development was driven by the LW team, in the last year or so the EA Forum team has hired more engineers and is now larger than the LW team by headcount (and very likely by lines-of-code shipped, or whatever other metric you want to use to analyze “how much stuff did they do”).
I don’t think that the top-level comment is particularly responsive to the post, except insofar as it might have taken the title as a call to action (and then ignored the rest of the post). It’s also quite vague. But I agree that a ban seems like an unusually harsh response, absent additional context which supports the “provocation” interpretation.
Speak the truth, even if your voice trembles
Yes, I agree that there’s a non-trivial divide in attitude. I don’t think the difference in discussion is surprising, at least based on a similar pattern observed with the response to FTX. From a quick search and look at the tag, there were on the order of 10 top-level posts on the subject on LW. There are 151 posts under the FTX collapse tag on the EA forum, and possibly more untagged.
There’s definitely no censorship of the topic on LessWrong. Obviously I don’t know for sure why discussion is sparse, but my guess is that people mostly (and, in my opinion, correctly) don’t think it’s a particularly interesting or fruitful topic to discuss on LessWrong, or that the degree to which it’s an interesting subject is significantly outweighed by mindkilling effects.
Edit: with respect to the rest of the comment, I disagree that rationalists are especially interested in object-level discussion of the subjects, but probably are much more likely to disapprove of the idea that discussion of the subject should be verboten.I think the framing where Bostrom’s apology is a subject which has to be deliberately ignored is mistaken. Your prior for whether something sees active discussion on LessWrong is that it doesn’t, because most things don’t, unless there’s a specific reason you’d expect it to be of interest to the users there. I admit I haven’t seen a compelling argument for there being a teachable moment here, except the obvious “don’t do something like that in the first place”, and perhaps “have a few people read over your apology with a critical eye before posting it” (assuming that didn’t in fact happen). I’m sure you could find a way to tie those in to the practice of rationality, but it’s a bit of a stretch.
It’s possible I’ve flipped the sign on what you’re saying, but if I haven’t, I’m pretty sure most EAs are not moral realists, so I don’t know where you got the impression that it’s an underlying assumption of any serious EA efforts.
If I did flip the sign, then I don’t think it’s true that moral realism is “too unquestioned”. At this point it might be more fair to say that too much time & ink has been spilled on what’s frankly a pretty trivial question that only sees as much engagement as it does because people get caught up in arguing about definitions of words (and, of course, because some other people are deeply confused).
If your headline claim is that someone has a “fairly poor track record of correctness”, then I think “using a representative set of examples” to make your case is the bare-minimum necessary for that to be taken seriously, not an isolated demand for rigor.
My guess is that he meant the sequences convey the kind of more foundational epistemology which helps people people derive better models on subjects like AI Alignment by themselves, though all of the sequences in The Machine in the Ghost and Mere Goodness have direct object-level relevance.
Excepting Ngo’s AGI safety from first principles, I don’t especially like most of those resources as introductions exactly because they offer readers very little opportunity to test or build on their beliefs. Also, I think most of them are substantially wrong. (Concrete Problems in AI Safety seems fine, but is also skipping a lot of steps. I haven’t read Unsolved Problems in ML Safety.)
All else equal, this depends on what increase in risk of nuclear war you’re trading off against what decrease in x-risk from AI. We may have “increased” risk of nuclear war by providing aid to Ukraine in its war against Russia, but if it was indeed an increase it was probably small and worth the trade-off[1] against our other goals (such as disincentivizing the beginning of wars which might lead to nuclear escalation in the first place). I think approximately the only unusual part of Eliezer’s argument is the fact that he doesn’t beat around the bush in spelling out the implications.
Asserted for the sake of argument; I haven’t actually demonstrated that this is true but my point is more that there are many situations where we behave as if it is obviously a worthwhile trade-off to marginally increase the risk of nuclear war.