Not sure why this is on EAF rather than LW or maybe AF, but anyway. I find this interesting to look at because I have been following Eliezer’s work since approximately 2003 on SL4, and so I remember this firsthand, as it were. I disagree with several of the evaluations here (but of course agree with several of the others—I found the premise of Flare to be ludicrous at the time, and thankfully, AFAICT, pretty much zero effort went into that vaporware*):
calling LOGI and related articles ‘wrong’ because that’s not how DL looks right now is itself wrong. Yudkowsky has never said that DL or evolutionary approaches couldn’t work, or that all future AI work would look like the Bayesian program and logical approach he favored; he’s said (consistently since at least SL4 that I’ve observed) that they would be extremely dangerous when they worked, and extremely hard to make safe to the high probability that we need them to when deployed to the real world indefinitely and unboundedly and self-modifyingly, and that rigorous program-proof approaches which can make formal logical guarantees of 100% safety are what are necessary and must deal with the issues and concepts discussed in LOGI. I think this is true: they do look extremely dangerous by default, and we still do not have adequate solutions to problems like “how do we talk about human values in a way which doesn’t hardwire them dangerously into a reward function which can’t be changed?” This is something actively researched now in RL & AI safety, and which continues to lack any solution you could call even ‘decent’. (If you have ever been surprised by any result from causal influence diagrams, then you have inadvertently demonstrated the value of this.) More broadly, we still do not have any good proof or approach that we can feasibly engineer any of that with prosaic alignment approaches, which tend towards the ‘patch bugs as you find them’ or ‘make systems so complex you can’t immediately think of how they fail’ approach to security that we already knew back then was a miserable failure. Eliezer hasn’t been shown to be wrong here.
I continue to be amazed anyone can look at the past decade of DL and think that Hanson is strongly vindicated by it, rather than Yudkowsky-esque views. (Take a look at his OB posts on AI the past few years. Hanson is not exactly running victory laps, either on DL, foom, or ems. It would be too harsh to compare him to Gary Marcus… but I’ve seen at least one person do so anyway.) I would also say that to the extent that Yudkowsky-style research has enjoyed any popularity of late, it’s because people have been looking at the old debate and realizing that extremely simple generic architectures written down in a few dozen lines of code, with large capability differences between very similar lines of code, solving many problems in many fields and subsuming entire subfields as simply another minor variant, with large generalizing models (as opposed to the very strong small-models-unique-to-each-individual-problem-solved-case-by-case-by-subject-experts which Hanson & Drexler strongly advocated and which was the ML mainstream at the time) powered by OOMs more compute, steadily increasing in agency, is a short description of Yudkowsky’s views on what the runup will look like and how DL now works.
“his arguments focused on a fairly specific catastrophe scenario that most researchers now assign less weight to than they did when they first entered the field.”
Yet, the number who take it seriously since Eliezer started advocating it in the 1990s is now far greater than it was when he started and was approximately the only person anywhere. You aren’t taking seriously that these surveyed researchers (“AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI”) wouldn’t exist without Eliezer as he created the AI safety field as we know it, with everyone else downstream (like Bostrom’s influential Superintelligence—Eliezer with the serial numbers filed off and an Oxford logo added). This is missing the forest for a few trees; if you are going to argue that a bit of regression to the mean in extreme beliefs should be taken as some evidence against Eliezer, then you must also count the initial extremity of the beliefs leading to these NGOs doing AI safety & people at them doing AI safety at all as much evidence for Eliezer.† (What a perverse instance of Simpson’s paradox.)
There’s also the caveat mentioned there that the reduction may simply be because they have moved up other scenarios like the part 2 scenario where it’s not a singleton hard takeoff but a multipolar scenario (a distinction of great comfort, I’m sure), which is a scenario which over the past few years is certainly looking more probable due to how DL scaling and arms races work. (In particular, we’ve seen some fast followups—because the algorithms are so simple that once you hear the idea described at all, you know most of it.) I didn’t take the survey & don’t work at the listed NGOs, but I would point out that if I had gone pro sometime in the past decade & taken it, under your interpretation of this statistic, you would conclude “Gwern now thinks Eliezer was wrong”. Something to think about, especially if you want to consider observations like “this statistic claims most people are moving away from Eliezer’s views, even though when I look at discussions of scaling, research trends, and what startups/NGOs are being founded, it sure looks like the opposite...”
* Flare has been, like Roko’s Basilisk, one of those things where the afterlife of it has been vastly greater than the thing itself ever was, and where it gets employed in mutually contradictory ways by critics
† I find it difficult to convey what incredibly hot garbage AI researcher opinions in the ’90s were about these topics. And I don’t mean the casual projections that AGI would take until 2500 AD or whatever, I mean basics like the orthogonality thesis and instrumental drives. Like ‘transhumanism’, these are terms used in inverse proportion to how much people need them. Even on SL4, which was the fringiest of the fringe in AI alarmism, you had plenty of people reading and saying, “no, there’s no problem here at all, any AI will just automatically be friendly and safe, human moral values aren’t fragile or need to be learned, they’re just, like, a law of physics and any evolving system will embody our values”. If you ever wonder how old people in AI like Kurzweil or Schmidhuber can be so gungho about the prospect of AGI happening and replacing (ie. killing) humanity and why they have zero interest in AI safety/alignment, it’s because they think that this is a good thing and our mind-children will just automatically be like us but better and this is evolution. (“Say, doth the dull soil / Quarrel with the proud forests it hath fed,
/ And feedeth still, more comely than itself?”...) If your response to reading this is, “gwern, do you have a cite for all of that? because no real person could possibly believe such a both deeply naive and also colossally evil strawman”, well, perhaps that will convey some sense of the intellectual distance traveled.
like Bostrom’s influential Superintelligence—Eliezer with the serial numbers filed off and an Oxford logo added
It’s not accurate that the key ideas of Superintelligence came to Bostrom from Eliezer, who originated them. Rather, at least some of the main ideas came to Eliezer from Nick. For instance, in one message from Nick to Eliezer on the Extropians mailing list, dated to Dec 6th 1998, inline quotations show Eliezer arguing that it would be good to allow a superintelligent AI system to choose own its morality. Nick responds that it’s possible for an AI system to be highly intelligent without being motivated to act morally. In other words, Nick explains to Eliezer an early version of the orthogonality thesis.
Nick was not lagging behind Eliezer on evaluating the ideal timing of a singularity, either—the same thread reveals that they both had some grasp of the issue. Nick said that the fact that 150,000 people die per day must be contextualised against “the total number of sentiences that have died or may come to live”, foreshadowing his piece on Astronomical Waste, that would be published five years later. Eliezer said that having waited billions of years, the probability of a success is more important than any delay of hundreds of years.
These are indeed two of the most-important macrostrategy insights relating to AI. A reasonable guess is that a lot of the big ideas in Superintelligence were discovered by Bostrom. Some surely came from Eliezer and his sequences, or from discussions between the two, and I suppose that some came from other utilitarians and extropians.
I think chapter 4, The Kinetics of an Intelligence Explosion, has a lot of terms and arguments from EY’s posts in the FOOM Debate. (I’ve been surprised by this in the past, thinking Bostrom invented the terms, then finding things like resource overhangs getting explicitly defined in the FOOM Debate.)
calling LOGI and related articles ‘wrong’ because that’s not how DL looks right now is itself wrong. Yudkowsky has never said that DL or evolutionary approaches couldn’t work, or that all future AI work would look like the Bayesian program and logical approach he favored;
I mainly have the impression that LOGI and related articles were probably “wrong” because, so far as I’ve seen, nothing significant has been built on top of them in the intervening decade-and-half (even though LOGI’s successor was seemingly predicted to make it possible for a small group to build AGI). It doesn’t seem like there’s any sign that these articles were the start of a promising path to AGI that was simply slower than the deep learning path.
I have had the impression, though, that Yudkowsky also thought that logical/Bayesian approaches were in general more powerful/likely-to-enable-near-term-AGI (not just less safe) than DL. It’s totally possible this is a misimpression—and I’d be inclined to trust your impression over mine, since you’ve read more of his old writing than I have. (I’d also be interested if you happen to have any links handy.) But I’m not sure this significantly undermine the relevance of the LOGI case.
I continue to be amazed anyone can look at the past decade of DL and think that Hanson is strongly vindicated by it, rather than Yudkowsky-esque views.
I also think that, in various ways, Hanson also doesn’t come off great. For example, he expresses a favorable attitude toward the CYC project, which now looks like a clear dead end. He is also overly bullish about the importance of having lots of different modules. So I mostly don’t want to defend the view “Hanson had a great performance in the FOOM debate.”
I do think, though, his abstract view that compute and content (i.e. data) are centrally important are closer to mark than Yudkowsky’s expressed view. I think it does seem hard to defend Yudkowsky’s view that it’s possible for a programming team (with mid-2000s levels of compute) to acquire some “deep new insights,” go down into their basement, and then create an AI system that springboards itself into taking over the world. At least—I think it’s fair to say—the arguments weren’t strong enough to justify a lot of confidence in that view.
Yet, the number who take it seriously since Eliezer started advocating it is now far greater than it was when he started and was approximately the only person anywhere. You aren’t taking seriously that these surveyed researchers (“AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI”) wouldn’t exist without Eliezer as he created the AI safety field as we know it, with everyone else downstream (like Bostrom’s influential Superintelligence—Eliezer with the serial numbers filed off and an Oxford logo added).
This is certainly a positive aspect of his track-record—that many people have now moved closer to his views. (It also suggests that his writing was, in expectation, a major positive contribution to the project of existential risk reduction—insofar as this writing has helped move people up and we assume this was the right direction to move.) But it doesn’t imply that we should give him many more “Bayes points” to him than we give to the people who moved.
Suppose, for example, that someone says in 2020 that there was a 50% chance of full-scale nuclear war in the next five years. Then—due to Russia’s invasion of Ukraine—most people move their credences upward (although they still remained closer to 0% than 50%). Does that imply the person giving the early warning was better-calibrated than the people who moved their estimates up? I don’t think so. And I think—in this nuclear case—some analysis can be used to justify the view that the person giving the early warning was probably overconfident; they probably didn’t have enough evidence or good enough arguments to actually justify a 50% credence.
It may still be the case that the person giving the early warning (in the hypothetical nuclear case) had some valuable and neglected insights, missed by others, that are well worth paying attention to and seriously reflecting on; but that’s a different matter from believing they were overall well-calibrated or should be deferred to much more than the people who moved.
[[EDIT: Something else it might be worth emphasizing, here, is that I’m not arguing for the view “ignore Eliezer.” It’s closer to “don’t give Eliezer’s views outsized weight, compared to (e.g.) the views of the next dozen people you might be inclined to defer to, and factor in evidence that his risk estimates might have a sigificant upward bias to them.”]]
I’m going to break a sentence from your comment here into bits for inspection. Also, emphasis and elisions mine.
I would also say that to the extent that Yudkowsky-style research has enjoyed any popularity of late, it’s because people have been looking at the old debate and realizing that
extremely simple generic architectures written down in a few dozen lines of code
with large capability differences between very similar lines of code
solving many problems in many fields and subsuming entire subfields as simply another minor variant
with large generalizing models...
powered by OOMs more compute
steadily increasing in agency
is
a short description of Yudkowsky’s views on what the runup will look like
and how DL now works.
We don’t have a formalism to describe what “agency” is. We do have several posts trying to define it on the Alignment Forum:
While it might not be the best choice, I’m going to use Gradations of Agency as a definition, because it’s more systematic in its presentation.
“Level 3” is described as “Armed with this ability you can learn not just from your own experience, but from the experience of others—you can identify successful others and imitate them.”
This doesn’t seem like what any ML model does. So we can look at “Level 2,” which gives the example ” You start off reacting randomly to inputs, but you learn to run from red things and towards green things because when you ran towards red things you got negative reward and when you ran towards green things you got positive reward.”
This seems like how all ML works.
So using the “Gradations of Agency” framework, we might view individual ML systems as improving in power and generality within a single level of agency. But they don’t appear to be changing levels of agency. They aren’t identifying other successful ML models and imitating them.
Gradations of Agency doesn’t argue whether or not there is an asymptote of power and generality within each level. Is there a limit to the power and generality possible within level 2, where all ML seems to reside?
This seems to be the crux of the issue. If DL is approaching an asymptote of power and generality below that of AGI as model and data sizes increase, then this cuts directly against Yudkowsky’s predictions. On the other hand, if we think that DL can scale to AGI through model and data size increases alone, then that would be right in line with his predictions.
A 10 trillion parameter model now exists, and it’s been suggested that a 100 trillion parameter model, which might even be created this year, might be roughly comparable to the power of the human brain.
It’s scary to see that we’re racing full-on toward a very near-term ML project that might plausibly be AGI. However, if a 100-trillion parameter ML model is not AGI, then we’d have two strikes against Yudkowski. If neither a small coded model nor a 100-trillion parameter trained model using 2022-era ML results in AGI, then I think we have to take a hard look at his track record on predicting what technology is likely to result in AGI. We also have his “AGI well before 2050” statement from “Beware boasting” to work with, although that’s not much help.
On the other hand, I think his assertiveness about the importance of AI safety and risk is appropriate even if he proves wrong about the technology by which AGI will be created.
I would critique the OP, however, for not being sufficiently precise in its critiques of Yudkowsky. As its “fairly clearcut examples,” it uses 20+-year-old predictions that Yudkowsky has explicitly disavowed. Then, at the end, it complains that he hasn’t “acknowledged his mixed track record.” Yet in the post it links, Yudkowsky’s quoted as saying:
To be a slightly better Bayesian is to spend your entire life watching others slowly update in excruciatingly predictable directions that you jumped ahead of 6 years earlier so that your remaining life could be a random epistemic walk like a sane person with self-respect.
6 years is not 20 years. It’s perfectly consistent to say that a youthful, 20+-years-in-the-past version of you thought wrongly about a topic, but that you’ve since come to be so much better at making predictions within your field that you’re 6 years ahead of Metaculus. We might wish he’d stated these predictions in public and specified what they were. But his failure to do so doesn’t make him wrong, but rather lacking evidence of his superior forecasting ability. These are distinct failure modes.
Overall, I think it’s wrong to conflate “Yudkowsky was wrong 20+ years ago in his youth” with “not everyone in AI safety agrees with Yudkowsky” with “Yudkowsky hasn’t made many recent, falsifiable near-term public predictions about AI timelines.” I think this is a fair critique of the OP, which claims to be interrogating Yudkowsky’s “track record.”
But I do agree that it’s wise for a non-expert to defer to a portfolio of well-chosen experts, rather than the views of the originator of the field alone. While I don’t love the argument the OP used to get there, I do agree with the conclusion, which strikes me as just plain common sense.
Re gradations of agency: Level 3 and level 4 seem within reach IMO. IIRC there are already some examples of neural nets being trained to watch other actors in some simulated environment and then imitate them. Also, model-based planning (i.e. level 4) is very much a thing, albeit something that human programmers seem to have to hard-code. I predict that within 5 years there will be systems which are unambiguously in level 3 and level 4, even if they aren’t perfect at it (hey, we humans aren’t perfect at it either).
Level 3″ is described as “Armed with this ability you can learn not just from your own experience, but from the experience of others—you can identify successful others and imitate them.” This doesn’t seem like what any ML model does.
This sounds like straightforward transfer learning (TL) or fine tuning, common in 2017.
So you could just write 15 lines of python which shops between some set of pretrained weights and sees how they perform. Often TL is many times (1000x) faster than random weights and only needs a few examples.
As speculation: it seems like in one of the agent simulations you can just have agents grab other agents weights or layers and try them out in a strategic way (when they detect an impasse or new environment or something). There is an analogy to biology where species alternate between asexual vs sexual reproduction, and trading of genetic material occurs during periods of adversity. (This is trivial, I’m sure a second year student has written a lot more.)
This doesn’t seem to fit any sort of agent framework or improve agency though. It just makes you train faster.
Eh, there seems like a connection to interpretability.
For example, if the ML architecture “were modular+categorized or legible to the agents”, they would more quickly and effectively swap weights or models.
So there might be some way where legibility can emerge by selection pressure in an environment where say, agents had limited capacity to store weights or data, and had to constantly and extensively share weights with each other. You could imagine teams of agents surviving and proliferating by a shared architecture that let them pass this data fluently in the form of weights.
To make sure the transmission mechanism itself isn’t crazy baroque you can, like, use some sort of regularization or something.
I’m 90% sure this is a shower thought but like it can’t be worse than “The Great Reflection”.
The above seems voluminous and I believe this is the written output with the goal of defending a person.
I will reluctantly engage directly, instead of just launching into another class of arguments or something or go for a walk (I’m being blocked by moral maze sort of reasons and unseasonable weather).
You aren’t taking seriously that these surveyed researchers (“AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI”) wouldn’t exist without Eliezer as he created the AI safety field as we know it
Yeah, no, it’s the exact opposite.
So one dude, who only has a degree in social studies, but seems to write well, wrote this:
I’m copying a screenshot to show the highlighting isn’t mine:
This isn’t what is written or is said, but using other experience unrelated to EA or anyone in it, I’m really sure even a median thought leader would have better convinced the person written this.
So they lost 4 years of support (until Superintelligence was written)
The above seems voluminous and I believe this is the written output with the goal of defending a person.
Yes, much like the OP is voluminous and is the written output with the goal of criticizing a person. You’re familiar with such writings, as you’ve written enough criticizing me. Your point?
Yeah, no, it’s the exact opposite.
No, it’s just as I said, and your Karnofsky retrospective strongly supports what I said. (I strongly encourage people to go and read it, not just to see what’s before and after the part He screenshots, but because it is a good retrospective which is both informative about the history here and an interesting case study of how people change their minds and what Karnofsky has learned.)
Karnofsky started off disagreeing that there is any problem at all in 2007 when he was introduced to MIRI via EA, and merely thought there were some interesting points. Interesting, but certainly not worth sending any money to MIRI or looking for better alternative ways to invest in AI safety. These ideas kept developing, and Karnofsky kept having to engage, steadily moving from ‘there is no problem’ to intermediate points like ‘but we can make tool AIs and not agent AIs’ (a period in his evolution I remember well because I wrote criticisms of it), which he eventually abandons. You forgot to screenshot the part where Karnofsky writes that he assumed ‘the experts’ had lots of great arguments against AI risk and the Yudkowsky paradigm and that was why they just bother talking about it, and then moved to SF and discovered ‘oh no’, that not only did those not exist, the experts hadn’t even begun to think about it. Karnofsky also agrees with many of the points I make about Bostrom’s book & intellectual pedigree (“When I’d skimmed Superintelligence (prior to its release), I’d felt that its message was very similar to—though more clearly and carefully stated than—the arguments MIRI had been making without much success.” just below where you cut off). And so here we are today, where Karnofsky has not just overseen donations of millions of dollars to MIRI and AI safety NGOs or the recruitment of MIRI staffers like ex-MIRI CEO Muehlhauser, but it remains a major area for OpenPhil (and philanthropies imitating it like FTX). It all leads back to Eliezer. As Karnofsky concludes:
One of the biggest changes is the one discussed above, regarding potential risks from advanced AI. I went from seeing this as a strange obsession of the community to a case of genuine early insight and impact. I felt the community had identified a potentially enormously important cause and played a major role in this cause’s coming to be taken more seriously. This development became—in my view—a genuine and major candidate for a “hit”, and an example of an idea initially seeming “wacky” and later coming to seem prescient.
Of course, it is far from a settled case: many questions remain about whether this cause is indeed important and whether today’s preparations will look worthwhile in retrospect. But my estimate of the cause’s likely importance—and, I believe, conventional wisdom among AI researchers in academia and industry—has changed noticeably.
That is, Karnofsky explicitly attributes the widespread changes I am describing to the causal impact of the AI risk community around MIRI & Yudkowsky. He doesn’t say it happened regardless or despite them, or that it was already fairly common and unoriginal, or that it was reinvented elsewhere, or that Yudkowsky delayed it on net.
I’m really sure even a median thought leader would have better convinced the person written this.
No, it’s just as I said, and your Karnofsky retrospective strongly supports what I said.
I also agree that Karnfosky’s retrospective supports Gwern’s analysis, rather than doing the opposite.
(I just disagree about how strongly it counts in favor of deference to Yudkowsky. For example, I don’t think this case implies we should currently defer more to Yudkwosky’s risk estimates than we do to Karnofsky’s.)
Ugh. Y’all just made me get into “EA rhetoric” mode:
I also agree that Karnfosky’s retrospective supports Gwern’s analysis, rather than doing the opposite.
What?
No. Not only is this not true but this is indulging in a trivial rhetorical maneuver.
My comment said that the counterfactual would be better without the involvement of the person mentioned in the OP. I used the retrospective as evidence.
The retrospective includes at least two points for why the author changed their mind:
The book Superintelligence, which they explicitly said was the biggest event
The author moved to SF and learned about DL, and was informed by speaking to non-rationalist AI researchers, and then decided that LessWrong and MIRI were right.
In response to this, Gwern states the point #2, and asserts that this is causal evidence in favor of the person mentioned in the OP being useful.
Why? How?
Notice that #2 above doesn’t at all rule out that the founders or culture was repellent. In fact it seems like a lavish, and unlikely level amount of involvement.
I interpreted Gwern as mostly highlighting that people have updated toward’s Yudkowsky’s views—and using this as evidence in favor of the view we should defer a decent amount to Yudkowsky. I think that was a reasonable move.
There is also a causal question here (‘Has Yudkowsky on-net increased levels of concern about AI risk relative to where they would otherwise be?’), but I didn’t take the causal question to be central to the point Gwern was making. Although now I’m less sure.
I don’t personally have strong views on the causal question—I haven’t thought through the counterfactual.
(I strongly encourage people to go and read it, not just to see what’s before and after the part He screenshots, but because it is a good retrospective which is both informative about the history here and an interesting case study of how people change their minds and what Karnofsky has learned.)
By the way, I didn’t screenshot the pieces that fit my narrative—Gwern’s assertion of bad faith is another device being used.
Yes, much like the OP is voluminous and is the written output with the goal of criticizing a person. You’re familiar with such writings, as you’ve written enough criticizing me. Your point?
Gwern also digs up a previous argument. Not only is that issue entirely unrelated, its sort of exactly the opposite evidence he wants to show: Gwern appeared to borderline or threaten to dox someone who spoke out against him.
I commented. However I do not know anyone involved, such as who Gwern was, but only acting on the content and behaviour I saw, which was outright abusive.
There is no expected benefit to doing this. It’s literally the most principled thing to act in this way and I would do it again.
The consequences of that incident, the fact that this person with this behavior and content had this much status, was a large update for me.
More subtly and perniciously, Gwern’s adverse behavior in this comment chain and the incident mentioned above, is calibrated to the level of “EA rhetoric”. Digs like his above can sail through, with the tailwind of support of a subset of this community, a subset that values authority over content and Truth, to a degree much more than it understands.
On the other hand, in contrast, an outsider, who already has to dance through all the rhetorical devices and elliptical references, has to make a high effort, unemotional comment to try to make a point. Even or especially if they manage to do this, they can expect to be hit with a wall of text with various hostilities.
Like, this is awful. This isn’t just bad but it’s borderline abusive.
It’s wild that that this is the level of discourse here.
Because of the amount of reputation, money and ingroupness, this is probably one of the most extreme forms of tribalism that exists.
Charles, consider going for that walk now if you’re able to. (Maybe I’m missing it, but the rhetorical moves in this thread seem equally bad, and not very bad at that.)
Like, how can so many standard, stale patterns of internet forum authority, devices and rhetoric be rewarded and replicate in a community explicitly addressing topics like tribalism and “evaporative cooling”?
Rude/hostile language and condescension, especially from Charles He
Gwern brings in an external dispute — a thread in which Charles accuses them of doxing an anonymous critic on LessWrong. We think that bringing in external disputes interferes with good discourse; it moves the thread away from discussion of the topic in question, and more towards discussions of individual users’ characters
The conversation about the external dispute gets increasingly unproductive
The mentioned thread about doxing also breaks Forum norms in multiple ways. We’ve listed them on that thread.
The moderators are still considering a further response. We’ll also be discussing with both Gwern and Charles privately.
I honestly don’t see such a problem with Gwern calling out out Charles’ flimsy argument and hypocrisy using an example, be it a part of an external dispute.
On the other hand, I think Charles’ uniformly low comment quality should have had him (temporarily) banned long ago (sorry Charles). The material is generally poorly organised, poorly researched, often intentionally provocative, sometimes interspersed with irrelevant images, and high in volume. One gets the impression of an author who holds their reader in contempt.
I don’t necessarily disagree with the assessment of a temporary ban for “unnecessary rudeness or offensiveness”, or “other behaviour that interferes with good discourse”, but I disagree that Charles’ comment quality is “uniformly” low or that a ban might be merited primarily because of high comment volume and too low quality.There are some real insights and contributions sprinkled in in my opinion.
For me the unnecessary rudeness or offensiveness and other behavior interfering with discourse comes from things like comments that are technically replies to a particular person but seem like they’re mostly intended to win the argument in front of unknown readers, and containing things like rudeness, paranoia, and condescension towards the person they’re replying to. I think the doxing accusation, which if I remember correctly actually doxxed the victim much more than Gwern’s comment, is part of a similar pattern of engaging poorly with a particular person, partly through an incorrect assessment that the benefits to bystanders will outweigh the costs. I think this sort of behavior stifles conversation and good will.
I’m not sure a ban is a great solution though. There might be other, less blunt ways of tackling this situation.
What I would really like to see is a (much) higher lower limit of comment quality from Charles i.e. moving the bar for tolerating rudeness and bad behavior in a comment much higher even though it could be potentially justified in terms of benefits to bystanders or readers.
I don’t disagree with your judgement of banning but I point out there’s no banning for quality—you must be very frustrated with the content.
To get a sense of this, for the specific issue in the dispute, where I suggested the person or institution in question caused a a 4 year delay in funding, are you saying it’s an objectively bad read, even limited to just the actual document cited? I don’t see how that is.
Or is this wrong, but requires additional context or knowledge.
Re the banning idea, I think you could fall afoul of “unnecessary rudeness or offensiveness”, or “other behaviour that interferes with good discourse” (too much volume, too low quality). But I’m not the moderator here.
My point is that when you say that Gwern produces verbose content about a person, it seems fine—indeed quite appropriate—for him to point out that you do too. So it seems a bit rich for that to be a point of concern for moderators.
I’m not taking any stance on the doxxing dispute itself, funding delays, and so on.
Not sure why this is on EAF rather than LW or maybe AF, but anyway. I find this interesting to look at because I have been following Eliezer’s work since approximately 2003 on SL4, and so I remember this firsthand, as it were. I disagree with several of the evaluations here (but of course agree with several of the others—I found the premise of Flare to be ludicrous at the time, and thankfully, AFAICT, pretty much zero effort went into that vaporware*):
calling LOGI and related articles ‘wrong’ because that’s not how DL looks right now is itself wrong. Yudkowsky has never said that DL or evolutionary approaches couldn’t work, or that all future AI work would look like the Bayesian program and logical approach he favored; he’s said (consistently since at least SL4 that I’ve observed) that they would be extremely dangerous when they worked, and extremely hard to make safe to the high probability that we need them to when deployed to the real world indefinitely and unboundedly and self-modifyingly, and that rigorous program-proof approaches which can make formal logical guarantees of 100% safety are what are necessary and must deal with the issues and concepts discussed in LOGI. I think this is true: they do look extremely dangerous by default, and we still do not have adequate solutions to problems like “how do we talk about human values in a way which doesn’t hardwire them dangerously into a reward function which can’t be changed?” This is something actively researched now in RL & AI safety, and which continues to lack any solution you could call even ‘decent’. (If you have ever been surprised by any result from causal influence diagrams, then you have inadvertently demonstrated the value of this.) More broadly, we still do not have any good proof or approach that we can feasibly engineer any of that with prosaic alignment approaches, which tend towards the ‘patch bugs as you find them’ or ‘make systems so complex you can’t immediately think of how they fail’ approach to security that we already knew back then was a miserable failure. Eliezer hasn’t been shown to be wrong here.
I continue to be amazed anyone can look at the past decade of DL and think that Hanson is strongly vindicated by it, rather than Yudkowsky-esque views. (Take a look at his OB posts on AI the past few years. Hanson is not exactly running victory laps, either on DL, foom, or ems. It would be too harsh to compare him to Gary Marcus… but I’ve seen at least one person do so anyway.) I would also say that to the extent that Yudkowsky-style research has enjoyed any popularity of late, it’s because people have been looking at the old debate and realizing that extremely simple generic architectures written down in a few dozen lines of code, with large capability differences between very similar lines of code, solving many problems in many fields and subsuming entire subfields as simply another minor variant, with large generalizing models (as opposed to the very strong small-models-unique-to-each-individual-problem-solved-case-by-case-by-subject-experts which Hanson & Drexler strongly advocated and which was the ML mainstream at the time) powered by OOMs more compute, steadily increasing in agency, is a short description of Yudkowsky’s views on what the runup will look like and how DL now works.
“his arguments focused on a fairly specific catastrophe scenario that most researchers now assign less weight to than they did when they first entered the field.”
Yet, the number who take it seriously since Eliezer started advocating it in the 1990s is now far greater than it was when he started and was approximately the only person anywhere. You aren’t taking seriously that these surveyed researchers (“AI Impacts, CHAI, CLR, CSER, CSET, FHI, FLI, GCRI, MILA, MIRI, Open Philanthropy and PAI”) wouldn’t exist without Eliezer as he created the AI safety field as we know it, with everyone else downstream (like Bostrom’s influential Superintelligence—Eliezer with the serial numbers filed off and an Oxford logo added). This is missing the forest for a few trees; if you are going to argue that a bit of regression to the mean in extreme beliefs should be taken as some evidence against Eliezer, then you must also count the initial extremity of the beliefs leading to these NGOs doing AI safety & people at them doing AI safety at all as much evidence for Eliezer.† (What a perverse instance of Simpson’s paradox.)
There’s also the caveat mentioned there that the reduction may simply be because they have moved up other scenarios like the part 2 scenario where it’s not a singleton hard takeoff but a multipolar scenario (a distinction of great comfort, I’m sure), which is a scenario which over the past few years is certainly looking more probable due to how DL scaling and arms races work. (In particular, we’ve seen some fast followups—because the algorithms are so simple that once you hear the idea described at all, you know most of it.) I didn’t take the survey & don’t work at the listed NGOs, but I would point out that if I had gone pro sometime in the past decade & taken it, under your interpretation of this statistic, you would conclude “Gwern now thinks Eliezer was wrong”. Something to think about, especially if you want to consider observations like “this statistic claims most people are moving away from Eliezer’s views, even though when I look at discussions of scaling, research trends, and what startups/NGOs are being founded, it sure looks like the opposite...”
* Flare has been, like Roko’s Basilisk, one of those things where the afterlife of it has been vastly greater than the thing itself ever was, and where it gets employed in mutually contradictory ways by critics
† I find it difficult to convey what incredibly hot garbage AI researcher opinions in the ’90s were about these topics. And I don’t mean the casual projections that AGI would take until 2500 AD or whatever, I mean basics like the orthogonality thesis and instrumental drives. Like ‘transhumanism’, these are terms used in inverse proportion to how much people need them. Even on SL4, which was the fringiest of the fringe in AI alarmism, you had plenty of people reading and saying, “no, there’s no problem here at all, any AI will just automatically be friendly and safe, human moral values aren’t fragile or need to be learned, they’re just, like, a law of physics and any evolving system will embody our values”. If you ever wonder how old people in AI like Kurzweil or Schmidhuber can be so gungho about the prospect of AGI happening and replacing (ie. killing) humanity and why they have zero interest in AI safety/alignment, it’s because they think that this is a good thing and our mind-children will just automatically be like us but better and this is evolution. (“Say, doth the dull soil / Quarrel with the proud forests it hath fed, / And feedeth still, more comely than itself?”...) If your response to reading this is, “gwern, do you have a cite for all of that? because no real person could possibly believe such a both deeply naive and also colossally evil strawman”, well, perhaps that will convey some sense of the intellectual distance traveled.
It’s not accurate that the key ideas of Superintelligence came to Bostrom from Eliezer, who originated them. Rather, at least some of the main ideas came to Eliezer from Nick. For instance, in one message from Nick to Eliezer on the Extropians mailing list, dated to Dec 6th 1998, inline quotations show Eliezer arguing that it would be good to allow a superintelligent AI system to choose own its morality. Nick responds that it’s possible for an AI system to be highly intelligent without being motivated to act morally. In other words, Nick explains to Eliezer an early version of the orthogonality thesis.
Nick was not lagging behind Eliezer on evaluating the ideal timing of a singularity, either—the same thread reveals that they both had some grasp of the issue. Nick said that the fact that 150,000 people die per day must be contextualised against “the total number of sentiences that have died or may come to live”, foreshadowing his piece on Astronomical Waste, that would be published five years later. Eliezer said that having waited billions of years, the probability of a success is more important than any delay of hundreds of years.
These are indeed two of the most-important macrostrategy insights relating to AI. A reasonable guess is that a lot of the big ideas in Superintelligence were discovered by Bostrom. Some surely came from Eliezer and his sequences, or from discussions between the two, and I suppose that some came from other utilitarians and extropians.
I think chapter 4, The Kinetics of an Intelligence Explosion, has a lot of terms and arguments from EY’s posts in the FOOM Debate. (I’ve been surprised by this in the past, thinking Bostrom invented the terms, then finding things like resource overhangs getting explicitly defined in the FOOM Debate.)
Thanks for the comment! A lot of this is useful.
I mainly have the impression that LOGI and related articles were probably “wrong” because, so far as I’ve seen, nothing significant has been built on top of them in the intervening decade-and-half (even though LOGI’s successor was seemingly predicted to make it possible for a small group to build AGI). It doesn’t seem like there’s any sign that these articles were the start of a promising path to AGI that was simply slower than the deep learning path.
I have had the impression, though, that Yudkowsky also thought that logical/Bayesian approaches were in general more powerful/likely-to-enable-near-term-AGI (not just less safe) than DL. It’s totally possible this is a misimpression—and I’d be inclined to trust your impression over mine, since you’ve read more of his old writing than I have. (I’d also be interested if you happen to have any links handy.) But I’m not sure this significantly undermine the relevance of the LOGI case.
I also think that, in various ways, Hanson also doesn’t come off great. For example, he expresses a favorable attitude toward the CYC project, which now looks like a clear dead end. He is also overly bullish about the importance of having lots of different modules. So I mostly don’t want to defend the view “Hanson had a great performance in the FOOM debate.”
I do think, though, his abstract view that compute and content (i.e. data) are centrally important are closer to mark than Yudkowsky’s expressed view. I think it does seem hard to defend Yudkowsky’s view that it’s possible for a programming team (with mid-2000s levels of compute) to acquire some “deep new insights,” go down into their basement, and then create an AI system that springboards itself into taking over the world. At least—I think it’s fair to say—the arguments weren’t strong enough to justify a lot of confidence in that view.
This is certainly a positive aspect of his track-record—that many people have now moved closer to his views. (It also suggests that his writing was, in expectation, a major positive contribution to the project of existential risk reduction—insofar as this writing has helped move people up and we assume this was the right direction to move.) But it doesn’t imply that we should give him many more “Bayes points” to him than we give to the people who moved.
Suppose, for example, that someone says in 2020 that there was a 50% chance of full-scale nuclear war in the next five years. Then—due to Russia’s invasion of Ukraine—most people move their credences upward (although they still remained closer to 0% than 50%). Does that imply the person giving the early warning was better-calibrated than the people who moved their estimates up? I don’t think so. And I think—in this nuclear case—some analysis can be used to justify the view that the person giving the early warning was probably overconfident; they probably didn’t have enough evidence or good enough arguments to actually justify a 50% credence.
It may still be the case that the person giving the early warning (in the hypothetical nuclear case) had some valuable and neglected insights, missed by others, that are well worth paying attention to and seriously reflecting on; but that’s a different matter from believing they were overall well-calibrated or should be deferred to much more than the people who moved.
[[EDIT: Something else it might be worth emphasizing, here, is that I’m not arguing for the view “ignore Eliezer.” It’s closer to “don’t give Eliezer’s views outsized weight, compared to (e.g.) the views of the next dozen people you might be inclined to defer to, and factor in evidence that his risk estimates might have a sigificant upward bias to them.”]]
I’m going to break a sentence from your comment here into bits for inspection. Also, emphasis and elisions mine.
We don’t have a formalism to describe what “agency” is. We do have several posts trying to define it on the Alignment Forum:
Gradations of Agency
Optimality is the tiger, and agents are its teeth
Agency and Coherence
While it might not be the best choice, I’m going to use Gradations of Agency as a definition, because it’s more systematic in its presentation.
“Level 3” is described as “Armed with this ability you can learn not just from your own experience, but from the experience of others—you can identify successful others and imitate them.”
This doesn’t seem like what any ML model does. So we can look at “Level 2,” which gives the example ” You start off reacting randomly to inputs, but you learn to run from red things and towards green things because when you ran towards red things you got negative reward and when you ran towards green things you got positive reward.”
This seems like how all ML works.
So using the “Gradations of Agency” framework, we might view individual ML systems as improving in power and generality within a single level of agency. But they don’t appear to be changing levels of agency. They aren’t identifying other successful ML models and imitating them.
Gradations of Agency doesn’t argue whether or not there is an asymptote of power and generality within each level. Is there a limit to the power and generality possible within level 2, where all ML seems to reside?
This seems to be the crux of the issue. If DL is approaching an asymptote of power and generality below that of AGI as model and data sizes increase, then this cuts directly against Yudkowsky’s predictions. On the other hand, if we think that DL can scale to AGI through model and data size increases alone, then that would be right in line with his predictions.
A 10 trillion parameter model now exists, and it’s been suggested that a 100 trillion parameter model, which might even be created this year, might be roughly comparable to the power of the human brain.
It’s scary to see that we’re racing full-on toward a very near-term ML project that might plausibly be AGI. However, if a 100-trillion parameter ML model is not AGI, then we’d have two strikes against Yudkowski. If neither a small coded model nor a 100-trillion parameter trained model using 2022-era ML results in AGI, then I think we have to take a hard look at his track record on predicting what technology is likely to result in AGI. We also have his “AGI well before 2050” statement from “Beware boasting” to work with, although that’s not much help.
On the other hand, I think his assertiveness about the importance of AI safety and risk is appropriate even if he proves wrong about the technology by which AGI will be created.
I would critique the OP, however, for not being sufficiently precise in its critiques of Yudkowsky. As its “fairly clearcut examples,” it uses 20+-year-old predictions that Yudkowsky has explicitly disavowed. Then, at the end, it complains that he hasn’t “acknowledged his mixed track record.” Yet in the post it links, Yudkowsky’s quoted as saying:
6 years is not 20 years. It’s perfectly consistent to say that a youthful, 20+-years-in-the-past version of you thought wrongly about a topic, but that you’ve since come to be so much better at making predictions within your field that you’re 6 years ahead of Metaculus. We might wish he’d stated these predictions in public and specified what they were. But his failure to do so doesn’t make him wrong, but rather lacking evidence of his superior forecasting ability. These are distinct failure modes.
Overall, I think it’s wrong to conflate “Yudkowsky was wrong 20+ years ago in his youth” with “not everyone in AI safety agrees with Yudkowsky” with “Yudkowsky hasn’t made many recent, falsifiable near-term public predictions about AI timelines.” I think this is a fair critique of the OP, which claims to be interrogating Yudkowsky’s “track record.”
But I do agree that it’s wise for a non-expert to defer to a portfolio of well-chosen experts, rather than the views of the originator of the field alone. While I don’t love the argument the OP used to get there, I do agree with the conclusion, which strikes me as just plain common sense.
Re gradations of agency: Level 3 and level 4 seem within reach IMO. IIRC there are already some examples of neural nets being trained to watch other actors in some simulated environment and then imitate them. Also, model-based planning (i.e. level 4) is very much a thing, albeit something that human programmers seem to have to hard-code. I predict that within 5 years there will be systems which are unambiguously in level 3 and level 4, even if they aren’t perfect at it (hey, we humans aren’t perfect at it either).
This sounds like straightforward transfer learning (TL) or fine tuning, common in 2017.
So you could just write 15 lines of python which shops between some set of pretrained weights and sees how they perform. Often TL is many times (1000x) faster than random weights and only needs a few examples.
As speculation: it seems like in one of the agent simulations you can just have agents grab other agents weights or layers and try them out in a strategic way (when they detect an impasse or new environment or something). There is an analogy to biology where species alternate between asexual vs sexual reproduction, and trading of genetic material occurs during periods of adversity. (This is trivial, I’m sure a second year student has written a lot more.)
This doesn’t seem to fit any sort of agent framework or improve agency though. It just makes you train faster.
Eh, there seems like a connection to interpretability.
For example, if the ML architecture “were modular+categorized or legible to the agents”, they would more quickly and effectively swap weights or models.
So there might be some way where legibility can emerge by selection pressure in an environment where say, agents had limited capacity to store weights or data, and had to constantly and extensively share weights with each other. You could imagine teams of agents surviving and proliferating by a shared architecture that let them pass this data fluently in the form of weights.
To make sure the transmission mechanism itself isn’t crazy baroque you can, like, use some sort of regularization or something.
I’m 90% sure this is a shower thought but like it can’t be worse than “The Great Reflection”.
n00b q: What’s AF?
Alignment Forum (for technical discussions about AI alignment)
It’s short for the Alignment Forum: https://www.alignmentforum.org/
Eh.
The above seems voluminous and I believe this is the written output with the goal of defending a person.
I will reluctantly engage directly, instead of just launching into another class of arguments or something or go for a walk (I’m being blocked by moral maze sort of reasons and unseasonable weather).
Yeah, no, it’s the exact opposite.
So one dude, who only has a degree in social studies, but seems to write well, wrote this:
https://docs.google.com/document/d/1hKZNRSLm7zubKZmfA7vsXvkIofprQLGUoW43CYXPRrk/edit#
I’m copying a screenshot to show the highlighting isn’t mine:
This isn’t what is written or is said, but using other experience unrelated to EA or anyone in it, I’m really sure even a median thought leader would have better convinced the person written this.
So they lost 4 years of support (until Superintelligence was written)
Yes, much like the OP is voluminous and is the written output with the goal of criticizing a person. You’re familiar with such writings, as you’ve written enough criticizing me. Your point?
No, it’s just as I said, and your Karnofsky retrospective strongly supports what I said. (I strongly encourage people to go and read it, not just to see what’s before and after the part He screenshots, but because it is a good retrospective which is both informative about the history here and an interesting case study of how people change their minds and what Karnofsky has learned.)
Karnofsky started off disagreeing that there is any problem at all in 2007 when he was introduced to MIRI via EA, and merely thought there were some interesting points. Interesting, but certainly not worth sending any money to MIRI or looking for better alternative ways to invest in AI safety. These ideas kept developing, and Karnofsky kept having to engage, steadily moving from ‘there is no problem’ to intermediate points like ‘but we can make tool AIs and not agent AIs’ (a period in his evolution I remember well because I wrote criticisms of it), which he eventually abandons. You forgot to screenshot the part where Karnofsky writes that he assumed ‘the experts’ had lots of great arguments against AI risk and the Yudkowsky paradigm and that was why they just bother talking about it, and then moved to SF and discovered ‘oh no’, that not only did those not exist, the experts hadn’t even begun to think about it. Karnofsky also agrees with many of the points I make about Bostrom’s book & intellectual pedigree (“When I’d skimmed Superintelligence (prior to its release), I’d felt that its message was very similar to—though more clearly and carefully stated than—the arguments MIRI had been making without much success.” just below where you cut off). And so here we are today, where Karnofsky has not just overseen donations of millions of dollars to MIRI and AI safety NGOs or the recruitment of MIRI staffers like ex-MIRI CEO Muehlhauser, but it remains a major area for OpenPhil (and philanthropies imitating it like FTX). It all leads back to Eliezer. As Karnofsky concludes:
That is, Karnofsky explicitly attributes the widespread changes I am describing to the causal impact of the AI risk community around MIRI & Yudkowsky. He doesn’t say it happened regardless or despite them, or that it was already fairly common and unoriginal, or that it was reinvented elsewhere, or that Yudkowsky delayed it on net.
Hard to be convincing when you don’t exist.
I also agree that Karnfosky’s retrospective supports Gwern’s analysis, rather than doing the opposite.
(I just disagree about how strongly it counts in favor of deference to Yudkowsky. For example, I don’t think this case implies we should currently defer more to Yudkwosky’s risk estimates than we do to Karnofsky’s.)
Ugh. Y’all just made me get into “EA rhetoric” mode:
What?
No. Not only is this not true but this is indulging in a trivial rhetorical maneuver.
My comment said that the counterfactual would be better without the involvement of the person mentioned in the OP. I used the retrospective as evidence.
The retrospective includes at least two points for why the author changed their mind:
The book Superintelligence, which they explicitly said was the biggest event
The author moved to SF and learned about DL, and was informed by speaking to non-rationalist AI researchers, and then decided that LessWrong and MIRI were right.
In response to this, Gwern states the point #2, and asserts that this is causal evidence in favor of the person mentioned in the OP being useful.
Why? How?
Notice that #2 above doesn’t at all rule out that the founders or culture was repellent. In fact it seems like a lavish, and unlikely level amount of involvement.
I interpreted Gwern as mostly highlighting that people have updated toward’s Yudkowsky’s views—and using this as evidence in favor of the view we should defer a decent amount to Yudkowsky. I think that was a reasonable move.
There is also a causal question here (‘Has Yudkowsky on-net increased levels of concern about AI risk relative to where they would otherwise be?’), but I didn’t take the causal question to be central to the point Gwern was making. Although now I’m less sure.
I don’t personally have strong views on the causal question—I haven’t thought through the counterfactual.
By the way, I didn’t screenshot the pieces that fit my narrative—Gwern’s assertion of bad faith is another device being used.
Gwern also digs up a previous argument. Not only is that issue entirely unrelated, its sort of exactly the opposite evidence he wants to show: Gwern appeared to borderline or threaten to dox someone who spoke out against him.
I commented. However I do not know anyone involved, such as who Gwern was, but only acting on the content and behaviour I saw, which was outright abusive.
There is no expected benefit to doing this. It’s literally the most principled thing to act in this way and I would do it again.
The consequences of that incident, the fact that this person with this behavior and content had this much status, was a large update for me.
More subtly and perniciously, Gwern’s adverse behavior in this comment chain and the incident mentioned above, is calibrated to the level of “EA rhetoric”. Digs like his above can sail through, with the tailwind of support of a subset of this community, a subset that values authority over content and Truth, to a degree much more than it understands.
On the other hand, in contrast, an outsider, who already has to dance through all the rhetorical devices and elliptical references, has to make a high effort, unemotional comment to try to make a point. Even or especially if they manage to do this, they can expect to be hit with a wall of text with various hostilities.
Like, this is awful. This isn’t just bad but it’s borderline abusive.
It’s wild that that this is the level of discourse here.
Because of the amount of reputation, money and ingroupness, this is probably one of the most extreme forms of tribalism that exists.
Do you know how much has been lost?
Charles, consider going for that walk now if you’re able to. (Maybe I’m missing it, but the rhetorical moves in this thread seem equally bad, and not very bad at that.)
You are right, I don’t think my comments are helping.
Like, how can so many standard, stale patterns of internet forum authority, devices and rhetoric be rewarded and replicate in a community explicitly addressing topics like tribalism and “evaporative cooling”?
The moderators feel that some comments in this thread break Forum norms and are discussing what to do about it.
Here are some things we think break Forum norms:
Rude/hostile language and condescension, especially from Charles He
Gwern brings in an external dispute — a thread in which Charles accuses them of doxing an anonymous critic on LessWrong. We think that bringing in external disputes interferes with good discourse; it moves the thread away from discussion of the topic in question, and more towards discussions of individual users’ characters
The conversation about the external dispute gets increasingly unproductive
The mentioned thread about doxing also breaks Forum norms in multiple ways. We’ve listed them on that thread.
The moderators are still considering a further response. We’ll also be discussing with both Gwern and Charles privately.
The moderation team is issuing Charles a 3-month ban.
I honestly don’t see such a problem with Gwern calling out out Charles’ flimsy argument and hypocrisy using an example, be it a part of an external dispute.
On the other hand, I think Charles’ uniformly low comment quality should have had him (temporarily) banned long ago (sorry Charles). The material is generally poorly organised, poorly researched, often intentionally provocative, sometimes interspersed with irrelevant images, and high in volume. One gets the impression of an author who holds their reader in contempt.
I don’t necessarily disagree with the assessment of a temporary ban for “unnecessary rudeness or offensiveness”, or “other behaviour that interferes with good discourse”, but I disagree that Charles’ comment quality is “uniformly” low or that a ban might be merited primarily because of high comment volume and too low quality.There are some real insights and contributions sprinkled in in my opinion.
For me the unnecessary rudeness or offensiveness and other behavior interfering with discourse comes from things like comments that are technically replies to a particular person but seem like they’re mostly intended to win the argument in front of unknown readers, and containing things like rudeness, paranoia, and condescension towards the person they’re replying to. I think the doxing accusation, which if I remember correctly actually doxxed the victim much more than Gwern’s comment, is part of a similar pattern of engaging poorly with a particular person, partly through an incorrect assessment that the benefits to bystanders will outweigh the costs. I think this sort of behavior stifles conversation and good will.
I’m not sure a ban is a great solution though. There might be other, less blunt ways of tackling this situation.
What I would really like to see is a (much) higher lower limit of comment quality from Charles i.e. moving the bar for tolerating rudeness and bad behavior in a comment much higher even though it could be potentially justified in terms of benefits to bystanders or readers.
This is useful and thoughtful. I will read and will try to update on this (in general life, if not the forum?) Please continue as you wish!
I want to notify you and others, that I don’t expect such discussion to materially affect any resulting moderator action, see this comment describing my views on my ban.
Below that comment, I wrote some general thoughts on EA. It would be great if people considered or debated the ideas there.
Comments on Global Health
Comments on Animal Welfare
Comments on AI Safety
Comments on two Meta EA ideas
I don’t disagree with your judgement of banning but I point out there’s no banning for quality—you must be very frustrated with the content.
To get a sense of this, for the specific issue in the dispute, where I suggested the person or institution in question caused a a 4 year delay in funding, are you saying it’s an objectively bad read, even limited to just the actual document cited? I don’t see how that is.
Or is this wrong, but requires additional context or knowledge.
Re the banning idea, I think you could fall afoul of “unnecessary rudeness or offensiveness”, or “other behaviour that interferes with good discourse” (too much volume, too low quality). But I’m not the moderator here.
My point is that when you say that Gwern produces verbose content about a person, it seems fine—indeed quite appropriate—for him to point out that you do too. So it seems a bit rich for that to be a point of concern for moderators.
I’m not taking any stance on the doxxing dispute itself, funding delays, and so on.
I agree with your first paragraph for sure.