yooo x-risk is a cult you should get out while you can <3
beth
Karma: 221
I believe your assessment is correct, and I fear that EA hasn’t done due diligence on AI Safety, especially seeing how much effort and money is being spent on it.
I think there is a severe lack of writing on the side of “AI Safety is ineffective”. A lot of basic arguments haven’t been written down, including some quite low-hanging fruit.
As per my initial comment, I’d compare it to pre-WWII Netherlands banning government registration of religion. It could safe tens of thousands of people from deportation and murder.
Just in the past weeks, San Francisco, Oakland and Cambridge.
For a more extreme hypothesis, Ariel Conn at FLI has voiced the omnipresent Western fear of resurgent ethnic cleansing, citing the ease of facial recognition of people’s race—but has that ever been the main obstacle to genocide? Moreover, the idea of thoughtless machines dutifully carrying out a campaign of mass murder takes a rather lopsided view of the history of ethnic cleansing and genocide, where the real death and suffering is not mitigated by the presence of humans in the loop more often than it is caused or exacerbated by human passions, grievances, limitations, and incompetency.
I am not a historian, but during the Nazi regime, The Netherlands had among the highest percentages of Jews killed in all of Western Europe. I remember historians blaming this on the Dutch having thorough records of who the Jews were and where they lived. Access to information is definitely a big factor in how succesful a genocidal regime can be.
The worry is not so much about killer robots enacting a mass murder campaign. The worry is that humans will use facial recognition algorithms to help state-sanctioned ethnic cleansing. This is not a speculative worry. There are a lot of papers on Uyghur facial recognition.
I don’t have any specific instances in mind.
Regarding your accounting of cases, that was roughly my recollection as well. But while the posts might not address the second concern directly, I don’t think that the two concerns are separable. The actual mechanisms and results might largely overlap.
Regarding the second concern you mention specifically, I would not expect those complaints to be written down by any users. Most people on any forum are lurkers, or at the very least they will lurk a bit to get a feel for what the community is like and what it values before participating. This makes people with oft-downvoted opinions self-select out of the community before ever letting us know that this is happening.
The hovering is helpful, thank you.
Are there any plans to evaluate the current karma system? Both the OP and multiple comments expressed worries about the announced scoring system, and in the present day we regularly see people complain about voting behaviour. It would be worth knowing if the concerns from a year ago turn out to have been correct.
Related to this, I have a feature request. Would it be possible to break down scores in a more transparent way, for example by number of upvotes and downvotes? The current system gives very little insight to authors about how much people like their posts and comments. The lesson to learn from getting both many upvotes and many downvotes is very different from the lesson to learn if nobody bothered to read and vote on your content.
Thank you so much for posting this. It is nice to see others in our community willing to call it like it is.
I was talking with a colleague the other day about an AI organization that claims:
AGI is probably coming in the next 20 years.
Many of the reasons we have for believing this are secret.
They’re secret because if we told people about those reasons, they’d learn things that would let them make an AGI even sooner than they would otherwise.
To be fair to MIRI (who I’m guessing are the organization in question), this lie is industry standard even among places that don’t participate in the “strong AI” scam. Not just in how any data-based algorithm engineering is 80% data cleaning while everyone pretends the power is in having clever algorithms, but also in how startups pretend use human labor to pretend they have advanced AI or how short self-driving car timelines are a major part of Uber’s value proposition.
The emperor has no clothes. Everyone in the field likes to think they are aware of this fact already when told, but it remains helpful to point it out explicitly at every opportunity.
This is mostly a problem with an example you use. I’m not sure whether it points to an underlying issue of your premise:
You link to the exponential growth of transistor density. But that growth is really restricted to just that: transistor density. Growing your number of transistors doesn’t necessarily grow your capability to compute things you care about, both from a theoretical perspective (potential fundamental limits in the theory of computation) as well as a practical perspective (our general inability to write code that makes use of much circuitry at the same time + the need for dark silicon + Wirth’s law). Other numbers, like FLOP/s, don’t necessarily mean what you’d think either.
Moore’s law does not posit exponential growth in amount of “compute”. It is not clear that the exponential growth of transistor density translates to exponential growth of any quantity you’d actually care about. I think it is rather speculative to assume it does and even more so to assume it will continue to.
These are some issues that actively frustrate me to the point of driving me away from this site.
Loading times for most pages are unbearably slow. So are most animations (like the menu from clicking your username top right).
Many features break badly when Javascript is turned off.
Text field for bio is super small and cannot be rescaled.
Super upvotes have their use but the super downvote just encourages harsh voting behaviour.
The contrast on the collapse comment button is minimal, same for a number of other places.
Basic features take much effort to navigate to. Going to all posts either means two clicks (hamburger menu then all posts) or clicking a link that can not always be seen without scrolling (which is a mess because the page height will change when recent comments have finished loading)
Sure it is, but I know a lot more about myself than I do about other people. I could make a good guess on impact on myself of a worse guess on impact on others. It’s a bias/variance trade-off of sorts.
I’d say the two are valuable in different ways, not that one is necessarily better than the other.
Any technology comes with its own rights struggle. Universal access to super-longevity, the issue of allowing birth vs exploding overpopulation if everyone were to live many times longer, em rights, just to name a few. New tech will hardly have any positive effect if these social issues resolve in a wrong way.
Can you make a case as to why the two have enough notability separately to deserve their own separate Wikipedia pages?
Regarding 1), if I were to guess which events of the past 100 years made the most positive impact on my life today, I’d say those are the defeat of the Nazis, the long peace, trans rights and women’s rights. Each of those carries a major socio-political dimension, and the last two arguably didn’t require any technological progress.
I very much think that socio-political reform and institutional change are more important for positive long-term change than technology. Would you say that my view is not empirically grounded?
it reflects a sentiment that effective altruism is not about one thing, about having the right politics, about saying the right things, about adopting groupthink, or any of the many other things we associate with ideology.
Can you expand a bit on this statement? I don’t see how you can say only other ideologies of being full of groupthink and having the right politics, even though most posts on the EA forum that don’t agree with the ideological tennets listed in the OP tends to get heavily downvoted. When I personally try to advocate against the idea that AI Safety is an effective cause, I experience quite some social disapproval for that within EA.
I think the points you’re complaining about affect EA just as much as any other ideology, but that they are hard to see when you are in the midst of it. Your own politics and groupthink don’t feel like politics and groupthink, they feel like that is the way the world is.
Let me try to illustrate this using an example. Plenty of people accuse any piece of popular media with a poc/female/lgbt protagonist as being overly political, seemingly thinking that white cishet male protagonists are the unique non-political choice. Whether you like this new trend or not, it is absurd to think that one position here is political and the other isn’t. But your own view always looks apolitical from the inside. For EA this phenomenon might be compounded by the fact that there is no singular opposing ideology.
It is most apparent in this piece of the review:
He also points out that Tanzanian natives using their traditional farming practices were more productive than European colonists using scientific farming. I’ve had to listen to so many people talk about how “we must respect native people’s different ways of knowing” and “native agriculturalists have a profound respect for the earth that goes beyond logocentric Western ideals” and nobody had ever bothered to tell me before that they actually produced more crops per acre, at least some of the time. That would have put all of the other stuff in a pretty different light.
He remains focused on the expected crops per acre, even though every case study in the book illustrates that such a single variable doesn’t encompass the multitude of uses that the acre in question has. I don’t think I could describe it better than Reddit user u/TheHiveMindSpeaketh does:
The point of the book is not to point and laugh at the technocrats who failed to squeeze the most X out of Y because they didn’t listen to the noble savages. The point is that ‘how do we squeeze the most X out of Y’ is a bad way to position yourself in relation to your surroundings. The point is that technocrats often succeed in squeezing more X out of Y over a relevant period of time via their techniques, but that treating a forest like a timber-maximizer is already missing the [..] point because a forest is also a home for woodland creatures, and a source for medicinal herbs and fruits and berries, and a nice place to take a hike and stare at the stars. The point is that the mistake was not made at the level of what was implemented, the mistake was made at the level of what was valued, and the implementation mistake was an inevitable downstream consequence of that. The point is that even if traditional Tanzanian farming methods didn’t produce more crops per acre, they might still be preferable, because they are more sustainable or less time-intensive or etc, but that these benefits become unintelligible to the technocrat who has already committed to a value system where land is only judged by its yield per acre.
I personally think this is an important question for EA’s to grapple with: can we reason abstractly about doing good without this abstraction causing mistakes at the level of what to value. Scott’s technocrats surely did not think they were making that mistake, but they were. If we believe that we are somehow different, that is kind of arrogant.
For a different take on the consequences of being “rational”, I would highly recommend James C. Scott’s book Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. The book summary of SSC is pretty good, but when he gives his opinion on the book he seems to have missed the point of the book entirely.
Thank you for your response.
Yes, that is what I meant. If you could convince me that AGI Safety were solvable with increased funding, and only solvable with increased funding, that would go a long way in convincing me of it being an effective cause.
In response to your question of giving up: If AGI were a long way off from being built, then helping others now is still a useful thing to do, no matter if either of the scenarios you describe were to happen. Sure, extinction would be bad, but at least from some person-affecting viewpoints I’d say extinction is not worse than existing animal agriculture.
Let me try to rephrase this part, as I consider it to be the main part of my argument and it doesn’t look like I managed to convey what I intended to:
AI Safety would be a worthy cause if a superintelligence were powerful and dangerous enough to be an issue but not so powerful and dangerous as to be uncontrollable.
The most popular cause evaluation framework within EA seems to be Importance/Neglectedness/Tractability. AI Safety enthusiasts tell a convincing story on importance and neglectedness being good and make an effort at arguing that tractability is as well.
But here is the thing: all arguments given in favour of AI being risky (to establish importance) can be rephrased as arguments against tractability. Similarly for neglectedness.
I’ll illustrate this with a caricature, but it takes little effort to transfer this line of thought to the real arguments being made. Let’s say the pro-AIS argument is “AGI will become infinitely smart, so it can out-think all humans and avoid all our security measures. Hence AGI is likely to escape any restrictions we put on it, so it will be able to tile the universe with paperclips if it wants to”. Obviously, if it can out smart any security measure, then no sufficient security exists, AI Safety research will never lead to anything and the problem is intractable.
AI Safety is only effective if you can simultaneously argue for each of importance/neglectedness/tractability without detracting from the others. Moreover, your arguments have to address the exact same scenarios. It is not enough for AIS to be important with 50% probability and tractable with 50% probability, these two properties have be likely to hold simultaneously. A coin flip has 50% probability of heads and 50% probability of tails, but they will never happen at the same time.
AI Safety can only be an effective cause (on the margin) if solving it is possible (tractability) but not trivial (importance/neglectedness). I think this is a narrow window to hit, and current arguments are all way off-target.
Same for the unconference, should be this link.
Fighting human rights violations around the globe.