(have only skimmed the post, but sending this link in case it is relevant) - https://cltc.berkeley.edu/seeking-input-and-feedback-ai-risk-management-standards-profile-for-increasingly-multi-purpose-or-general-purpose-ai/
I am part of a team collaborating with him
It is still the EU, so those things aren’t going anywhere :) And in the discussion on real-time Biometric monitoring, “protection of democracy” was a main in talking point, making references to social scoring etc in China
The value
NIL
is not of type
SIMPLE-STRING
when binding #:USER-ID162
I’m not sure I agree with the conclusion but I like the overall analysis, I think it is very useful.
I’m confused, we make this caution compromise all the time—for example, medicine trial ethics. Can we go faster? Sure, but the risks are higher. Yes, that can mean that some people will not get a treatment that is developed a few years too late.
Another closer example is gain of function research. The point is, we could do a lot, but we chose not to—AI should be no different.
Seems to me that this post is a little detached from real world caution considerations, even if it isn’t making an incorrect point.
Might be better to look for a Prof doing interesting/relevant research, rather than specifically a PhD program
Ah right, I had that thought but wasn’t sure, makes sense!
From my playing with it, ChatGPT uses complex language even when told not to. In notion, there’s a AI assistant (GPT3 based) and it has a “simplify writing” feature. The outputs were still pretty verbose and had overly long sentences. Soon though, sure!
Well said, though I think your comment could use that advice :) Specific phrases/words I noticed: reign in, tendancy, bearing in mind, inhibit, subtlety, IQ-signal (?).
I’m non-native and I do know these words, but I’m mostly native level at this point (spent half my life in an English speaking country) I think many non-native speakers won’t be as familiar
I came across this some time ago through lesswrong - https://putanumonit.com/2016/02/03/015-dating_1/
might be what your looking for :)
As a (semi humorous) devils advocate, if we applied existential risk/longtermist ideas to non human animals, couldn’t the animal lives on Mars still be net positive, as they help their respective species flourish for billions of years, and reduce their risk of going extinct if they were only on earth?
I’m not sure I take this seriously yet, but it’s interesting to think about.
This was really well written! I appreciate the concise and to the point writing style, as well as a summary at the top.
Regarding the arguments, I think they make sense to me. Although this is where the whole discussion of longtermism does tend to stay pretty abstract, if we can’t actually put real numbers on it.
For ex, in the spirit of your example—does working on AI safety at MIRI prevent extinction, while assuming a sufficiently great future compared to, say, working on AI capabilities at OpenAI? (That is, maybe a misaligned AI can cause a greater future?)
I don’t think it’s actually possible to do a real calculations in this case, and so we make the (reasonable) base assumption that a future with alligned AI is better than a future with a misaligned AI, and go from there.
Maybe I am overly biased against longtermism either way, but in this example it seems to me like the problem you mention isnt really a real-world worry, but only really a theoretically possible pascals mugging.
Having said that I still think it is a good argument against strong longtermism
While this is important (clarifying of misinformation), I want to mention that I don’t think this takes away from the main message of the post. I think it’s important to remember that even with a culture of rationality, there are times when we won’t have enough information to say what happened (unlike in Scotts case), and for that reason Mayas post is very relevant and I am glad it was shared.
It also doesn’t seem appropriate to mention this post as “calling out”. While it’s legitimate to fear reputations being damaged with unsubstantiated claims, this post doesn’t strike me as doing such.
Great post, well explained!
I like the idea of having a more relatable message than “do the most good”, but I am not sure how much more relatable “do alot of good” is. To me it seems that there might not be that much of a difference between the two, at least in how they are used in day to day discussion (that is, applying a filter of “practicality” to the maximization problem).
For example, I thought it was common EA Knowledge that there are “top recommended cause areas” (on 80K), where some are higher on the list but with a big * of uncertainty. Theres also enough people to work on all of them, so there’s no need for a final judgement of a top 3, let alone 1 most important cause. In a way this could be a “macro” EA perspective—asking not what is the most good an individual could do, but what is the most good a group/society can do, with appropriate allocation between cause areas of high ITN.
I think EA can come across as a bit elitist to others, especially to people voulenteering in non-EA charities or trying to do good in “traditional” ways (doctors, Med-tech, activism, etc). Perhaps the “do alot of good” can help with that—but I still think it would come to similar conclusions in some cases. I have a friend who is volenteering in “Make a Wish” for the past 10 years, and I felt a little uneasy telling him about EA without offending him—though I was able to, and while he was intrigued I don’t think he was convinced.
I had a thought a while ago, that perhaps the world would be much better if there were alot of people committed to doing “at least a little good” , rather than a (relatively) small group of highly ambitious people doing “the most good”. However, perhaps there is room for that as a separate movement from EA. Plus, someone for sure needs to work on the “big things” too, which seems like a good niche for EA.
I really loved reading the Sequences, but I don’t think they are actually a practical rationality guide. (And EY also acknowledges this as far as I know).
I try to keep some cognitive biases in Anki as flashcards, and review some personal reflections and writings, some of which have some valuable rationality reminders, or articulate ideas in more relatable ways to my own life experience.
The site “Effectiviology” has some great practical cognitive bias/rationality practical tips. The biggest one I try to implement is externalizing my thought process—both by talking to others (from casual chat to formal design reviews), and by metacognition—keeping a written log of my thoughts as I work, especially when I get stuck.
There’s also the CFAR handbook, though I have yet to go though it.
Isn’t it possible that we already have 2 (or more) separate “Qualias”, i.e the left and right brain? But, being highly integrated, don’t result in any noticable experimental consequences in the way that Split brain patients do (it could also be that only one side is conscious, and the other side only seems) I’m thinking a bit in the way it’s described in the short story “Learning to be Me” by Greg Egan.
I also like the thought experiment in Shelly Kagans great YouTube lecture series (philosophy of death) - if we assumed the soul existed (and was “you” in the sense of conscious experience/qualia), you wouldn’t be able to tell if you are the same soul you were yesterday, or even 1 millisecond ago. Someone could in theory replace your “soul”, while keeping all of the memories intact, and the result would be subjectively the same.
To me this is a nice reminder that a single continuous self (or consciousness) in the way I like to think of it might not make much sense, or perhaps that other possibilities exist that fit the data.
One could imagine me being the right brain for one moment, the left brain for another, or being both at the same time—eliminating the contradiction from the Split brain experiment.
This was a great read! I liked the structure as well, which allowed me to jump across sections before deciding to read the whole thing.
I got into meditation a few years ago and recently did a Vipassana 10 day retreat. I guess it’s hard to quantify systematic change but it is interesting to imagine what would happen if meditation was taught in schools—though I prefer it in non-spiritual form, just pure practice.
I’m still mostly a beginner in EA, so I am not sure if I’m qualified yet to judge your views on longtermism but it seems to make sense to me.
Regarding the book—perhaps it wouldn’t be a bad thing that it is more geared towards a “popular audience”. It is too easy to aim too high, and that perhaps makes much of EA writing somewhat impenetrable to many people who aren’t yet onboard.
This makes sense to me, good writeup!