trevor1

• trevor1Sep 15, 2023, 5:09 PM

  6 points

  2 ∶ 2

  on: Who should we in­ter­view for The 80,000 Hours Pod­cast?

  Tetlock. Yes, again.

• trevor1Sep 10, 2023, 7:46 PM

  0 points

  0 ∶ 3

  in reply to: Grumpy Squid’s comment on: Grumpy Squid’s Quick takes

  EAforum needs a broad “obvious bad actor” ban, since EA is the kind of group where an unusually large proportion of people are sufficiently capable (and motivated) at thinking outside the box to work around/​bend rules in order to achieve a goal.

  EAforum currently has guidelines on this, but no clear policy:

  

  It’s possible that strategic ambiguity might be neccesary, since such a large number of people are willing and able to spend a ton of time thinking of ways to work around the rules in pursuit of an objective. If the rules aren’t clear, then people can’t form complex plots to find loopholes or historically unprecedented approaches.

• trevor1Sep 10, 2023, 7:38 PM

  3 points

  2 ∶ 0

  in reply to: Yadav’s comment on: Cruxes on US lead for some do­mes­tic AI regulation

  I heard that chip fabs/​factories are extremely soft targets and would be destroyed by the losing side either way. This is definitely the right kind of way to think about this though.

• trevor1Aug 31, 2023, 2:29 AM

  2 points

  0 ∶ 0

  on: Long-Term Fu­ture Fund Ask Us Any­thing (Septem­ber 2023)

  How small and short can a grant be? Is it possible for a grant to start out small, and then gradually gets bigger and sources more people if the research area turns out to be significantly more valuable than it initially appeared? If there’s very few trustworthy math/​quant/​AI people in my city, could you help me source some hours from some reliable AI safety people in the Bay Area if the research area clearly ends up being worth their time?

• trevor1Aug 31, 2023, 2:21 AM

  1 point

  0 ∶ 3

  on: Long-Term Fu­ture Fund Ask Us Any­thing (Septem­ber 2023)

  Is there any way for me to print out and submit a grant in paper, non-digital form, also without mailing it? e.g. I send an intermediary to meet one of your intermediaries at some berkeley EA event or something, and they hand over an envelope containing several identical paper copies of the grant proposal. No need for any conversation, or fuss, or awkwardness, and the papers can be disposed of afterwards and normal communication would take place if the grant is accepted. I know it sounds weird, but I’m pretty confident that this mitigates risks of a specific class.

• trevor1Aug 31, 2023, 2:14 AM

  0 points

  1 ∶ 2

  on: Long-Term Fu­ture Fund Ask Us Any­thing (Septem­ber 2023)

  Infohazard policy/​committment? I’d like to make sure that the person who read the grant takes AI safety seriously and much more seriously than other X-risks, to me that’s the main and only limiting factor (I don’t worry about taking credit for others ideas, profiting off of knowledge, or sharing info with with others as long as the sharing is done in a way that takes AI safety seriously, only that the reader is not aligned with AI safety). I’m worried that my AI-related grant proposal will distract large numbers of people from AI safety, and I think that someone who also prioritizes AI safety would, like me, act to prevent that (consistently enough for the benefits of the research to outweigh the risks).

• trevor1Aug 29, 2023, 6:00 AM

  1 point

  2 ∶ 2

  in reply to: Jonas V’s comment on: Jonas Vol­lmer’s Shortform

  All of this looks fantastic and like it should have been implemented 10 years ago. This is not something to sleep on.

  The only nitpick I have is with how object level vs social reality is described. Lots of people are nowhere near ready to make difficult calculations, e.g. the experience from the COVID reopening makes it hard to predict that pandemic lockdown in the next 5 years is 40% even if that is the correct number. There’s lots of situations where the division of labor is such that deferring to people at FHI etc. is the right place to start, since these predictions are really important and not about people giving their own two cents or beginning learning the ropes of forecasting, which is what happens all too often (of course, that shouldn’t get in the way of new information and models travelling upwards, or fun community building/​talent development workshops where people try out forecasting to see if its a good fit for them).

• trevor1Aug 29, 2023, 5:31 AM

  2 points

  1 ∶ 0

  in reply to: Joseph’s comment on: Joseph Lemien’s Shortform

  I think this is a really big and valuable finding, and generally agree with your thinking about language and morality differences, which are valuable research areas.

  Anyone doing a deeper dive in the paper might want to think about whether Chinese survey participants are surprised to see relatively extreme and serious crimes like theft and violence and decide not to touch those concepts with a ten foot pole, and default to things that people frequently talk about or are frequently criticized by official news sources and propaganda.

  Not that they’re super afraid of checking a box or anything; it’s just that it’s only a survey and they don’t know the details of what’s going on, and by default the tiny action is not worth something complicated happening or getting involved in something weird that they don’t understand. Or maybe it’s only that they think it’s acceptable to criticize things that everyone is obviously constantly criticizing, especially in an unfamiliar environment where everything is being recorded on paper permanently (relative to verbal conversations which are widely considered safer and more comfortable). It’s not that people are super paranoid, but, like, why risk it if some unfair and bizarre situation could theoretically happen (e.g. corruption-related, someone’s filling quotas), and conformity is absolutely guaranteed to be safe and cause no major or minor disturbances to your daily life?

  I didn’t read the paper, and these musings should only be seriously considered as potentially helpful for people reading the paper. The paper seems to have run other forms of surveys that point towards similar conclusions.

• trevor1Aug 27, 2023, 11:31 PM

  −19 points

  1 ∶ 10

  in reply to: Lizka’s comment on: Eliezer Yud­kowsky Is Fre­quently, Con­fi­dently, Egre­giously Wrong

  I really appreciate this mod comment. I was seriously wracking my brain, trying to think of how to put into words exactly what was wrong with this post, and nothing I pulled out was anywhere near as concise or precise as this comment. This was done extremely well, something that I’m able to recognize after the fact but nowhere near able to do myself.

  I still wish it had come sooner. It’s also worth noting that this post made a really serious effort to optimize for maximizing damage to the reputation to at least one of the major Schelling points in the Rationality community (the Lesswrong Sequences), which many people in EA also use for upskilling in preparation for high-EV projects. It’s not clear how many people read the arguments at the beginning and assumed they were true, rather than that the author had insulated themself from accountability. Rather than hosting bad-faith attacks on other communities for more than 20 hours and following up with heavy-handed actions, like moving to personal blog, that couldn’t have been done sooner than ~20 hours, it would make more sense to take lighter actions sooner, such as just posting a superior-quality disclaimer warning very early on in the process (and, again, this comment nailed the situation in a really effective way). This will maximize the number of people who can make informed decisions about the content itself, rather than just go wherever it was designed to take them.

  I’m not sure how that something like this would be, as I’m not a moderator and I don’t know what the base rates are for moderator capability, as well as other factors that I’m not aware of; I know that this situation was ultimately handled well but possibly could have been done better.

  Yudkowsky made a long reply, but it also contained this point:

  As the entire post violates basic rules of epistemic conduct by opening with a series of not-yet-supported personal attacks, I will not be responding to the rest in detail. I’m sad about how anything containing such an egregious violation of basic epistemic conduct got this upvoted, and wonder about sockpuppet accounts or alternatively a downfall of EA.

  Although forum cybersecurity is very important since we don’t know what kinds of people will be the adversaries in the future, I don’t think this is as much of an indictment or a dissappointment as Yudkowsky thinks. As a thought experiment, if EA were to grow by a factor of two each year, then that would mean, at any given time, a disproportionately large proportion of people in EA would have only had their first exposure to EA ideas less than 6 months ago. However, it would still mean that it is even more important to acknowledge that, at any given time, many newer users aren’t yet ready to handle complex bad-faith arguments and mind games that inevitably end up getting cooked up by an intelligent disgruntled person here or there; this would happen even if EA is doing everything right, and continuously getting better every day, by expanding and bringing in many new people, which is already hard enough as is in the current ambiently-hostile environment.

• trevor1Aug 27, 2023, 7:48 PM

  8 points

  4 ∶ 1

  in reply to: seanrson’s comment on: Eliezer Yud­kowsky Is Fre­quently, Con­fi­dently, Egre­giously Wrong

  None of these issues are “central” to AI or the cognitive science that’s relevant to AI, AI alignment, or human upskilling. The author’s area of interest is more about consciousness, animal welfare, and qualia.

  The issues are the sole thing justifying Omnizoid’s rather heated indictments against Yudkowsky, such as:

  Eliezer has swindled many of the smartest people into believing a whole host of wildly implausible things. Some of my favorite writers—e.g. Scott Alexander—seem to revere Eliezer. It’s about time someone exposed the mountain of falsehoods on which his arguments rest. If one of the world’s most influential thinkers is just demonstrably wrong about lots of topics, often in ways so egregious that they demonstrate very basic misunderstandings, then that’s quite newsworthy, just as it would be if a presidential candidate supported a slate of terrible policies.

  Most readers will only read the accusations in the introduction and then bounce off the evidence backing them; because all of them are topics that, like string theory, only a handful of people on earth are capable of engaging with them. It just so happens that the author is one of them. Virtually nobody can read the actual arguments behind this post without dedicating >4 hours of their life to it, which makes it pretty well optimized to attract attention and damage Yudkowsky’s reputation as much as possible with effectively zero accountability.

• trevor1Aug 27, 2023, 6:02 PM

  −1 points

  0 ∶ 1

  on: Eliezer Yud­kowsky Is Fre­quently, Con­fi­dently, Egre­giously Wrong

  I feel like there are two types of thinkers, the first we might call innovators and the second systematizers. Innovators are the kinds of people who think of wacky, out-of-the-box ideas, but are less likely to be right. They enrich the state of discourse by being clever, creative, and coming up with new ideas, rather than being right about everything. A paradigm example is Robin Hanson—no one feels comfortable just deferring to Robin Hanson across the board, but Robin Hanson has some of the most ingenious ideas.

  Systematizers, in contrast, are the kinds of people who reliably generate true beliefs on lots of topics. A good example is Scott Alexander. I didn’t research Ivermectin, but I feel confident that Scott’s post on Ivermectin is at least mostly right.

  I think people think of Eliezer as a systematizer. And this is a mistake, because he just makes too many errors. He’s too confident about things he’s totally ignorant about. But he’s still a great innovator. He has lots of interesting, clever ideas that are worth hearing out. In general, however, the fact that Eliezer believes something is not especially probative. Eliezer’s skill lies in good writing and ingenious argumentation, not forming true beliefs.

  Although this is a pretty gross oversimplification, it does touch on some valuable points.

  I don’t think it makes sense to assume an equivalence between Hanson and Yudkowsky. Hanson has bumbled into the AI arena in a way that Yudkowsky never would. It’s possible that Hanson has only developed skill at making groundbreaking discoveries in economics and sociology, and stumbled instead of hitting the ground running when he ventured into AI, and that Yud did a similar thing with consciousness ethics. But even if true, that’s where the similarities would end, because both human neurotypes, and Hanson and Scott Alexander and Yudkowsky’s personal histories and skills, are each vastly more complicated than the oversimpified innovator-systematizer dichotomy that this post leaves as its concluding argument.

• trevor1Aug 27, 2023, 4:16 PM

  1 point

  0 ∶ 2

  on: trevor1′s Quick takes

  ‘It’s genuinely a criterion for genius to say a bunch of wrong stupid things sometimes. Someone who says zero stupid things isn’t reasoning from first principles, isn’t taking risks, and has downloaded all the “correct” views.’

  Source (I did not write this).

• trevor1Aug 26, 2023, 7:31 PM

  −1 points

  2 ∶ 1

  in reply to: NunoSempere’s comment on: EA is un­der­es­ti­mat­ing in­tel­li­gence agen­cies and this is dangerous

  Thanks! I think that things with even a ~3% chance of destroying EA are still worthwhile for people to look into.

EA is un­der­es­ti­mat­ing in­tel­li­gence agen­cies and this is dangerous

• trevor1Aug 23, 2023, 9:20 PM

  0 points

  0 ∶ 0

  in reply to: Joseph’s comment on: Joseph Lemien’s Shortform

  oh my GOD I cannot tell you how much I needed this

• trevor1Aug 23, 2023, 5:50 PM

  4 points

  1 ∶ 0

  in reply to: Joseph’s comment on: Joseph Lemien’s Shortform

  This is actually a pretty big issue. It was basically locked in to Meditations on Moloch because it was too good. The essay does a really good job explaining it, and giving examples that create the perspective you need to understand the broad applicability of the concept, but has too many words; “incentives” or even a single phrase (e.g. “race to the bottom”) would have fewer words, but it wouldn’t give the concept the explanation that it’s worth. Maybe there could be some kind of middle ground.

• trevor1Aug 22, 2023, 7:33 PM

  2 points

  0 ∶ 0

  in reply to: ag4000’s comment on: trevor1′s Quick takes

  I definitely agree- the calculation that I had in mind, and physically wrote, clearly had in mind the 1% compounding on itself.

  I have two defenses that it is still a helpful way of looking at it:

  1. This was a heuristic, and getting a little better every day will still stack in the way that I and the reader had in mind when reading/​writing the quote, even if the math wasn’t completely accurate for something like a daily 1% yielding something towards a 30x improvement per year.

  2. Getting better at different tasks over the course of the year will still multiply, such as getting better at different tasks related to working on/​thinking about/​solving AI alignment.

• trevor1Aug 22, 2023, 12:31 AM

  10 points

  2 ∶ 0

  on: Public Cog­ni­tive Dis­so­nance About Ex­is­ten­tial Risk Is Terrifying

  My model of this is that the human brain evolved, over the last ~3000 years (possibly 10,000 or 1000), to avoid being “hacked” by the words of other humans; specifically, hacked into sacrificing physical resources or social status to someone who optimized for finding a combination of words or ideas that could do that. For example, the people who were persuaded by ancient priests to give an unusually large tithe to the sun god temple (e.g. via proto pascal’s wager arguments) had less viable offspring, and people resistant to persuasion about anything had dominated the genepool by the time of the industrial revolution. A dynamic like this, but much more nuanced (e.g. maybe conformity-obsession was the mechanic itself, not resistance to new ideas).

  Unfortunately, the current state of our culture (in EU and particularly NA) is that because nobody bled to death or got burned to a crisp by the million for almost 80 years, we then end up in the pitiful situation where most people extrapolate the last 80 years of peace of our parents and grandparents, onto the next 80 years for our lives and our children and grandchildren. Given that the world was alright for 80 years, that means that the predictions of many “doomsayers” necessarily must have been wrong for 80 years. Ancient Rome had a similar situation before it collapsed.

  There’s also the problem of EA being “Punchable”, in the worlds of Holden; EA is in fact an attack on the legitimacy of the Left and Right by default, merely by making an effort to be altruistic:

  Having seen the EA brand under the spotlight, I now think it isn’t a great brand for wide public outreach. It throws together a lot of very different things (global health giving, global catastrophic risk reduction, longtermism) in a way that makes sense to me but seems highly confusing to many, and puts them all under a wrapper that seems self-righteous and, for lack of a better term, punchable?

  The issue is that when you arrive and make your pitch, the average person is oriented towards their reality where they are in the right for expecting a peaceful existence, rather than the real reality where you are in the right for making your pitch. It is helpful to think of it maybe more like a startup founder pitching their idea to an investor, except rather than a transaction of cash that the investor is professional and experienced about, it’s an exchange of a way of life, and you are arguing that their lifeplan was pretty poorly planned out, and they aren’t professional or experienced about hearing this the way that investors are professional and experienced about hearing elevator pitches.

  In the 2020s, you also have to tack on the rise of social media misinformation and competing messages. People’s friend groups are now filled with hot takes that optimize for memetic spread, and people see ideas that cause fear as something that their friends have fallen victim to, specifically on social media. Competing intense messages make people withdraw into their shell as a pretty basic adjustment/​immune response, regardless of what reality looks like.

  What helped me was reading about dath ilan, specifically the basement. Seeing it from the perspective of a civilization where people give the problem a proportionate amount of weight, gave me a point of reference that helped have more precise thoughts about the current situation.

  TL;DR The root cause of this phenomenon is well described by Kevin Simler’s Down the Rabbit Hole, and this image:

  

  This is just how people are. It’s the full context of the situation. The enemy’s gate is down.

• trevor1Aug 21, 2023, 2:18 AM

  −3 points

  0 ∶ 2

  in reply to: Matt_Sharp’s comment on: trevor1′s Quick takes

  HPMOR technically isn’t built to be time-efficient, the highlights of the sequences is better for that. HPMOR is meant to replace other things you do for fun like reading fun novels or TV shows or social media, and replace that with material that offers passive upskilling. In that sense, it is profoundly time-efficient, because it replaces fun time spent not upskilling at all, with fun time spent upskilling.

  A very large proportion of EA-adjacent people in the bay area swear by it as a way to become more competent in a very broad and significant way, but I’m not sure how it compares with other books like Discworld which are also intended for slack/​leisure time. AFAIK CEA has not even done a survey explicitly asking about the self-improvement caused by HPMOR, let alone study measuring the benefits of having different kinds of people read it.

• trevor1Aug 21, 2023, 2:07 AM

  4 points

  0 ∶ 1

  in reply to: Joseph’s comment on: trevor1′s Quick takes

  Upvoted. I was deliberately vague about “1% better every day” because the idea was the person becomes 1% better as a whole; my thinking on this wasn’t complete.

  Now that I’ve read your comment, I can make the concept more nuanced: the person becomes 1% better by improving a skill by 1% and moving on to a different skill as they approach diminishing returns.

  So they would do 1% more push-ups, then as they started hitting diminishing returns they spend more time on memorizing vocabulary, then linear algebra, then security mindset, then integrating bayes theorem deeply into their thoughts, then data science, then reading Christiano’s papers, then networking, etc.