I don’t think the relevant value here is P(goes to school | smart person). I think it’s P(smart person | goes to school). The latter seems much higher for Ivies (Ivies = top ranked schools) than anywhere else, except places that excel in a certain area (e.g., CMU, although this might also be top ranked).
oh54321
Karma: 229
I’m maximizing good, not my contribution to good
[Question] Where can I find good criticisms of EA made by non-EAs?
tl;dr, I think deference is more concerning for EA than other cultures. Relative to how much we should expect EAs to defer, they defer way too much.
1) We should expect EA to have much less deference culture than other cultures, since a lot of EA claims are based on things like answers to philosophical questions, long term future predictions, ect. These kinds of things are really hard to answer, and I don’t think it’s the case that most experts have a much better shot at answering these than some relatively smart and quantitative University students. Questions about moral philosophy are the exact kinds of questions you expect to have a super wide range of answers to, so the number of EAs that claim they’re longtermist is kind of surprising and unexpected. I think this is a sign there’s more deference than their should be.
On the other hand, for more concrete and established scientific fields where experts do have a much better chance at making decisions than students, it makes way more sense to defer to them about what things are important.
2) EAs are optimizing for altruism, so decisions on what to work on require lots of thought. I’m guessing most non-EA people choose to work on things they enjoy or are emotionally invested in.
I can easily tell you, without any evidence or deference, what things I thing are fun and am emotionally invested in. But it takes a lot more time and research to come up with what I think is the most impactful.
I think EAs having more evidence and reasoning to back up what we’re working on just naturally arises from being an EA, and doesn’t necessarily mean we have better epistemics than other communities.
3) Explicitly saying when you’re deferring to someone seems like it does a better job of convincing people “wow! these EA people seem more correct than most other communities” and does a worse job at actually being more correct than most other communities. Being explicit about when we defer to people still means we might defer way too much.
4) Edit: I think this point is not actually about deference. Also, I know very little about MIRI and have no idea if this is in any way realistic. I’m guessing you could replace MIRI with some other org and this kind of story would be true, but I’m not totally sure.
Also, idk I feel like some things that look like original, detailed thinking actually just ends up being closer to deference than I’d like. I think perhaps a story that’s happened before is “MIRI researcher thinks hard about AI stuff, and comes up with some original thoughts with lots of evidence. Writes on alignment form. Tons of karma, yay.”
Sure, the thinking is original, has evidence to back it up, and looks really nice, pretty, and useful. That being said, even if this is original thinking, I’m guessing if you looked at how this person was using the opinions of other people to shape their own opinions, it would look like
Talking to other MIRI people − 80%
Talking to non-MIRI EAs − 10%
Reading books/opinions written by non-EAs relevant to what they’re working on − 5%
Talking to non-EAs − 5%
So even if this thinking looks really original and intelligent, this still seems like a problem with deference. Not deferring to other MIRI researchers an unhealthy amount probably looks more like getting more insight from mainstream academia and non-EAs.
I guess the point here is that it’s much easier to look like you’re not deferring to people too much than to actually not defer to people too much.
5) I think people in general defer way too much and do not think hard enough about what to work on. I think EAs defer too much and occassionally don’t think hard enough about what to work on. Being better than the latter doesn’t really mean I’m satisfied with the former.
This post irked me because I think it’s wrong if you raise the bar for what “Ivy-smart” means. If it means something like “being good at an academic thing on a national (or even state) level” then I think most of these people are at Ivy League schools. For example, I know ~40 people who went to Olympiad training camps or were finalists in research contests, and pretty much all of them other than one or two are at top 10 schools. As another example, one competitive math research program list its results for alumni:
This research program I think is considered around as tough to get into as making the national math Olympiad.
I’m estimating there are maybe 5,000 − 10,000 people who would qualify as being good at something on the national level per four year groups (how I’m getting this estimate—good at national level = top 500 in something, there are maybe 10ish academic areas, and each of this has 1-2 ways to be good. For example, for sciences there are olympiads and research, for CS there’s CS research and building impressive projects, etc. ) Meanwhile, after a quick google search there are about 24,000 who would get a 150+ SAT score. It seems to me that there are a lot of people who are great at something on a national level that don’t know what EA is. Maybe enough that we should focus on Ivies significantly more than anywhere else?
As others have said, this really depends on what the context is. If we’re choosing how much money to spend on which University groups, and all University groups have around equal need for money, I’m guessing that the vast majority of the money should go towards Ivy University groups.
[Question] What are the best ways to encourage de-escalation in regards to Ukraine?
I interpreted this post as also complaining about there not being any sort of consesus among EAs as a community for how to deal with negative press. Most of what you listed would not qualify as a community-wide strategy. Sure, Open Phil, CEA, 80k, etc. having some PR guidance seems awesome, but what about everyone else? I’m guessing there are many orgs, individual people who have a lot of twitter followers or whatever, etc. that don’t know how to respond to negative press.
There’s a communications strategy being drafted.
seems to be the only thing you listed that actually addresses this. Which is great! But it seems pretty unrealistic to expect OP to know that this was happening. I’m also guessing this wouldn’t have been answered in an EA forum question; questions don’t seem to gain that much traction on EA forum, and the vast majority of users probably had no clue a communications strategy was being drafted.
Source is that I remember Ajeya mentioning at one point that it led to positive changes and she doesn’t think it was a bad decision in retrospect, but cannot get into said changes for NDA reasons.
I think it makes more sense to look at articles that Rachel has written about SBF/EA. Here’s one:
I (very briefly) skimmed it and didn’t see any major red flags.
AFAIK this is not something that can be shared publicly.
[Question] What highly-impactful work is the most similar to solving fun math puzzles (if any)?
A few more thoughts:
(EDIT: Seems like in an above comment that current culture-steering is trying to encourage perspective and stuff, so what I’m irked about here isn’t really an issue). Limiting EA globals to only people that fit into certain cultures could severely limit perspective/diversity, and in bad cases could make EAs as “cult”-y as some people want to believe it is. I think this is probably only true for some misguided attempts to steer culture, and in fact you can steer culture through admissions to encourage things like diversity/perspective. Selecting attendees based on “people Oliver Habryka would hang out with outside the conference” does seem to be sort of narrow, but I’m guessing this wasn’t what you actually want admissions to look like?
In respone to “I would like to increase the number of people who don’t need to apply to attend because they are part of some group or have some obvious signal that means they should pass the bar”: I guess it depends what you mean here, but I agree with Zach in that it doesn’t seem obvious why this would actually help (although I’d be intersted in the specific details of what you mean). I mean, some ways of doing this could make people upset, since the outside perspective could be something like certain “inner circles” of EA have easy backdoor access to EAGs. This could also make even very impressive/high-impact EAs feel excluded if they’re not a part of said groups. Seems to me like there are tons of ways this could go wrong.
I strong upvoted since this makes sense to me, but is EA global admissions actually making an effort to steer culture in any way? I was under the impression that EAG admissions was mostly just based on engagement with EA and how impressive someone is, which doesn’t really seem to qualify as steering culture. This certainly seems to line up with some things said in this post about people being rejected for not being impressive.
Not an answer, but why are you trying to do this? If you’re excited about Biology, there seem to be plenty of ways to do impactful biological work.
Even if you’re purely trying to maximize your impact, for areas like AI Alignment, climate change, or bioweapons, the relevant question is something like: what is the probability that me working on this area prevents a catastrophic event? According to utilitarianism, your # of lives saved is basically this time the total number of people that will ever live or something like this.
So if there’s a 10% chance of AI killing anyone, and you working on this brings it down to 9.999999%, this is less impactful than if there’s a 0.5 % chance of climate change killing everyone, and you working on this brings it down to 0.499 %. Since it’s much more likely to do impactful work in an area that excites you, seems like bio is solid, since it’s relevant to bioweapons and climate change?
I’m relatively unconvinced by most arguments I’ve read that claim deceptive alignment will be a thing (which I understand to be a model that intentionally behaves differently on its training data and test data to avoid changing its parameters in training).
Most toy examples I’ve seen, or thought experiments, don’t really seem to actually be examples of deceptive alignment since the model is actually trained on the “test” data in these examples. For example, while humans can decieve their teachers in etiquette school then use poor manners outside the school, you modify neurons in your brain on both using manners inside etiquette school and using them outside of etiquette school, so it makes sense that you would distinguish them.
I certainly think it is possible for a model to be deceptive, but this seems to be much, much more complicated and harder-to-find in the gradient descent landscape, and it seems to me that there’s pretty much no reason for a model to learn to be deceptive in the first place. This makes it seem like it won’t come up at all in practice, or at the least will be very easy to avoid.
Why do people buy deceptive alignment? Could you give some concrete examples where it could come up?
Thanks! Makes sense.
I’m not sure I follow why conservative criticism would lead to expanding the movement to a less represented audience
Hm, fairly confused by the downvotes. I’m guessing a) people disagree with this being a good decision or b) there’s a really obvious answer? If it’s b), can you please tell me?
While I agree that these kinds of “bad EA optics” posts are generally unproductive and it makes sense for them to get downvoted, I’m surprised that this specific one isn’t getting more upvoted? Unlike most links to hit pieces and criticisms of EA, this post actually contains new information that has changed my perception of EA and EA leadership.
Why? If I was making a decision to be interviewed by Rachel or not, probably the top thing I’d be worried about is whether they’ve previously written not-very-journalistic hit pieces on tech-y people (which is not all critical pieces in general! some are pretty good and well researched). I agree that there’s such thing as going too far, but I don’t think my comment was doing that.
I think “there are situations this is valid (but not for the WSJ!)” is wrong? There have been tons of examples of kind of crap articles in usually highly credible newspapers. For example, this article in the NYT seemed to be pretty wrong and not that good.