I found Ezra’s grumpy complaints about EA amusing and useful. Maybe 80K should arrange to have more of their guests’ children get sick the day before they tape the interviews.
Buck
Karma: 6,860
I agree that we should tolerate people who are less well read than GPT-4 :P
For what it’s worth, gpt4 knows what rat means in this context: https://chat.openai.com/share/bc612fec-eeb8-455e-8893-aa91cc317f7d
I think this is a great question. My answers:
I think that some plausible alignment schemes seem like they could plausibly involve causing suffering to the AIs. I think that it seems pretty bad to inflict huge amounts of suffering on AIs, both because it’s unethical and because it seems potentially inadvisable to make AIs justifiably mad at us.
If unaligned AIs are morally valuable, then it’s less bad to get overthrown by them, and perhaps we should be aiming to produce successors who we’re happier to be overthrown by. See here for discussion. (Obviously the plan A is to align the AIs, but it seems good to know how important it is to succeed at this, and making unaligned but valuable successors seems like a not-totally-crazy plan B.)
My attitude, and the attitude of many of the alignment researchers I know, is that this problem seems really important and neglected, but we overall don’t want to stop working on alignment in order to work on this. If I spotted an opportunity for research on this that looked really surprisingly good (e.g. if I thought I’d be 10x my usual productivity when working on it, for some reason), I’d probably take it.
It’s plausible that I should spend a weekend sometime trying to really seriously consider what research opportunities are available in this space.
My guess is that a lot of the skills involved in doing a good job of this research are the same as the skills involved in doing good alignment research.
Thanks Lizka. I think you mean to link to this video:
Holden’s beliefs on this topic have changed a lot since 2012. See here for more.
I really like this frame. I feel like EAs are somewhat too quick to roll over and accept attacks from dishonest bad actors who hate us for whatever unrelated reason.
Yes, I think this is very scary. I think this kind of risk is at least 10% as important as the AI takeover risks that I work on as an alignment researcher.
I don’t think Holden agrees with this as much as you might think. For example, he spent a lot of his time in the last year or two writing a blog.
I think it’s absurd to say that it’s inappropriate for EAs to comment on their opinions on the relative altruistic impact of different actions one might take. Figuring out the relative altruistic impact of different actions is arguably the whole point of EA; it’s hard to think of something that’s more obviously on topic.
Obviously it would have been better if those organizers had planned better. It’s not clear to me that it would have been better for the event to just go down in flames; OP apparently agreed with me, which is why they stepped in with more funding.
I don’t think the Future Forum organizers have particularly strong relationships with OP.
The main bottleneck I’m thinking of is energetic people with good judgement to execute on and manage these projects.
I disagree, I think that making controversial posts under your real name can improve your reputation in the EA community in ways that help your ability to do good. For example, I think I’ve personally benefited a lot from saying things that were controversial under my real name over the years (including before I worked at EA orgs).
Stand up a meta organization for neartermism now, and start moving functions over as it is ready.
As I’ve said before, I agree with you that this looks like a pretty good idea from a neartermist perspective.
Neartermism has developed meta organizations from scratch before, of course.
[...]
which is quite a bit more than neartermism had when it created most of the current meta.
I don’t think it’s fair to describe the current meta orgs as being created by neartermists and therefore argue that new orgs could be created by neartermists. These were created by people who were compelled by the fundamental arguments for EA (e.g. the importance of cause prioritization, cosmopolitanism, etc). New meta orgs would have to be created by people who are compelled by these arguments but also not compelled by the current arguments for longtermism, which is empirically a small fraction of the most energetic/ambitious/competent people who are compelled by arguments for the other core EA ideas.
More importantly, meta orgs that were distanced from the longtermist branch would likely attract people interested in working in GHD, animal advocacy, etc. who wouldn’t currently be interested in affiliating with EA as a whole. So you’d get some experienced hands and a good number of new recruits
I think this is the strongest argument for why neartermism wouldn’t be substantially weaker without longtermists subsidizing its infrastructure.
Two general points:
There are many neartermists who I deeply respect; for example, I feel deep gratitude to Lewis Bollard from the Open Phil farmed animal welfare team and many other farmed animal welfare people. Also, I think GiveWell seems like a competent org that I expect to keep running competently.
It makes me feel sad to imagine neartermists not wanting to associate with longtermists. I personally feel like I am fundamentally an EA, but I’m only contingently a longtermist. If I didn’t believe I could influence the long run future, I’d probably be working on animal welfare; if I didn’t believe that there were good opportunities there, I’d be working hard to improve the welfare of current humans. If I believed it was the best thing to do, I would totally be living frugally and working hard to EtG for global poverty charities. I think of neartermist EAs as being fellow travelers and kindred spirits, with much more in common with me than almost all other humans.
Fwiw my guess is that longtermism hasn’t had net negative impact by its own standards. I don’t think negative effects from AI speed up outweigh various positive impacts (e.g. promotion of alignment concerns, setting up alignment research, and non-AI stuff).
and then explains why these longtermists will not be receptive to conventional EA arguments.
I don’t agree with this summary of my comment btw. I think the longtermists I’m talking about are receptive to arguments phrased in terms of the classic EA concepts (arguments in those terms are how most of us ended up working on the things we work on).
Holden Karnofsky on evaluating people based on public discourse:
I think it’s good and important to form views about people’s strengths, weaknesses, values and character. However, I am generally against forming negative views of people (on any of these dimensions) based on seemingly incorrect, poorly reasoned, or seemingly bad-values-driven public statements. When a public statement is not misleading or tangibly harmful, I generally am open to treating it as a positive update on the person making the statement, but not to treating it as worse news about them than if they had simply said nothing.
The basic reasons for this attitude are:
I think it is very easy to be wrong about the implications of someone’s public statement. It could be that their statement was poorly expressed, or aimed at another audience; that the reader is failing to understand subtleties of it; or that the statement is in fact wrong, but that it merely reflects that the person who made it hasn’t been sufficiently reflective or knowledgeable on the topic yet (and could become so later).
I think public discourse would be less costly and more productive for everyone if the attitude I take were more common. I think that one of the best ways to learn is to share one’s impressions, even (especially) when they might be badly wrong. I wish that public discourse could include more low-caution exploration, without the risks that currently come with such things.
I generally believe in evaluating people based on what they’ve accomplished and what they’ve had the opportunity to accomplish, plus any tangible harm (including misinformation) they’ve caused. I think this approach works well for identifying people who are promising and people whom I should steer clear of; I think other methods add little of value and mostly add noise.
I update negatively on people who mislead (including expressing great confidence while being wrong, and especially including avoidable mischaracterizations of others’ views); people who do tangible damage (usually by misleading); and people who create little of value despite large amounts of opportunity and time investment. But if someone is simply expressing a view and being open about their reasons for holding it, I try (largely successfully, I think) not to make any negative updates simply based on the substance.
FWIW I’m somewhat more judgemental than Holden, but I think the position Holden advocates is not that unusual for seniorish EAs.
I think you’re imagining that the longtermists split off and then EA is basically as it is now, but without longtermism. But I don’t think that’s what would happen. If longtermist EAs who currently work on EA-branded projects decided to instead work on projects with different branding (which will plausibly happen; I think longtermists have been increasingly experimenting with non-EA branding for new projects over the last year or two, and this will probably accelerate given the last few months), EA would lose most of the people who contribute to its infrastructure and movement building.
My guess is that this new neartermist-only EA would not have the resources to do a bunch of things which EA currently does—it’s not clear to me that it would have an actively maintained custom forum, or EAGs, or EA Funds. James Snowden at Open Phil recently started working on grantmaking for neartermist-focused EA community growth, and so there would be at least one dedicated grantmaker trying to make some of this stuff happen. But most of the infrastructure would be gone.
I agree that longtermism’s association with EA has some costs for neartermist goals, but it’s really not clear to me that the association is net negative for neartermism overall. Perhaps we’ll find out.
(I personally like the core EA ideas, and I have learned a lot from engaging with non-longtermist EA over the last decade, and I feel great fondness towards some neartermist work, and so from a personal perspective I like the way things felt a year ago better than a future where more of my peers are just motivated by “holy shit, x-risk” or similar. But obviously we should make these decisions to maximize impact rather than to maximize how much we enjoy our social scenes.)
I think it was unhelpful to refer to “Harry Potter fanfiction” here instead of perhaps “a piece of fiction”—I don’t think it’s actually more implausible that a fanfic would be valuable to read than some other kind of fiction, and your comment ended up seeming to me like it was trying to use the dishonest rhetorical strategy of implying without argument that the work is less likely to be valuable to read because it’s a fanfic.