sapphire

Karma: 1,242

sapphire 1 Mar 2023 12:04 UTC
46 points
15 ∶ 1
on: Counterproductive Altruism: The Other Heavy Tail
Less theoretical example: FWIW im not sold on ‘more than anyone’ but the top 2-3 current AI labs are all downstream of AI safety!

sapphire 14 Jan 2023 12:51 UTC
45 points
27 ∶ 14
on: Thread for discussing Bostrom’s email and apology
(reposting from the ‘does it matter’ thread)
It is commonly theorized that having friends who hold a viewpoint should make one more charitable to that viewpoint. This has not been the case with HBD. I have a close friend of around decade who has gotten increasingly obsessed with HBD. In general they are a smart and friendly person. But the things they has started espousing have become really shocking.
Example of their views: If Black people are not heavily under-represented in a cognitively demanding organization that is strong evidence the organization is racist against White and Asian individuals!
Obviously these points of view are completely at odds with any sort of fair and inclusive community or organization. They have also moved further and further rightwing. This resulted in a lot of personal problems when I came out as trans. They don’t ‘just’ have some abstract objections, they were quite toxic to an old and supportive friend when she was having a hard time. They explicitly admit that a huge driving force for them moving rightward in general is belief in HBD. The logic for why is not hard to see. If you believe in HBD you can start to feel ‘persecuted’ by people on the left or center-left. It’s easy to start sympathizing with the right and far right.

I’ve been in the rationalist community for over a decade and the EA community for a somewhat shorter period. I have seen tons of seemingly kind and reasonable friends become increasingly far-right after they got into HBD. Im honestly not surprised FLI was considering funding an explicit far-right nazi-adjacent group. The sympathies run deep. Neo-reaction has been close to the EA and rationalist communities for a very large fraction of our history.
It would be extremely hypocritical for me to hold people to views they no longer support. I endorsed HBD in 2015 and 2016. Like many rationalists I was introduced to HBD by reading Slatestarcodex. Promoting HBD in anyway, including privately exposing people to the ideas, is one of the biggest regrets of my life. It is a seriously harmful philosophy. Im very, very sorry for any negative impact my actions may have caused. For obvious reasons I have sympathy for people who have gotten into the racist pipeline. I honestly only got out because the right is so shitty to trans people and is pretty anti-vegan. Like many eggs I had a lot of trans friends. Independently I was quite convinced veganism was a positive lifestyle. HBD is a very harmful pseudo-science and it is totally unacceptable that people with power in Effective Altruism believe in it.
I truly hope the EA movement can move toward a better future free from this toxicity.
What links here?

sapphire 8 Mar 2023 11:33 UTC
43 points
15 ∶ 4
in reply to: CatGoddess’s comment on: Abuse in LessWrong and rationalist communities in Bloomberg News
With the exception of Brent, who is fully ostracized afaik, I think you seriously understate how much support these abusers still have. My model is sadly that a decent number of important rationalists and EAs just dont care that much about the sort of behavior in the article. CFAR investigated Brent and stood by him until there was public outcry! I will repost what Anna Salomon wrote a year ago, long after his misdeeds were well known. Lots of people have been updating TOWARD Vassar:
I hereby apologize for the role I played in X’s ostracism from the community, which AFAICT was both unjust and harmful to both the community and X. There’s more to say here, and I don’t yet know how to say it well. But the shortest version is that in the years leading up to my original comment X was criticizing me and many in the rationality and EA communities intensely, and, despite our alleged desire to aspire to rationality, I and I think many others did not like having our political foundations criticized/eroded, nor did I and I think various others like having the story I told myself to keep stably “doing my work” criticized/eroded. This, despite the fact that attempting to share reasoning and disagreements is in fact a furthering of our alleged goals and our alleged culture. The specific voiced accusations about X were not “but he keeps criticizing us and hurting our feelings and/or our political support” — and nevertheless I’m sure this was part of what led to me making the comment I made above (though it was not my conscious reason), and I’m sure it led to some of the rest of the ostracism he experienced as well. This isn’t the whole of the story, but it ought to have been disclosed clearly in the same way that conflicts of interest ought to be disclosed clearly. And, separately but relatedly, it is my current view that it would be all things considered much better to have X around talking to people in these communities, though this will bring friction.
There’s broader context I don’t know how to discuss well, which I’ll at least discuss poorly:
Should the aspiring rationality community, or any community, attempt to protect its adult members from misleading reasoning, allegedly manipulative conversational tactics, etc., via cautioning them not to talk to some people? My view at the time of my original (Feb 2019) comment was “yes”. My current view is more or less “heck no!”; protecting people from allegedly manipulative tactics, or allegedly misleading arguments, is good — but it should be done via sharing additional info, not via discouraging people from encountering info/conversations. The reason is that more info tends to be broadly helpful (and this is a relatively fool-resistant heuristic even if implemented by people who are deluded in various ways), and trusting who can figure out who ought to restrict their info-intake how seems like a doomed endeavor (and does not degrade gracefully with deludedness/corruption in the leadership). (Watching the CDC on covid helped drive this home for me. Belatedly noticing how much something-like-doublethink I had in my original beliefs about X and related matters also helped drive this home for me.)
Should some organizations/people within the rationality and EA communities create simplified narratives that allow many people to pull in the same direction, to feel good about each others’ donations to the same organizations, etc.? My view at the time of my original (Feb 2019) comment was “yes”; my current view is “no — and especially not via implicit or explicit pressures to restrict information-flow.” Reasons for updates same as above.
It is nevertheless the case that X has had a tendency to e.g. yell rather more than I would like. For an aspiring rationality community’s general “who is worth ever talking to?” list, this ought to matter much less than the above. Insofar as a given person is trying to create contexts where people reliably don’t yell or something, they’ll want to do whatever they want to do; but insofar as we’re creating a community-wide include/exclude list (as in e.g. this comment on whether to let X speak at SSC meetups), it is my opinion that X ought to be on the “include” list.
Thoughts/comments welcome, and probably helpful for getting to shared accurate pictures about any of what’s above.

sapphire 3 Apr 2024 8:41 UTC
42 points
19 ∶ 6
in reply to: RobBensinger’s comment on: Quick Update on Leaving the Board of EV
Not to state the obvious but the ‘criticism of EA’ posts didn’t pose a real risk to the power structure. It is uhhhhh quite common for ‘criticism’ to be a lot more encouraged/tolerated when it isnt threatening.

sapphire 20 Feb 2023 22:44 UTC
41 points
39 ∶ 10
in reply to: JoshuaBlake’s comment on: EV UK board statement on Owen’s resignation
The fact this is true, despite issues being reported to the community health team, is a serious indictment.

sapphire 9 Feb 2023 0:03 UTC
37 points
13 ∶ 3
in reply to: Lizka’s comment on: Solidarity for those Rejected from EA Global
FWIW core EAs have openly said a major reason to keep EAG small is the ‘quality of conversation’ at the event. This is a big reason they made EAG smaller again. So there is definitely a level of judgment going on.

sapphire 21 Nov 2018 17:40 UTC
37 points
0 ∶ 1
on: Towards Better EA Career Advice
80K Hour’s advice seems aimed, perhaps implicitly, at extremely talented people. I would roughly describe the level of success/talent as ‘top half of Oxford’. If you do not have that level of ability, then the recommend career paths are going to be long shots at best. Most people are not realistically capable of getting a job at Jane Street (I am certainly not). It is also very hard to get a job at a well regarded EA organization.
Unless someone has a very good track record of success I would advise them not to follow 80K style advice. Trying to get a ‘high impact job’ has lead to failure for every rationalist I know who was not ‘top half of Oxford’ talented. In some cases they made it to ‘work sample’ got an internship, but they still failed to land a job. Many of these rationalists are well regarded and considered quite intelligent. These people are fairly talented and in many cases make low six figures.
80K is very depressing to read. Making ‘only’ 200K and donating 60K a year is implicitly treated like a failure. We at least need advise for people who are ‘only’ Google-programmer levels of talented. And ideally we need advice for EAs of all skill levels. But the fact that our standard advice is not even applicable to ‘normal Google programmer’ levels of talent is extremely depressing.
What links here?
- John_Maxwell's comment on Can the EA community copy Teach for America? (Looking for Task Y) by alex lawsen (previously alexrjl) (22 Feb 2019 6:29 UTC; 11 points)

sapphire 25 Dec 2022 10:32 UTC
34 points
1 ∶ 0
on: May The Factory Farms Burn
And will we bring all creatures in,
Circle, grow and grow.
feather, fur, or silicon?
Circle, grow and grow.
Though their unseen thought beguile —
strange the substrate they employ —
all who suffer or enjoy
are brother soul, in body wild.

Circle, circle, grow and grow.

And will we bring the future in?
Circle, grow and grow.
All of time is ours to win!
Circle, grow and grow.
Will our children rise in power?
overwhelm the starry deep?
Lights unborn, for you we keep
will and hope, though dark the hour.

Circle, circle, grow and grow.
“Proper?” asked the Comet King. “I come to you with a plan to fight off Hell and save the world, and you tell me it isn’t proper?”
Vihaan stared at the priest, as if begging him to step in. “I swear,” said Father Ellis, “it’s like explaining the nature of virtue to a rock”.
“Do you know,” interrupted Jalaketu, “that whenever it’s quiet, and I listen hard, I can hear them? The screams of everybody suffering. In Hell, around the world, anywhere. I think it is a power of the angels which I inherited from my father.” He spoke calmly, without emotion. “I think I can hear them right now.”
Ellis’ eyes opened wide. “Really?” he asked. “I’m sorry. I didn’t…”
“No,” said the Comet King. “Not really.”
They looked at him, confused.
“No, I do not really hear the screams of everyone suffering in Hell. But I thought to myself, ‘I suppose if I tell them now that I have the magic power to hear the screams of the suffering in Hell, then they will go quiet, and become sympathetic, and act as if that changes something.’ Even though it changes nothing. Who cares if you can hear the screams, as long as you know that they are there? So maybe what I said was not fully wrong. Maybe it is a magic power granted only to the Comet King. Not the power to hear the screams. But the power not to have to.”
Merry Christmas. May hell be destroyed.

sapphire 3 Feb 2023 15:23 UTC
25 points
40 ∶ 44
on: EA, Sexual Harassment, and Abuse
This is what happens when you centralize power so much. I’m so sorry for what happened. So many people remaining silent and covering for abusers.

(it shouldn’t matter but for the record I have multiple partners)

sapphire 6 Mar 2019 19:52 UTC
25 points
0 ∶ 0
in reply to: Raemon’s comment on: What to do with people?
I think we are pretty far from exhausting all the good giving oppurtunities. And even if all the highly effective charities are filled something like Give Directly can be scaled up. It is possible in the future we will eventually get to the point where there are so few people in poverty that cash transfers are ineffective. But if that happens there is nothing to be sad about. the marignal value of donations will go down as more money flows into EA. That is an argument for giving more now. A future where marginal EA donaions are ineffective is a very good future.

sapphire 4 Jul 2022 20:06 UTC
24 points
0 ∶ 0
in reply to: John G. Halstead’s comment on: The Future Might Not Be So Great.
In most cases where I am actually familiar with the facts CEA has behaved very poorly. They have both been way too harsh on good actors and failed to take sufficient action against bad actors (ex Kathy Forth). They did handle some very obvious cases reasonably though (Diego). I don’t claim I would do a way better job but I don’t trust CEA to make these judgments.

sapphire 8 Jun 2022 15:43 UTC
23 points
0 ∶ 0
in reply to: MichaelStJules’s comment on: Transcript of Twitter Discussion on EA from June 2022
This level of support for centralization and deferral is really unusual. I actually don’t know of any community besides EA that endorses it. I’m aware it’s a common position in effective altruism. But the arguments for it haven’t been worked out in detail anywhere I know.

“Keep in mind that many things you might want to fund are in scope of an existing fund, including even small grants for things like laptops. You can just recommend they apply to these funds. If they don’t get any money, I’d guess there were better options you would have missed but should have funded first. You may also be unaware of ways it would backfire, and the reason something doesn’t get funded is because others judge it to be net negative.”
I genuinely don’t think there is any evidence (besides some theory-crafting around unilateralists curse) to think this level of second-guessing yourself and deferring is effective. Please keep in mind the history of the EA funds. Several funds basically never dispersed the funds. And the fund managers explicitly said they didn’t have time. Of course things can improve but this level of deferral is really extreme given the communities history.
Suffice to day I don’t think further centralizing resources is good nor is making things more bureaucratic. Im also not sure there is actually very much risk of ‘unilateralist curse’ unless you are being extremely careless. I trust most EAs to be at least as careful as the leadership. Probably the most dangerous thing you could possible fund is AI capabilities. Openphil gave 30M to OpenAI and the community has been pretty accepting of ai capabilities. This is way more dangerous than anything I would consider funding!
What links here?
- Charles He's comment on The Strange Shortage of Moral Optimizers by Richard Y Chappell (9 Jun 2022 15:32 UTC; 2 points)

sapphire 21 Feb 2023 0:09 UTC
22 points
20 ∶ 6
in reply to: Robi Rahman’s comment on: A statement and an apology
I think thats the public image but isn’t how things actually work internally. Id really recommend reading this comment by Buck about how “You’ve also made the (IMO broadly correct) point that a lot of EA organizations are led and influenced by a pretty tightly knit group of people who consider themselves allies”. Notably the post is pretty explicit that any proposed changes should be geared toward getting this small group onboard.

It is less public (at this point) but some of the core EAs have definitely been capricious in terms of who they want to receive any kind of support or endorsement. And they feel quite willing to do this without any community buy in.

sapphire 5 Jul 2022 17:11 UTC
21 points
0 ∶ 0
on: The Future Might Not Be So Great.
I find the following simple argument disturbing:

P1 - Currently, and historically, low power being (animals, children, old dying people) are treated very cruelly if treating them cruelly benefits the powerful even in minor ways. Weak benefits for the powerful empirically justify cruelty at scale.
P2 - There is no good reason to be sure the powerful wont have even minor reasons to be cruel to the powerless (ex: suffering sub-routines, human CEV might include spreading earth like life widely or respect for tradition)
P3 - Inequality between agents is likely to become much more extreme as AI develops
P4 - The scale of potenital suffering will increase by many orders of magnitude

C1 - We are fucked?
Personal Note—There is also no reason to assume me or my loved ones will remain relatively powerful beings
C2 - Im really fucked!

sapphire 30 Nov 2018 0:30 UTC
21 points
0 ∶ 0
in reply to: Raemon’s comment on: Earning to Save
I feel like this post illustrates large inferential gaps. In my experience trying to work in EA works for a rather small number of people. I certainly don’t recommend it. Let me quote something I posted on the 80K hours thread:
80K Hour’s advice seems aimed, perhaps implicitly, at extremely talented people. I >would roughly describe the level of success/talent as ‘top half of Oxford’. If you do >not have that level of ability, then the recommend career paths are going to be long >shots at best. Most people are not realistically capable of getting a job at Jane Street > (I am certainly not). It is also very hard to get a job at a well regarded EA organization.
Unless someone has a very good track record of success I would advise them not to > follow 80K style advice. Trying to get a ‘high impact job’ has lead to failure for every > rationalist I know who was not ‘top half of Oxford’ talented. In some cases they made > it to ‘work sample’ got an internship, but they still failed to land a job. Many of these > rationalists are well regarded and considered quite intelligent. These people are fairly > talented and in many cases make low six figures.
80K is very depressing to read. Making ‘only’ 200K and donating 60K a year is implicitly treated like a failure. We at least need advise for people who are ‘only’ Google-programmer levels of talented. And ideally we need advice for EAs of all skill levels. But the fact that our standard advice is not even applicable to ‘normal Google programmer’ levels of talent is extremely depressing.
Maybe there are talent constraints but they don’t seem to me like talent constraints that are satisfied by pushing more EAs into trying to work in EA. I think that mostly works if you are unusually talented or extremely dedicated and ‘agenty’. I do think you can probably find a way to work on an EA cause if you are willing to accept low wages and hustle.
EA is really not set up to handle an influx of people trying to work in the field. Maybe this is a crux?

sapphire 29 Nov 2018 13:58 UTC
21 points
0 ∶ 0
in reply to: mingyuan’s comment on: Earning to Save
I feel like you are generalizing from a small sample of very dedicated EAs. In my opinion the data does not support ‘EAs have often prioritized giving 10% and living frugally *too* heavily’. See data here: https://forum.effectivealtruism.org/posts/S2ypk8fsHFrQopvyo/ea-survey-2017-series-donation-data.
The median donation percentage among EAs who reported 10K+ income was only 4.28%. The following example you give is not typical ‘For example, I’ve heard from some of the early Australian EAs that when EA was just starting out they all lived illegally in a hallway and ate out of the garbage. That was probably not good for their productivity or their physical or mental health.’.
My posts invovled a specific salary number for NYC. And the claim you quote gives a specific condition ‘make at least as much as median local household income’. Conversely Ray’s post has no numbers in it at all. You will also notice in the concrete budget I posted I included 200 dollars/week in consumption. My stated advice does not support living off garbage.
I can support more nuanced advice that tells young or very dedicated EAs not to harm themselves in order to donate 10%. But I think most EAs should actually donate more. So I am pretty skeptical of advice that suggests donating less unless it comes with appropriate concrete caveats. And I really do think the caveats need to be concrete. Its very easy to implicitly treat luxuries as nescessary. Several people I talked to seemed skeptical it was possible to find rent in NYC for 1K (I was able to quickly point out places).

sapphire 11 Jan 2023 2:48 UTC
20 points
6 ∶ 1
on: AGI and the EMH: markets are not expecting aligned or unaligned AI in the next 30 years
Id suggest investing in companies correlated with AI progress not shorting bonds. Some of the best investments are private but you can still buy stocks like MSFT, TSM, Samsung and ASML. This seems like a much better way to bet on AI progress.
Also if you expect aligned AI you should probably consume much less? Inequality might get locked in and returns to capital might skyrocket. Bad time to be a big spender. If we get unaligned AI and the world ends or aligned AI paradise so be it.

sapphire 16 May 2021 6:18 UTC
18 points
0 ∶ 0
in reply to: SammyDMartin’s comment on: Vitalik Buterin just donated $54M (in ETH) to GiveWell
Multiple people connected to the lesswrong/ea investing groups tried to contact him. We both contacted him directly and got some people closer to Vitalik to talk to him. I am unsure how much influence we had. He donated less than two days after the facebook threads went up.

We definitely tried!

sapphire 9 May 2021 1:09 UTC
18 points
0 ∶ 0
on: Should you do a PhD in science?
It is hard for me to think of much advice that has gone worse for rationalists/EAs on average than ‘Get a PHD’. I know dozens of people in the community who spent at least some time in a PHD program. There is a huge amount of people expressing strong regret. A small number of people think their PHD went ok. Very few people think their PHD was a great and effective idea. Notably, I am only counting people who have already left grad school and had some time to reflect.
The track record is incredibly bad in the community. The opportunity cost is extremely high. I very strongly urge people to reconsider.
Another angle is that it is just a very unhealthy environment on average. Here is Ben Kuhn explaining the data:
First I looked for a bigger survey of graduate student depression and anxiety rates. It wasn’t too hard to find one, and the numbers were almost the same: 41% of graduate students had “moderate to severe” anxiety compared to 6% of the general population; 39% had moderate to severe depression compared to 6% of the general public.

sapphire 26 Jan 2019 20:46 UTC
18 points
0 ∶ 0
on: Simultaneous Shortage and Oversupply
At least some people at OpenAI are making a ton of money: https://www.nytimes.com/2018/04/19/technology/artificial-intelligence-salaries-openai.html /. Of course not everyone is making that much but I doubt salaries at OpenAI/DeepMind are low. I think the obvious explanation is the best one. These companies want to hire top talent. Top talent is hard to find.
The situation is different for organizations that cannot afford high salaries. Let me link to Nate’s explanation from three years ago:
I want to push back a bit against point #1 ("Let's divide problems into 'funding constrained' and 'talent constrained'.) In my experience recruiting for MIRI, these constraints are tightly intertwined. To hire talent, you need money (and to get money, you often need results, which requires talent). I think the "are they funding constrained or talent constrained?" model is incorrect, and potentially harmful. In the case of MIRI, imagine we're trying to hire a world-class researcher for $50k/year, and can't find one. Are we talent constrained, or funding constrained? (Our actual researcher salaries are higher than this, but they weren't last year, and they still aren't anywhere near competitive with industry rates.)
Furthermore, there are all sorts of things I could be doing to loosen the talent bottleneck, but only if I knew the money was going to be there. I could be setting up a researcher stewardship program, having seminars run at Berkeley and Stanford, and hiring dedicated recruiting-focused researchers who know the technical work very well and spend a lot of time practicing getting people excited -- but I can only do this if I know we're going to have the money to sustain that program alongside our core research team, and if I know we're going to have the money to make hires. If we reliably bring in only enough funding to sustain modest growth, I'm going to have a very hard time breaking the talent constraint.
And that's ignoring the opportunity costs of being under-funded, which I think are substantial. For example, at MIRI there are numerous additional programs we could be setting up, such as a visiting professor + postdoc program, or a separate team that is dedicated to working closely with all the major industry leaders, or a dedicated team that's taking a different research approach, or any number of other projects that I'd be able to start if I knew the funding would appear. All those things would lead to new and different job openings, letting us draw from a wider pool of talented people (rather than the hyper-narrow pool we currently draw from), and so this too would loosen the talent constraint -- but again, only if the funding was there. Right now, we have more trouble finding top-notch math talent excited about our approach to technical AI alignment problems than we have raising money, but don't let this fool you -- the talent constraint would be much, much easier to address with more money, and there are many things we aren't doing (for lack of funding) that I think would be high impact.
source: https://forum.effectivealtruism.org/posts/k6bBgWFdHH5hgt9RF/peter-hurford-thinks-that-a-large-proportion-of-people#DvKfX3iN5Z8kuaFs7