Sharmake

Karma: 892

Sharmake 11 Nov 2022 15:09 UTC
141 points
38 ∶ 8
on: The FTX Future Fund team has resigned
What do EA and the FTX Future Team think of a claim by Kerry Vaughan that Sam Bankman-Fried did severely unethical behavior before and EA and FTX covered it up and laundered his reputation, effectively getting away with it.

I’m posting because of true, this suggests big changes to EA norms are necessary to deal with bad actors like him, and that Sam Bankman-Fried should be outright banned from the forum and EA events.

Link to tweets here:

https://twitter.com/KerryLVaughan/status/1590807597011333120
What links here?
- Doing EA Better by ConcernedEAs (17 Jan 2023 20:09 UTC; 258 points)
- Greg_Colbourn's comment on Why you’re not hearing as much from EA orgs as you’d like by Shakeel Hashim (16 Nov 2022 8:18 UTC; 15 points)

Sharmake 11 Nov 2022 15:11 UTC
54 points
14 ∶ 3
on: Some comments on recent FTX-related events
Question for Holden Karnofsky:

What do EA and Holden Karnofsky think of a claim by Kerry Vaughan that Sam Bankman-Fried did severely unethical behavior before and EA and FTX covered it up and laundered his reputation, effectively getting away with it.

I’m posting because of true, this suggests big changes to EA norms are necessary to deal with bad actors like him, and that Sam Bankman-Fried should be outright banned from the forum and EA events.

Link to tweets here:

https://twitter.com/KerryLVaughan/status/1590807597011333120

Sharmake 15 Dec 2023 19:34 UTC
42 points
15 ∶ 2
in reply to: DPiepgrass’s comment on: Nonlinear’s Evidence: Debunking False and Misleading Claims
Yeah, at least several comments have much more severe issues than tone or stylistic choices, like rewording ~every claim by Ben, Chloe and Alice, and then assuming that the transformed claims had the same truth value as the original claim.

I’m in a position very similar to Yarrow here: While I think Kat Woods has mostly convinced me that the most incendiary claims are likely false, and I’m sympathetic to the case for suing Ben and Habryka, there was dangerous red flags in the responses, so much so that I’d stop funding Nonlinear entirely, and I think it’s quite bad that Kat Woods responded the way they did.

Sharmake 12 Nov 2022 15:17 UTC
30 points
11 ∶ 0
on: CEA/EV + OP + RP should engage an independent investigator to determine whether key figures in EA knew about the (likely) fraud at FTX
This is even more important than people think, since this may not be the first time EA had questionable relationships with millionaires/billionaires. I consider Kerry Vaughan’s tweets about it plausible because this is a common theme of failure: small failures are normalized or rationalized, this allowing bigger failures to happen, like this fraud.

Here’s links to normalization of deviance and Kerry Vaughan’s tweets:

https://danluu.com/wat/

https://twitter.com/KerryLVaughan/status/1590807597011333120

Sharmake 12 Jun 2023 13:32 UTC
19 points
7 ∶ 1
in reply to: Larks’s comment on: Critiques of prominent AI safety labs: Conjecture
I actually would find this at least somewhat concerning, because selection bias/selection effects are my biggest worry with smart people working in an area. If a study area is selected based upon any non-truthseeking motivations, or if people are pressured to go along with a view for non-truthseeking reasons, then it’s very easy to land into nonsense, where the consensus is based totally on selection effects, making them useless to us.

There’s a link to the comment by lukeprog below on the worst case scenario for smart people being dominated by selection effects:

One marker to watch out for is a kind of selection effect.

In some fields, only ‘true believers’ have any motivation to spend their entire careers studying the subject in the first place, and so the ‘mainstream’ in that field is absolutely nutty.

Case examples include philosophy of religion, New Testament studies, Historical Jesus studies, and Quranic studies. These fields differ from, say, cryptozoology in that the biggest names in the field, and the biggest papers, are published by very smart people in leading journals and look all very normal and impressive but those entire fields are so incredibly screwed by the selection effect that it’s only “radicals” who say things like, “Um, you realize that the ‘gospel of Mark’ is written in the genre of fiction, right?”

https://www.lesswrong.com/posts/fyZBtNB3Ki3fM4a6Y/some-heuristics-for-evaluating-the-soundness-of-the-academic#scZakQrAYm2Mck9QQ

Sharmake 15 Jan 2023 14:35 UTC
18 points
6 ∶ 1
in reply to: Devin Kalish’s comment on: CEA statement on Nick Bostrom’s email
I think what Habryka is saying is that while EA does have some notion of equality, the reason it sticks so close to mainstream egalitarianism is because humans don’t differ much. If there were multiple species civilizations like those in Orion’s Arm for example, where multiple orders of magnitude differences in abilities are present, than a lot of stratification and non-egalitarianism will happen solely by the value of freedom/empowerment.

And this poses a real moral dilemma for EA, primarily because of impossibility results around fairness/egalitarianism.

Sharmake 11 Nov 2022 17:50 UTC
17 points
8 ∶ 0
in reply to: Arepo’s comment on: The FTX Future Fund team has resigned
I find myself having a mixed opinion of how EA responded. It wasn’t outright terrible epistemics, unlike most of the world reacting to a similar event, but there were real failures of epistemics.

On the other hand, there was also successes in EA epistemics, as well.

Sharmake 9 Nov 2022 21:48 UTC
17 points
3 ∶ 0
in reply to: richard_ngo’s comment on: FTX will likely go bankrupt. What we know and some forecasts on what will happen next
Question, but why is it bad to wish well on people impacted by this? I get it, there’s a whole lot of rumors about fraud, but why is even the very basic norm of wishing well viewed as a bad thing?

Sharmake 16 Jan 2023 20:01 UTC
15 points
5 ∶ 0
on: Replace Neglectedness
Basically the problem with neglectedness is that it assumes strictly declining returns like a logarithmic scale. But if the problem has the quality “the more the merrier”, or increasing returns to scale, then neglectedness is a problem. In other words, leverage matters.

Sharmake 11 Nov 2022 16:43 UTC
15 points
1 ∶ 0
in reply to: Zach Furman’s comment on: The FTX Future Fund team has resigned
Sorry for misfiring here, I’ll retract my comment.

Sharmake 16 Jan 2023 21:01 UTC
14 points
5 ∶ 0
in reply to: Wil Perkins’s comment on: Smuggled assumptions in “declining epistemic quality”

I don’t see it that way. Lots of relatively normal folks put money into FTX. Journalists and VCs were very positive about SBF/FTX.

This. I do think that blaming rationalist culture is mostly a distraction, primarily because way too much normie stuff promoted SBF.

I had a very different opinion of the whole crypto train (that is, crypto needs to at least stop having real money, if not flat out banned altogether.)

Yes, EA failed. But let’s be more careful about suggesting that normies didn’t fail here.

Sharmake 11 Aug 2022 22:02 UTC
14 points
0 ∶ 0
on: Animal rights initiative with far-reaching consequences and possibly high chance of success
Important, potentially tractable and neglected? I’ve got to say this post makes a good case for supporting the referendum.

Sharmake 19 Nov 2023 2:54 UTC
13 points
11 ∶ 1
in reply to: JWS’s comment on: Sam Altman fired from OpenAI
Yeah, this is one of the few times where I believe that the EAs on the board likely overreached here, because they probably didn’t give enough evidence to justify their excoriating statement there that Sam Altman was dishonest, and he might be coming back to lead the company.

I’m not sure how to react to all of this, though.

Edit: My reaction is just WTF happened, and why did they completely play themselves? Though honestly, I just believe that they were inexperienced.

Sharmake 13 Jan 2023 13:55 UTC
13 points
6 ∶ 8
in reply to: Chris Leong’s comment on: CEA statement on Nick Bostrom’s email
IMO, I actually agree with the point that high decouplers and low decouplers can’t live together very well. Pessimistically speaking, EA probably has to split because of incompatible cultures from another comment I read:

Hey, I’ve been a follower the community for years and your post made me want to create an account and leave a thought. Full disclosure: I’m trans FtM from a not-Western country, never been to an EA meetup but have always been curious. I’m listing these characteristic as I think they’re directly relevant to what I describe below. Also, I have massive Asperger’s (and my English is not perfect), so I want to explicitly note that I 100% value your post, support having a discussion about this, and have no great solutions. Flagging this because the tone of my text may not convey this otherwise.

Some thoughts, ordered from most to least banal:

Punishment for confirmed bad sexual behavior should be swift, harsh, and very likely. Great care must be exercised to make the abused come forward. Great care must be exercised to thoroughly vet accusations and protect the rights of the accused. #1 and #2 involve a false positive/negative tradeoff between each other, and I suspect a solution to satisfy a good chunk of people across many backgrounds is likely impossible. This is likely not news to anyone reading this, so I apologize for wasting your time so far. I think your post adeptly highlights the reason for why EA as a movement will almost inevitably (IMO) collapse or split. I want to elaborate on the last point most of all. As an observer, I always thought that core point of EA was based in recognizing the fallibility of human emotion in guiding us about priorities, as strong emotional states can cloud our thinking (AFAIK this is not very controversial and is supported by research). This is why Paul Bloom’s “Against Empathy” is referenced relatively frequently in the community, right? My not important personal opinion is that this is a partial picture, because emotion can also be used as a driver/inspiration to care about injustices. Of course the difficult question is whether you can get the best parts of emotion into the movement without getting the worst of emotion as well.

But there’s something broader that concerns me, and I think spells trouble for the EA movement as a whole. I will start from a story that has to do with my FtM self. When I started hormonal therapy I noticed that some of my basic intuitions started changing, the feelings and views I wasn’t even aware of existing. I kind of took them for granted, they were transparent to me. For example, before, I felt a lot of instinctive anger at injustice, and now I still feel it, but it’s more detached and principled. “Less emotional” doesn’t quite describe it, it’s something about the anger becoming more calculating. Like I can be a bit more removed and hypothetical about different problems; I could turn off my empathy a bit (which actually felt scary!) to be more analytical. And to repeat, I assumed my previous emotional responses to be same as everyone’s, it didn’t even occur to me things could be any other way (I’m sorry if this all sounds like pseudoscience).

Now a lot changed in me that could have resulted in the above, and even though other T people I know have on average followed a similar switch (depending on the direction), I didn’t think much of it. So it could be totally useless anecdotally, but it did get me reading on the “gender divide” in Western politics. And to simplify it greatly, it seems like there’s pretty convincing argument that the gradual “feminization” of Western cultures in the last several decades drives a lot of the conflict in politics and across institutions, being in a tension with the typical “masculine” culture. And this is rooted partly in that women and men, on average, have different views on things like speech/harm tradeoff: women tend to prefer to limit speech to minimize harm, whereas men lean in the other direction. (Please note that these are average, population-level statements, and the distributions of these traits do overlap a lot!)

These are some of the more notable readings I’ve read on this divide:

NYT coverage of the gender divide in American politics: https://archive.ph/1LAwU One interesting highlight: Male students preferred protecting free speech over an inclusive and diverse society by a decisive 61 to 39. Female students took the opposite position, favoring an inclusive, diverse society over free speech by 64 to 35.

A Quillette piece with a lot more stats showing patterns like in the piece above, broadly showing that women on average lean toward diversity/inclusion over speech/”truth-seeking” when faced with a choice, and mean lean in the other direction: https://quillette.com/2022/10/08/sex-and-the-academy/ A bunch of Marginal Revolution posts, highlighting what the authors consider to be both “good” and “bad” parts of “feminization”: https://marginalrevolution.com/marginalrevolution/2019/12/sex-differences-in-personality-are-large-and-important.html https://marginalrevolution.com/marginalrevolution/2021/09/a-simple-illustration-of-the-benefits-of-feminization.html https://marginalrevolution.com/marginalrevolution/2021/08/when-did-we-all-become-women.html https://marginalrevolution.com/marginalrevolution/2022/03/the-complexities-of-feminization.html https://marginalrevolution.com/marginalrevolution/2020/01/the-feminization-of-society-installment-1637.html https://marginalrevolution.com/marginalrevolution/2017/05/much-educational-political-polarization-due-feminization.html A very polemical, negative, and not particularly epistemically sound piece very critical of feminization: https://richardhanania.substack.com/p/womens-tears-win-in-the-marketplace (including it here solely for the value of showing a clear “anti” voice, but I’d read this much more critically than anything else in the list) A similarly negative piece, but from someone I’d consider of higher integrity: http://www.arnoldkling.com/blog/feminized-culture/ Okay, so what’s the point of my super-rambley comment? In short, whether my and my T friends’ anecdotal experiences are related to this or not, I think centuries of masculinity-type culture have not left us prepared to a world where culture becomes feminized. We are now entering this world, and have to deal with how to strike a balance between these sets of values. And the “Effective Altruism” movement has historically been (to my understanding) a movement that prioritizes asking difficult (even if very offensive) questions and analyzing costs/benefits in a way that increases accuracy (in part, by trying to be less emotional/swayed about the topic of the analysis). It seems to me that as designed, the movement either has to split into two, or just morph into a charity/NGO like many other.

Pessimistically, I do not think there’s a way to accommodate these sets of views. Having to some limited extent “experienced both,” I don’t even know how to feel about them myself, internally, and struggle with this daily. How can we expect an entire movement to align then? Perhaps if the leaders and big thinkers of the movement come together, recognize this as the urgent problem, and figure out a way to approach it, there is some sort of resolution (a Scott Alexander megapost/initiative to organize this?). But again, my prediction is that the unobstructed, calculated, detached, low-anger-type discourse would have to give way as EA becomes more inclusive and accommodating of feminized culture.

Sharmake 25 Dec 2022 16:15 UTC
13 points
5 ∶ 1
on: On applause lights and costly counters
Another point to keep in mind is that fairness and high impact is impossible to achieve both at the same time, and that continuous fighting is therefore guaranteed between those that aim for high impact and those that want fairness or equality.

Link below:

https://www.latimes.com/opinion/story/2022-03-27/algorithms-unfair-racial-bias-math

Paper below:

https://dl.acm.org/doi/abs/10.1145/3097983.3098095

Similarly, group fairness and individual fairness can’t both be achieved at the same time:

Paper below:

https://arxiv.org/abs/1609.07236

Also another impossibility paper, in that 3 fairness conditions, balance for the positive/negative class and well calibrated decisions only work in 2 regimes: Perfect prediction or equal base rates

Paper below:

https://arxiv.org/abs/1609.05807

Sharmake 24 Sep 2022 18:01 UTC
13 points
1 ∶ 3
on: Two reasons we might be closer to solving alignment than it seems
Strong upvote for giving some outside perspective to the field, and this is an important point of why AI Alignment is likely to be tractable at all. It also means getting many more researchers and money, fast is important for AI Safety.

Sharmake 20 Nov 2023 3:32 UTC
11 points
2 ∶ 2
in reply to: Lukas_Gloor’s comment on: JWS’s Shortform
While I generally agree that they almost certainly have more information on what happened, which is why I’m not really certain on this theory, my main reason here is that for the most part, AI safety as a cause basically managed to get away with incredibly weak standards of evidence for a long time, until the deep learning era in 2019-, especially with all the evolution analogies, and even now it still tends to have very low standards (though I do believe it’s slowly improving right now). This probably influenced a lot of EA safetyists like Ilya, who almost certainly imbibed the norms of the AI safety field, and one of them is that there is a very low standard of evidence needed to claim big things, and that’s going to conflict with corporate/legal standards of evidence.

But I don’t think most people who hold influential positions within EA (or EA-minded people who hold influential positions in the world at large, for that matter) are likely to be that superficial in their analysis of things. (In particular, I’m strongly disagreeing with the idea that it’s likely that the board “basically had no evidence except speculation from the EA/LW forum”. I think one thing EA is unusually good at – or maybe I should say “some/many parts of EA are unusually good at” – is hiring people for important roles who think for themselves and have generally good takes about things and acknowledge the possibility of being wrong about stuff. [Not to say that there isn’t any groupthink among EAs. Also, “unusually good” isn’t necessarily that high of a bar.])

I agree with this weakly, in the sense that being high up in EA is at least a slight update towards them actually thinking through things and being able to make actual cases. My disagreement here is that this effect is probably not strong enough to wash away the cultural effects of operating in a cause area where they don’t need to meet any standard of evidence except long-winded blog posts and getting rewarded, for many reasons.

Also, the board second-guessed it’s decision, which would be evidence for the theory that they couldn’t make a case that actually abided to the standard of evidence for a corporate/legal setting.

If it was any other cause like say GiveWell or some other causes in EA, I would trust them much more that they do have good reason. But AI safety has been so reliant on very low-non-existent standards of evidence or epistemics that they probably couldn’t explain themselves in a way that would abide by the strictness of a corporate/legal standard of evidence.

Edit: The firing wasn’t because of safety related concerns.

Sharmake 30 Oct 2023 22:15 UTC
11 points
2 ∶ 0
in reply to: Geoffrey Miller’s comment on: OpenAI’s new Preparedness team is hiring
I want to flag that I see quite a lot of inappropriate binarization happening, and I generally see quite a lot of dismissals of valid third options.

Either they take the extinction risks seriously, or they don’t.

There are other important possibilities, like a potential belief in AI progress helping or solving the existential risk, thinking that the intervention of increasing AI progress is actually the best strategy, etc. More generally, once we make weaker or no assumptions about AI risk, we no longer obtain the binary you’ve suggested.

So this doesn’t really work, because it basically requires us to assume the conclusion, especially for near-term people.

My conclusion is that they say they take the extinction risks seriously, but they’re lying, or they’re profoundly self-deceived. In any case, their revealed preferences are that they prefer a little extra money, power, and status for themselves over a lot of extra safety for everybody else—and for themselves.

Sharmake 21 Jan 2023 17:37 UTC
11 points
2 ∶ 5
in reply to: JWS’s comment on: FLI FAQ on the rejected grant proposal controversy

Finally, I do think that there is a risk of updating too quickly the other way. Back on the original post, some users have responded this comment saying that it’s ‘entirely correct’[1], but I don’t think it’s reasonable to view Expo’s piece a ‘major misrepresentation’ of what happened—their reporting on the case seems to have been accurate. While ‘Option 1’ seems to be what happened there is still a question of how the grant made it to stage 5 out of 7 on FLI’s grant-making procedure. It’s not the major scandal we feared, but it’s ramifications are more than being ‘mildly embarrassing’.

Similarly, I think EA’s response to the first apology was reasonable, because FLI and Tegmark gave a terrible apology, essentially amounting to a non-apology. It was at least reasonable to suspect that they didn’t actually have much of a problem with Nazism.

Now, a better apology has been produced, but it almost certainly required the criticism of FLI to stick.

Sharmake 25 Dec 2022 15:59 UTC
11 points
5 ∶ 1
in reply to: Will Aldred’s comment on: On applause lights and costly counters
I get some of that feeling, and I disagree with your conclusion. Keep on doing what you are doing, and keep writing possibly wrong posts.