Animal Justice et al. v A.G of Ontario 2024 was recently decided and struck down large portions of Ontario’s ag-gag law. A blog post is here. The suit was partially funded by ACE, which presumably means that many of the people reading this deserve partial credit for donating to support it.
Thanks to Animal Justice (Andrea Gonsalves, Fredrick Schumann,
Kaitlyn Mitchell, Scott Tinney), co-applicants Jessica Scott-Reid and Louise Jorgensen, and everyone who supported this work!
@Marcus Daniell, cofounder of High Impact Athletes, came back from knee surgery and is donating half of his prize money this year. He projects raising $100,000. Through a partnership with Momentum, people can pledge to donate for each point he gets; he has raised $28,000 through this so far. It’s cool to see this, and I’m wishing him luck for his final year of professional play!
I was lucky enough to see Marcus play this year at the Australian Open, and have pledged alongside him! Marcus is so hardworking—in tennis alongside his work at High Impact Athletes! Go Marcus!!!
Egg Innovations announced that they are “on track to adopt the technology in early 2025.” Approximately 300 million male chicks are ground up alive in the US each year (since only female chicks are valuable) and in-ovo sexing would prevent this.
UEP originally promised to eliminate male chick culling by 2020; needless to say, they didn’t keep that commitment. But better late than never!
Congrats to everyone working on this, including @Robert—Innovate Animal Ag, who founded an organization devoted to pushing this technology.[1]
Egg Innovations says they can’t disclose details about who they are working with for NDA reasons; if anyone has more information about who deserves credit for this, please comment!
For others who were curious about what time difference this makes: looks like sex identification is possible at 9 days after the egg is laid, vs 21 days for the egg to hatch (plus an additional ~2 days between fertilization and the laying of the egg.) Chicken embryonic development is really fast, with some stages measured in hours rather than days.
I asked Google when chicken embryos start to feel pain and this was the first result (i.e. I didn’t look hard and I didn’t anchor on a figure):
A recent study by the Technical University of Munich in Germany measured chicken embryos’ heart rate, brain activity, blood pressure and movements in response to potentially painful stimuli like heat and electricity and concluded that they didn’t seem to feel them until at least day 13. (14 Oct 2023)
Egg Innovations, which sells 300 million free-range and pasture-raised eggs a year
This interview with the CEO suggests that Egg Innovations are just in the laying (not broiler) business and that each hen produces ~400 eggs over her lifetime. So this will save ~750,000 chicks a year?
I fairly frequently have conversations with people who are excited about starting their own project and, within a few minutes, convince them that they would learn less starting project than they would working for someone else. I think this is basically the only opinion I have where I can regularly convince EAs to change their mind in a few minutes of discussion and, since there is now renewed interest in starting EA projects, it seems worth trying to write down.
It’s generally accepted that optimal learning environments have a few properties:
You are doing something that is just slightly too hard for you.
In startups, you do whatever needs to get done. This will often be things that are way too easy (answering a huge number of support requests) or way too hard (pitching a large company CEO on your product when you’ve never even sold bubblegum before).
Established companies, by contrast, put substantial effort into slotting people into roles that are approximately at their skill level (though you still usually need to put in proactive effort to learn things at an established company).
Repeatedly practicing a skill in “chunks”
Similar to the last point, established companies have a “rhythm” where e.g. one month per year where everyone has a priority of writing up reflections on how the sales cycle is going, commenting on each other’s writeups, and updating your own. Startups do things by the seat of their pants, which means employees are usually rapidly switching between tasks.
Feedback from experts/mentorship
Startup accelerators like YCombinator partially address this, but still a defining characteristic of starting your own project is that you are doing the work without guidance/oversight.
Moreover, even supposing you learn more at a startup, it’s worth thinking about what it actually is you learn. I know way more about the laws regarding healthcare insurance than I did before starting a company, but that knowledge isn’t super useful to me outside the startup context.
This isn’t a 100% universal knockdown argument – some established companies suck for professional development, and some startups are really great. But by default, I would expect startups to be worse for learning.
I think I agree with this. Two things that might make starting a startup a better learning opportunity than your alternative, in spite of it being a worse learning environment:
You are undervalued by the job market (so you can get more opportunities to do cool things by starting your own thing)
You work harder in your startup because you care about it more (so you get more productive hours of learning)
It depends on what you want to learn. At a startup, people will often get a lot more breadth of scope than they would otherwise in an established company. Yes, you might not have in-house mentors or seasoned pros to learn from, but these days motivated people can fill in the holes outside the org.
Sam Bankman-Fried’s trial is scheduled to start October 3, 2023, and Michael Lewis’s book about FTX comes out the same day. My hope and expectation is that neither will be focused on EA,[1] but several people have recently asked me about if they should prepare anything, so I wanted to quickly record my thoughts.
The Forum feels like it’s in a better place to me than when FTX declared bankruptcy: the moderation team at the time was Lizka, Lorenzo, and myself, but it is now six people, and they’ve put in a number of processes to make it easier to deal with a sudden growth in the number of heated discussions. We have also made a number of design changes, notably to the community section.
CEA has also improved our communications and legal processes so we can be more responsive to news, if we need to (though some of the constraints mentioned here are still applicable).
Nonetheless, I think there’s a decent chance that viewing the Forum, Twitter, or news media could become stressful for some people, and you may want to preemptively create a plan for engaging with that in a healthy way.
This market is thinly traded but is currently predicting that Lewis’s book will not explicitly assert that Sam misused customer funds because of “ends justify the means” reasoning
My hope and expectation is that neither will be focused on EA
I’d be surprised [p<0.1] if EA was not a significant focus of the Michael Lewis book – but agree that it’s unlikely to be the major topic. Many leaders at FTX and Alameda Research are closely linked to EA. SBF often, and publically, said that effective altruism was a big reason for his actions. His connection to EA is interesting both for understanding his motivation and as a story-telling element. There are Manifold prediction markets on whether the book would mention 80′000h (74%), Open Philanthropy (74%), and Give Well (80%), but these markets aren’t traded a lot and are not very informative.[1]
This video titled The Fake Genius: A $30 BILLION Fraud (2.8 million views, posted 3 weeks ago) might give a glimpse of how EA could be handled. The video touches on EA but isn’t centred on it. It discusses the role EAs played in motivating SBF to do earning to give, and in starting Alameda Research and FTX. It also points out that, after the fallout at Alameda Research, ‘higher-ups’ at CEA were warned about SBF but supposedly ignored the warnings. Overall, the video is mainly interested in the mechanisms of how the suspected fraud happened, where EA is only one piece of the puzzle. One can equally get a sense of “EA led SBF to do fraud” as “SBF used EA as a front to do fraud”.
ETA: The book description[2] “mentions “philanthropy”, makes it clear that it’s mainly about SBF and not FTX as a firm, and describes the book as partly a psychological portrait.
“In Going Infinite Lewis sets out to answer this question, taking readers into the mind of Bankman-Fried, whose rise and fall offers an education in high-frequency trading, cryptocurrencies, philanthropy, bankruptcy, and the justice system. Both psychological portrait and financial roller-coaster ride, Going Infinite is Michael Lewis at the top of his game, tracing the mind-bending trajectory of a character who never liked the rules and was allowed to live by his own—until it all came undone.”
Yeah, unfortunately I suspect that “he claimed to be an altruist doing good! As part of this weird framework/community!” is going to be substantial part of what makes this an interesting story for writers/media, and what makes it more interesting than “he was doing criminal things in crypto” (which I suspect is just not that interesting on its own at this point, even at such a large scale).
The Panorama episode briefly mentioned EA. Peter Singer spoke for a couple of minutes and EA was mainly viewed as charity that would be missing out on money. There seemed to be a lot more interest on the internal discussions within FTX, crypto drama, the politicians, celebrities etc.
Maybe Panorama is an outlier but potentially EA is not that interesting to most people or seemingly too complicated to explain if you only have an hour.
Yeah I was interviewed for a podcast by a canadian station on this topic (cos a canadian hedge fund was very involved). iirc they had 6 episodes but dropped the EA angle because it was too complex.
Agree with this and also with the point below that the EA angle is kind of too complicated to be super compelling for a broad audience. I thought this New Yorker piece’s discussion (which involved EA a decent amount in a way I thought was quite fair—https://www.newyorker.com/magazine/2023/10/02/inside-sam-bankman-frieds-family-bubble) might give a sense of magnitude (though the NYer audience is going to be more interested in these sort of nuances than most.
The other factors I think are: 1. to what extent there are vivid new tidbits or revelations in Lewis’s book that relate to EA and 2. the drama around Caroline Ellison and other witnesses at trial and the extent to which that is connected to EA; my guess is the drama around the cooperating witnesses will seem very interesting on a human level, though I don’t necessarily think that will point towards the effective altruism community specifically.
Yeah, “touches on EA but isn’t centred on it” is my modal prediction for how major stories will go. I expect that more minor stories (e.g. the daily “here’s what happened on day n of the trial” story) will usually not mention EA. But obviously it’s hard to predict these things with much confidence.
The Forum feels like it’s in a better place to me than when FTX declared bankruptcy: the moderation team at the time was Lizka, Lorenzo, and myself, but it is now six people, and they’ve put in a number of processes to make it easier to deal with a sudden growth in the number of heated discussions. We have also made a number of design changes, notably to the community section.
This is a huge relief to hear. I noticed some big positive differences, but I couldn’t tell where from. Thank you.
Accordingly, the defendant is precluded from referring to any alleged prior good acts by the defendant, including any charity or philanthropy, as indicative of his character or his guilt or innocence.
I guess technically the prosecution could still bring it up.
(I forgot to tell JP and Lizka in at EAG in NY a few weeks ago, but now’s as good a time as any):
Can my profile karma total be two numbers, one for community and one for other stuff? I don’t want a reader to think my actual work is valuable to people in proportion to my EA Forum karma, as far as I can tell I think 3-5x my karma is community sourced compared to my object-level posts. People should look at my profile as “this guy procrastinates through PVP on social media like everyone else, he should work harder on things that matter”.
Yeah, I kind of agree that we should do something here; maybe the two-dimensional thing you mentioned, or maybe community karma should count less/not at all.
Could see a number of potentially good solutions here, but think the “not at all” is possibly not the greatest idea. Creating a separate community karma could lead to a sort of system of social clout that may not be desirable, but I also think having no way to signal who has and has not in the past been a major contributor to the community as a topic would be more of a failure mode because I often use it to get a deeper sense of how to think about the claims in a given comment/post.
There would be some UX ways to make community clout feel lower status than the other clout, I agree with you that having community clout means more investment / should be preferred over a new account which for all you know is a driveby dunk/sneer after wandering in on twitter.
I’ll cc this to my feature request in the proper thread.
The Forum moderation team has been made aware that Kerry Vaughn published a tweet thread that, among other things, accuses a Forum user of doing things that violate our norms. Most importantly:
Where he crossed the line was his decision to dox people who worked at Leverage or affiliated organizations by researching the people who worked there and posting their names to the EA forum
The user in question said this information came from searching LinkedIn for people who had listed themselves as having worked at Leverage and related organizations.
This is not “doxing” and it’s unclear to us why Kerry would use this term: for example, there was no attempt to connect anonymous and real names, which seems to be a key part of the definition of “doxing”. In any case, we do not consider this to be a violation of our norms.
At one point Forum moderators got a report that some of the information about these people was inaccurate. We tried to get in touch with the then-anonymous user, and when we were unable to, we redacted the names from the comment. Later, the user noticed the change and replaced the names. One of CEA’s staff asked the user to encode the names to allow those people more privacy, and the user did so.
Kerry says that a former Leverage staff member “requests that people not include her last name or the names of other people at Leverage” and indicates the user broke this request. However, the post in question requests that the author’s last name not be used in reference to that post, rather than in general. The comment in question doesn’t refer to the former staff member’s post at all, and was originally written more than a year before the post. So we do not view this comment as disregarding someone’s request for privacy.
Kerry makes several other accusations, and we similarly do not believe them to be violations of this Forum’s norms. We have shared our analysis of these accusations with Leverage; they are, of course, entitled to disagree with us (and publicly state their disagreement), but the moderation team wants to make clear that we take enforcement of our norms seriously.
We would also like to take this opportunity to remind everyone that CEA’s Community Health team serves as a point of contact for the EA community, and if you believe harassment or other issues are occurring we encourage you to reach out to them.
I’ve found that communicating feedback/corrections often works best when I write something that approximates what I would’ve wished the other person had originally written.
Because of the need to sync more explicitly on a number of background facts and assumptions (and due to not having time for edits/revisions), my draft is longer than I think a moderator’s comment would need to be, were the moderation team to be roughly on the same page about the situation. While I am the Cathleen being referenced, I have had minimal contact with Leverage 2.0 and the EA Forum moderation team, so I expect this draft to be imperfect in various ways, while still pointing at useful and important parts of reality.
Here I’ve made an attempt to rewrite what I wish Ben West had posted in response to Kerry’s tweet thread:
The Forum moderation team has been made aware that Kerry Vaughn published a tweet thread that, among other things, accuses a Forum user of doing things that violate our norms. Most importantly:
Where he crossed the line was his decision to dox people who worked at Leverage or affiliated organizations by researching the people who worked there and posting their names to the EA forum.
We care a lot about ensuring that the EA Forum is a welcoming place where people are free to discuss important issues related to world improvement. While disagreement and criticism are an important part of that, we want to be careful not to allow for abuse to take place on our platform, and so we take such reports seriously.
After reviewing the situation, we have compiled the following response (our full review is still in process but we wanted to share what we have so far while the issue is live):
While Leverage was not a topic that we had flagged as “sensitive” back in Sept 2019 when the then-anonymous user originally made his post, the subsequent discussion around the individuals and organizations who were part of the Leverage/Paradigm ecosystem prior to its dissolution in June 2019 has led it to be classified as a sensitive topic to which we expend more scrutiny and are more diligent about enforcing our norms.
In reviewing the particular post referenced above, we found a number of odd elements:
In posing and then answering his own “Question” on the EA Forum, the user makes his accusation of Paradigm (the org) being a front or a secret replacement for Leverage (the org) despite having previously acknowledged the recent dissolution of the Leverage/Paradigm ecosystem (and in a context where the two organizations were publicly known to be closely connected).
The user largely acts as if he is sharing work history discovered on LinkedIn as his primary argument despite only 2 of the 13 named individuals having Leverage listed on their LinkedIn profiles.
The user names 4 additional people as having worked at Leverage despite citing no evidence of that fact (they did not have Leverage on their LinkedIn profiles).
The user then names 7 additional individuals who did not have Leverage on their LinkedIn profiles and who the user also did not believe to have originally worked at Leverage.
The user neglects to include any context or timelines from LinkedIn, e.g. whether there had been a recent change in work history at the point of the ecosystem dissolution, whether the individuals in question started at Leverage and then moved to Paradigm, started at Paradigm and then moved to Leverage, or started at either Leverage or Paradigm and then moved on to other projects. It is also unclear which, if any, continued to work at either Leverage or Paradigm after the ecosystem dissolution.
More than 2 years later, after reading a clarifying and detailed post about the relevant history of Leverage and Paradigm (which included a request to protect people’s identities) and discovering that the Forum mods had removed the named individuals from his comment, the only edit the user decided to make was to deanonymize the names he’d compiled of people previously affiliated with the Leverage ecosystem.
When this post was initially brought to our attention in July of 2020, along with an explanation of possible negative consequences for the people listed (including ~4 individuals who the user was spreading potential misinformation about), we tried to get in touch with the then-anonymous user, and when we were unable to, we redacted the names from the comment and left an explanation for how the comment could make its point without using the personal information of the named individuals.
At the time, we had been informed that the user was mistaken about the work history for some of the people he listed, in large part due to relying on his incorrect personal assumptions. We did not consider the way that some of the 4 (and possibly others) might’ve intentionally excluded Leverage from their work histories, as we were focused on the ones who were incorrectly identified as having worked at Leverage and the potential consequences of that misinformation. Without yet knowing or investigating the full extent of the anonymous user’s posting history across multiple accounts, we did not suspect a pattern of hostile posts. Because of these factors, we did not evaluate whether this post might be a case of doxing.
In Dec 2021, Cathleen, one of the 4 who had been listed as working at Leverage (despite no record on LinkedIn), published a detailed account of her experience at Leverage/Paradigm. In it, she shared her perspective on harassment and ill-will that had come from the EA and Rationalist community members, and the negative effects of misinformation spread via public community forums. She explained why she had intentionally excluded Leverage from her LinkedIn many years prior and asked that people protect her identity as well as the identities of others from the Leverage ecosystem due to the risk of cancellation and harassment.
A few days later, the EA Forum user (who had revealed his real identity a couple months prior) returned to his anonymous post from Sept 2019 and deanonymized the first and last names of all 13 individuals he had previously named. This included Cathleen as well as the other 3 individuals who he attributed to Leverage (despite no record on LinkedIn). He accompanied the edits with a false/misleading comment (using the anonymous account) minimizing the substantive merit of prior requests for corrections to his post and claiming that all of the relevant information had actually been originally drawn from LinkedIn.
(At a cursory glance, it’s difficult to determine the most natural interpretation of the scope of Cathleen’s request, in order to assess the likelihood that the user was knowingly violating her wishes. We initially had a quite narrow interpretation, reading the quote out of context, but I think the situation becomes clearer if you take the time to read her entire post or the section that the quote was pulled from, entitled “We want to move forward with our lives (and not be cancelled)”, which includes a direct reference to LinkedIn/her intent to keep her work history private.)
After receiving a new complaint about the potential harm of listing individuals’ names, including the spread of misinformation caused by the user’s updated post, we reviewed the case and this time found no violation. As a compromise, we did offer to ask the user if they would be willing to encode the full names to help protect the individuals from potential negative effects arising from a simple google search.
We have since realized that many people (including people on our moderation team) took the user at his word without carefully reviewing the post. We had become confused about the specifics (falsely believing that he was sharing publicly available information from LinkedIn and thus believing that the information could be reasonably treated as true and that the objections raised were splitting hairs). We also did not accurately recognize the general nature/intent of his original post nor the potential negative effects of allowing the information to stand, and we did not evaluate the deanonymizing edits in the context of Cathleen’s recent public request and voiced concerns.
In Oct 2021, when the user had revealed his identity and his use of multiple anonymous accounts, we also failed to review the complete body of evidence and the ways that his actions had potentially violated our norms (e.g. using multiple anonymous accounts to convey similar views on a topic), as well as notice that the full pattern of posts indicated a type of ill-will that we discourage and that is especially relevant given the sensitive nature of the topic of Leverage/Paradigm.
In retrospect, we recognize that while we would like to give users the benefit of the doubt, when there are complaints of doxing, harassment, or other poor behavior, it makes sense for us to look more carefully at the situation and potentially draw on CEA’s Community Health team’s expertise in assisting individuals who flag that they’ve been wronged by a user on the Forum.
Something else we did not consider (because we unfortunately don’t have the bandwidth to track all the goings on in the EA and Rationalist communities) is that the level of threat experienced by people who had previously been part of the Leverage ecosystem had become quite high. In evaluating cases of disclosing private information or even assembling and publishing public information, context matters.
As an example:
On the face of it, it seems fine to have openly communist views or be LGBTQ, but history has shown us that during certain eras, e.g. the Second Red Scare in the US, creating and posting lists of such people (even if true or otherwise individually knowable) would likely subject them to harassment or worse.
It is not an excuse if someone else could create a similarly damaging list, and it doesn’t seem right to ask people to hide their work history from potential employers on a professional networking platform for the sole purpose of protecting themselves from being subject to defamatory public posts from ill-willed and/or ill-informed EAs and Rationalists. It is already unfortunate that the damage to the reputation of the relevant orgs has made it difficult for individuals to decide when and how to associate themselves with their former projects.
Guilt by association is not a good faith argument here, and at a minimum, it seems reasonable to honor individual wishes for Forum users to refrain from using their full names in affiliation with prior projects when requested (and to be careful not to do so in a way that falsely implies that the person is (or should be) ashamed of their affiliation).
After reviewing the overall situation, we think it’s important for users as well as moderators to recognize that posts about former Leverage/Paradigm staff do not happen in a vacuum. We strongly condemn the sharing of information about an individual’s prior professional or social affiliation in a way that intentionally or negligently exposes them to undue negative consequences.
If people are proud of their work at an organization but feel the need to disassociate themselves from that org publicly, it seems like something has gone wrong.
Given the overall pattern of posts from this user’s accounts across the EA Forum as well as Less Wrong from 2018-2021, it seems plausible to our team that the actions of this user may have actually been a significant contributing factor in fomenting negative sentiment towards this group of people. In light of that, making the decision to list their full names in this comment in 2019 and then editing the comment to include them again in 2021, after Cathleen’s detailed post (which includes information relevant to the comment’s hypothesis as well as a request and argument for privacy), we find it harder to defend an interpretation where there was not an intent to cause harm to the named individuals.
Further, it is our understanding that only a fraction of those named worked for Leverage or Paradigm at the time of the original post, and only ~1 to 2 of those named worked for Leverage or Paradigm at the time of the subsequent deanonymizing edit; given that, and with the assumption that the argument for both relevancy of the post on the EA Forum as well as the argument for wrongdoing relies solely on the pattern of employment at these orgs, the weighing of potential benefit/value vs. cost/harm to prior project members seems particularly clear.
Intimidation and harassment can be executed in subtle ways, and while intent can be hard to ascertain, we encourage participants on the Forum to put in extra effort to ward against their posts landing in a gray area.
We don’t think that every case of bad behavior needs to fit neatly into our listed norms (and in evaluating cases like these, we also think it makes sense to revisit our listed norms to see if we should make changes for clarity or scope)[1], but it seems clear to us that the type of behavior exhibited by this user across their anonymous accounts is neither generous nor collaborative and it also seems likely to interfere with good discourse (not least of all by creating a hostile environment for some members of the community).
While we wish we would’ve done better, given our knowledge at the time, we don’t see this as a major failing, but we do recognize the harm that was caused and we want to emphasize that the use of anonymous accounts to harass individuals or groups is not something that we tolerate.
We have referred this case to the CEA Community Health team for further review. They will look at the totality of content from this user on this and related topics, examining patterns, severity, as well as the time period spanned by relevant posts and comments on the EA Forum as well as LW, as a way of assessing potential and actual negative impacts and intent. With the permission of relevant parties, they will also review registered complaints about the user’s behavior. With their input, we will deliberate further and decide whether there is mitigating action that the Forum moderators can and should take in this particular case.
If you have a world improvement related issue that you believe needs public attention but aren’t sure how to navigate it while minimizing unnecessary harm, we encourage you to reach out to the CEA Community Health team who can help organize your thoughts and perhaps mediate a discussion where more information can be exchanged before escalating to a public post. We recognize that in a situation where you suspect a conspiracy or are otherwise suspicious of others’ actions, it may be harder to prioritize the discussion norms of the Forum, but it is in those moments that the norms are most important to respect.
Be kind.
Stay civil, at the minimum. Don’t sneer or be snarky. In general, assume good faith. We may delete unnecessary rudeness and issue warnings or bans for it.
Substantive disagreements are fine and expected. Disagreements help us find the truth and are part of healthy communication.
Stay on topic.
No spam. This forum is for discussions about improving the world, not the promotion of services.
Be honest.
Don’t mislead or manipulate.
Communicate your uncertainty and the true reasons behind your beliefs as much as you can.
*(The Forum moderators are currently grappling with an issue that may be relevant to situations involving sensitive topics like these: it does not violate our norms to inadvertently publish false or misleading information – but in the case that a correction or material clarification is made and the OP doesn’t update their post or comment, an argument could be made that the user is either in violation of the norm of scout mindset/willingness to update their view, or (if they do update their understanding but don’t update their post) they could be in violation of knowingly/deliberately spreading misinformation. We generally have not wanted to act as the arbiters of truth, so it’s not yet clear how to best moderate a situation like this.)
To share a brief thought, the above comment gives me a bad juju because it puts a contested perspective into a forceful and authoritative voice, while being long enough that one might implicitly forget that this is a hypothetical authority talking[1]. So it doesn’t feel to me like a friendly conversational technique. I would have preferred it to be in the first person.
Garcia Márquez has a similar but longer thing going on in The Handsomest Drowned Man In The World, where everything after “if that magnificent man had lived in the village” is a hypothetical.
Thank you for your attention to this matter. I must confess that I found the response confusing as you seem to be responding to claims slightly (though importantly!) different from the ones I made in my Twitter post.
Specifically, my view is (1) you are mistaken about the definition of doxing and have instead provided a definition of deanonymization; (2) this person’s behavior meets the definitions of doxing; (3) norms that allow these posts would also allow more serious abuses.
I’d also like to clarify that I am not at the moment accusing you or the EA Forum moderators of failing to enforce EA Forum rules or norms in this case. I am not deeply familiar with the EA Forum rules or norms, so I am indifferent on this question.
This poster violated the norms that I think would be reasonable for a high-quality discussion forum. I also think this poster violated standards of reasonable behavior for a person with high-status membership in an intellectual community. I leave the question of EA Forum rules and norms to you and your team.
Additionally, while I alluded to but did not directly discuss this in the Twitter thread, the EA Forum moderation team took several actions that lessened the harm this poster intended to cause. While I think further actions are warranted, I would nevertheless like to express my gratitude to the team for what they have already done.
On the definition of doxing
The user in question said this information came from searching LinkedIn for people who had listed themselves as having worked at Leverage and related organizations. This is not “doxing” and it’s unclear to us why Kerry would use this term: for example, there was no attempt to connect anonymous and real names, which seems to be a key part of the definition of “doxing”.
This is not the definition of doxing.
A simple internet search reveals the following definitions:
Wikipedia: “Doxing or doxxing is the act of publicly providing personally identifiable information about an individual or organization, usually via the Internet. . . . Doxing may be carried out for reasons such as online shaming, extortion, and vigilante aid to law enforcement”
Urban Dictionary: “Using private information gleaned from the internet to attack someone with whom you disagree, often by publishing their personal info, opening them to abuse and possibly, danger.”
What you provide is the definition of the much narrower activity of deanonymization. I agree that the poster’s behavior is not deanonymization. I never indicated otherwise.
Deanonymization is neither a necessary nor sufficient condition for doxing. It is possible to dox someone even if they maintain no anonymous identities (e.g., by publishing non-public information that does not pertain to an anonymous identity for the purposes of harassment). It is also possible to reveal a person’s anonymous identity without doxing them if the reveal involves neither ill intent nor ill effects.
Why the poster’s behavior is doxing
Doxing has two important components: (1) revealing private information; (2) with ill-intent. Below I argue that the poster’s behavior meets both of these standards.
The information was private
In the EA Forum comment where the doxing occurred, the poster says the following (emphasis mine):
Paradigm Academy does not list its team on its website. However, a quick search on Linkedin yields 16 results (13 with public profiles) for people at Paradigm Academy. Of these 13 profiles, 7 include experience at Leverage Research or allied organizations, and a further 4 are well-known to have worked for Leverage Research.
The term “well-known” indicates that the information is not on their LinkedIn profiles, but is known to the poster. Thus, they relied on non-public information to associate the people with the organizations.
However, even if the poster had used only information from LinkedIn profiles and not their own knowledge, this would still constitute a reveal of private information because “private information” is context-dependent. Almost all canonical cases of doxing involve taking information that is findable on the public internet (albeit with difficulty) and then revealing that information in an adversarial context.
Consider, for example, the high-profile case of the doxing of Scott Alexander. Scott Alexander’s real name was not private in the sense of “not findable on the public internet.” Indeed, it could be revealed with a little bit of dedicated Googling. It was also widely known inside the EA and Rationality community. The Times’ threat to publish his real name was nevertheless a threat to dox him because it was not well-known to the readership of The Times.
Who was employed at Leverage and Paradigm was not well-known to the members of the EA community. Most of those who worked at Leverage and Paradigm had little interaction with the EA community. Thus, the information revealed was private in the relevant sense.
The poster acted with ill-intent
There are several ways to see that the poster acted with ill intent.
(1) The social context of this post is that there was a deterioration of the relationship between the Effective Altruism and Leverage Research communities. This is easy to see by reviewing the comments on the “Basic Facts” post, which are substantially negative. While one might dispute that the negativity is fair, accurate, or deserved, it was not considered so by members of Leverage or Paradigm staff. Thus, the EA Forum is an adversarial context from the perspective of Leverage and Paradigm employees.
(2) A review of the posting history of the anonymoose and throwaway accounts indicates that their primary goal was to spread negative information about Leverage and Paradigm.
(3) The context in which the people are doxed is in response to the following “question:” “To what extent is Paradigm Academy a front organization for, or a covert rebrand of Leverage Research?”
This is, of course, a quite blatant attempt to insulate nefariousness where none is warranted. There was never any attempt to conceal the relationship between Leverage and Paradigm by the staff of either organization. No information to indicate otherwise is ever presented by the poster.
(4) There is simply no reason to include the people’s names except for the purposes of doxing. The point of the comment is that there is a heavy association between people that worked at Paradigm and those that worked at Leverage. There is no reason that specific names need to be included to make this point.
(5) The post violated the letter and spirit of Cathleen’s request.
On this point, Ben indicates the following:
Kerry says that a former Leverage staff member “requests that people not include her last name or the names of other people at Leverage” and indicates the user broke this request. However, the post in question requests that the author’s last name not be used in reference to that post, rather than in general. The comment in question doesn’t refer to the former staff member’s post at all, and was originally written more than a year before the post. So we do not view this comment as disregarding someone’s request for privacy.
However, the full request by Cathleen is as follows (emphasis mine):
In discussions of this post (the content of which I can’t predict or control), I’d ask that you just refer to me as Cathleen, to minimize the googleable footprint. And I would also ask that, as I’ve done here, you refrain from naming others whose identities are not already tied up in all this.
As there is some confusion on this point, it is important to be clear. The central complaint in the Twitter thread is that *5 days* after Cathleen’s post, the poster edited their comment to add the names of Leverage and Paradigm employees back to the comment, including Cathleen’s last name. This violates Cathleen’s request.
Ben is correct that the comment originally occurred substantially before Cathleen’s post. But this is immaterial. The point is that the poster re-added the names after the request had been made.
Norms that allow these posts would also allow more serious abuses
A further issue with this understanding of EA norms is that it would allow more serious abuses on the EA Forum. If this is not clear, I can provide some examples.
However, even if the poster had used only information from LinkedIn profiles and not their own knowledge, this would still constitute a reveal of private information because “private information” is context-dependent.
If you list your organizational affiliation on LinkedIn (and if you are indeed correct that Paradigm Academy was not trying to be a cover for Leverage Research that shielded it from public scrutiny), then I don’t think you get to complain when someone quickly googles you and finds your LinkedIn profile.
Like, if Scott Alexander had listed his real name on his about page, or on LinkedIn, connected to his pseudonym, then yeah, I would have also had very little sympathy for the objections to the New York Times. There clearly are degrees of what kind of connections you can make here, and searching an organization on LinkedIn really does not cross any obvious line here.
Maybe you are objecting to drawing a connection between Leverage Research and Paradigm Academy, but you are also saying that nobody was saying that Paradigm Academy was intentionally trying to not appear to be part of what was broadly known as “Leverage”, so I don’t understand how you can make both of these arguments at the same time.
If you list your organizational affiliation on LinkedIn (and if you are indeed correct that Paradigm Academy was not trying to be a cover for Leverage Research that shielded it from public scrutiny), then I don’t think you get to complain when someone quickly googles you and finds your LinkedIn profile.
First, as I noted in my response to Ben, some of the information included in the doxing was from the poster’s personal knowledge and not from the people’s LinkedIn profiles. Thus, you can’t defend the doxing by saying that the information was publicly available. It simply wasn’t.
Second, I am not complaining about someone quickly googling anyone and finding their LinkedIn profile. I am complaining about making it easier to harass people by posting a central repository of information about those people on a forum frequented by people with a history of harassing them.
It is certainly the case that former Leveragers took the release of their names on the EA forum as an attempt to invite harassment by members of the EA community. We were contacted by former Leveragers to see if there was anything we could do to get the names removed for this very reason.
I suspect the actual crux of the discussion here is that most EAs are unaware of the history of poor and bizarre behavior towards Leverage/Paradigm by members of the EA community. To help with the knowledge gap, I’m considering writing a Twitter thread that will share some of the most egregious examples.
At this point, the moderators are trying to focus on any practical steps we should take. Given that the names in question are encoded, and no one currently listed in the comment has reached out to us, we do not plan to take further action. Anyone who feels that private or incorrect information about themselves is posted on the Forum is always free to contact us.
Could you explain what’s not practical about these simple steps that you could take:
1) Create an EA Forum policy against sockpuppeting, and apply it retroactively to this case. This might naturally result in deleting the offending posts or adding a notification indicating they are sockpuppets.
2) Remove the encoded names and replace them with numbers per the edit your team made originally.
3) Change the word “doxing” to “deanonymizing” in the following sentence from the Guide to Norms (since the behavior you intend to prohibit is not doxing):
Doxing — or revealing someone’s real name if they are anonymous on the Forum or elsewhere on the internet — is prohibited
[Disclaimer, I have very little context on this & might miss something obvious and important]
In discussions of this post (the content of which I can’t predict or control), I’d ask that you just refer to me as Cathleen, to minimize the googleable footprint. And I would also ask that, as I’ve done here, you refrain from naming others whose identities are not already tied up in all this.
As there is some confusion on this point, it is important to be clear. The central complaint in the Twitter thread is that *5 days* after Cathleen’s post, the poster edited their comment to add the names of Leverage and Paradigm employees back to the comment, including Cathleen’s last name. This violates Cathleen’s request.
AFAICT the disagreement between Kerry and Ben stems from interpreting the second part of Cathleen’s ask differently. There seem to be two ways of reading the second part: 1. Asking people to refrain from naming others who are not already tied into this in general. 2. Asking people to refrain from naming others who are not already tied into this in discussions of her post.
To me, it seems pretty clear that she means the latter, given the structure of the two sentences. If she was aiming for the first interpretation, I think she should have used a qualifier like “in general” in the second sentence. In the current formulation, the “And” at the beginning of the sentence connects the first ask/sentence very clearly to the second. I guess this can be up for debate, and one could interpret it differently, but I would certainly not fault anyone for going with interpretation 2.[1]
If we assume that 2 is the correct reading, Kerry’s claim (cited above) does not seem relevant anymore / Ben’s original remark (cited below) seems correct. The timeline of edits doesn’t change things. Ben’s original remark (emphasis mine):
The comment in question doesn’t refer to the former staff member’s post at all, and was originally written more than a year before the post. So we do not view this comment as disregarding someone’s request for privacy.
Even if she meant interpretation 1, it is unclear to me that this would be a request that I would endorse other people enforcing. Her request in interpretation 2 seems reasonable, in part because it seems like an attempt to avoid people using her post in a way she doesn’t endorse. A general “don’t associate others with this organisation” would be a much bigger ask. I would not endorse other organisations asking the public not to connect its employees to them (e.g. imagine a GiveWell employee making the generic ask not to name other employees/collaborators in posts about GiveWell), and the Forum team enforcing that.
I disagree with that interpretation especially given the context of Cathleen’s post. It includes lengthy discussions of poor and bizarre behavior by members of the EA community toward Leverage/Paradigm staff.
I think reading Cathleen’s post and then re-adding her name is either an intentional violation of her wishes or, at a minimum, shows a reckless disregard for her request for privacy.
In any case, I don’t think this issue is central. The original comment already was doxing given the context, which includes the “question” to which it was a reply, the poster’s overall pattern of behavior, and the history of very poor behavior towards Leverage/Paradigm staff by members of the EA community. The poster’s behavior after reading Cathleen’s request for privacy simply makes their intention clearer.
Cathleen has also now commented above which I think clarifies how her request was meant to be taken. Here’s a quote (as context, she’s formatted her comment to be an idea for how CEA might have responded instead):
(At a cursory glance, it’s difficult to determine the most natural interpretation of the scope of Cathleen’s request, in order to assess the likelihood that the user was knowingly violating her wishes. We initially had a quite narrow interpretation, reading the quote out of context, but I think the situation becomes clearer if you take the time to read her entire post or the section that the quote was pulled from, entitled “We want to move forward with our lives (and not be cancelled)”, which includes a direct reference to LinkedIn/her intent to keep her work history private.)
Another question I wanted to ask is whether the EA Forum has rules against creating sock puppet accounts. This is defined as follows:
Wikipedia: “Online, it came to be used to refer to a false identity assumed by a member of an internet community who spoke to, or about, themselves while pretending to be another person.”
Urban dictionary: “A false identity adopted by trolls and other malcontents to support their own postings.”
I ask because it appears to me that the poster acted deceptively in managing their multiple anonymous accounts by (1) using a second anonymous account to reply to their first anonymous account; (2) promoting the actions of their anonymous account from their public account without disclosing that they were behind the anonymous account.
Most forums I’ve been a part of have rules against sockpuppeting, but reviewing the EA Forum rules and guidelines, I did not see anything about that.
We have received an influx of people creating accounts to cast votes and comments over the past week, and we are aware that people who feel strongly about human biodiversity sometimes vote brigade on sites where the topic is being discussed. Please be aware that voting and discussion about some topics may not be representative of the normal EA Forum user base.
Huh, seems like you should just revert those votes, or turn off voting for new accounts. Seems better than just having people be confused about vote totals.
And maybe add a visible “new account” flag—I understand not wanting to cut off existing users creating throwaways, but some people are using screenshots of forum comments as evidence of what EAs in general think.
Thanks! Yeah, this is something we’ve considered, usually in the context of trying to make the Forum more welcoming to newcomers, but this is another reason to prioritize that feature.
Yeah, I think we should probably go through and remove people who are obviously brigading (eg tons of votes in one hour and no other activity), but I’m hesitant to do too much more retroactively. I think it’s possible that next time we have a discussion that has a passionate audience outside of EA we should restrict signups more, but that obviously has costs.
How do you differentiate someone who is sincerely engaging and happens to have just created an account now from someone who just wants their viewpoint to seem more popular and isn’t interested in truth seeking?
Or are you saying we should just purge accounts that are clearly in the latter category, and accept that there will be some which are actually in the latter category but we can’t distinguish from the former?
I think being like “sorry, we’ve reverted votes from recently signed-up accounts because we can’t distinguish them” seems fine. Also, in my experience abusive voting patterns are usually very obvious, where people show up and only vote on one specific comment or post, or on content of one specific user, or vote so fast that it seems impossible for them to have read the content they are voting on.
How about: getting a lot of downvotes from new accounts doesn’t decrease your voting-power and doesn’t mean your comments won’t show up on the frontpage? Half a dozen of my latest comments have responded to HBDers. Since they get a notification it doesn’t surprise me that those comments get immediate downvotes which hides them from the frontpage and subsequently means that they can easily decrease my voting-power on this forum (it went from 5 karma for a strong upvote to now 4 karma for a strong upvote). Giving brigaders the power to hide things from the frontpage and decide which people have more voting-power on this forum seems undesirable.
Note: I went through Bob’s comments and think it likely they were brigaded to some extent. I didn’t think they were in general excellent, but they certainly were not negative-karma comments. I strong-upvoted the ones that were below zero, which was about three or four.
I think it is valid to use the strong upvote as a means of countering brigades, at least where a moderator has confirmed there is reason to believe brigading is active on a topic. My position is limited to comments below zero, because the harmful effects of brigades suppressing good-faith comments from visibility and affirmatively penalizing good-faith users are particularly acute. Although there are mod-level solutions, Ben’s comments suggest they may have some downsides and require time, so I feel a community corrective that doesn’t require moderators to pull away from more important tasks has value.
I also think it is important for me to be transparent about what I did and accept the community’s judgment. If the community feels that is an improper reason to strong upvote, I will revert my votes.
Could you set a minimum karma threshold (or account age or something) for your votes to count? I would expect even a low threshold like 10 would solve much of the problem.
Yeah, interesting. I think we have a lot of lurkers who never get any karma and I don’t want to entirely exclude them, but maybe some combo like “10 karma or your account has to be at least one week old” would be good.
A couple months ago I remarked that Sam Bankman-Fried’s trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that people’s behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently don’t know what actually happened, and should hold multiple hypotheses simultaneously.[1]
It’s very hard to predict the outcome of media stories. Here are a few takes I’ve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): “OpenAI’s board members’ religion of ‘effective altruism’ and its misapplication could have set back the world’s path to the tremendous benefits of artificial intelligence”
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was “incredibly principled and brave” to confront Altman, even if it failed to explain its actions in public. “The board is rightly being attacked for incompetence,” the entrepreneur told me. “But if the new board is composed of normal tech people, then I doubt they’ll take safety issues seriously.”
The Economist: “The chief lesson is the folly of policing technologies using corporate structures … Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governments”
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we don’t yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people we’ve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexander’s recent post also has a nice summary of victories.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average person’s opinion is unchanged
I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they won’t know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So I’m not sure how much weight I’d give that article specifically.
Sure, here are the top hits for “Effective Altruism OpenAI” (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
“AI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.”
Top comment: ” I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up …”
“Coming just weeks after effective altruism’s most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Age—and failure to do so could have apocalyptic consequences.”
“EA is currently being scrutinized due to its association with Sam Bankman-Fried’s crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of “AI safety.”
“The first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.”
Ah yeah sorry, the claim of the post you criticized was not that FTX isn’t mentioned in the press, but rather that those mentions don’t seem to actually have impacted sentiment very much.
I thought when you said “FTX is heavily influencing their opinion” you were referring to changes in sentiment, but possibly I misunderstood you – if you just mean “journalists mention it a lot” then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I don’t think this is a “journalist only” thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesn’t seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for “effective altruism”. The first reference to FTX doesn’t come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isn’t actually the crux.
In contrast, I see 3 e/acc-type criticisms before that, two “I like EA but this AI stuff is too weird” things (including one retweeted by Yann LeCun??), two “EA is tech-bro/not diverse” complaints and one thing about Whytham Abbey.
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that it’s been more than a year, and these tweets are quite short.
I don’t know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didn’t seem the right type of filter.
Yeah, makes sense. Although I just tried doing the “latest” sort and went through the top 40 tweets without seeing a reference to FTX/SBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesn’t mention FTX.
Probably you need some longitudinal data to have this be useful.
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. “EA, the social movement involved in the FTX and OpenAI crises” is not a good epithet.
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.
I think this is an oversimplification. This effect is largely caused by competing messages; the modern internet optimizes information for memetic fitness e.g. by maximizing emotional intensity or persuasive effect, and people have so much routine exposure to stuff that leads their minds around in various directions that they get wary (or see having strong reactions to anything at all as immature, since a large portion of outcries on the internet are disproportionately from teenagers). This is the main reason why people take things with a grain of salt.
However, overton windows can still undergo big and lasting shifts (this process could also be engineered deliberately long before generative AI emerged, e.g. via clown attacks which exploit social status instincts to consistently hijack any person’s impressions of any targeted concept). The 80,000 hours podcast with Cass Sunstein covered how Overton windows are dominated by vague impressions of what ideas are acceptable or unacceptable to talk about (note: this podcast was from 2019). This dynamic could plausibly strangle EA’s access to fresh talent, and AI safety’s access to mission-critical policy influence, for several years (which would be far too long).
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about.
On the flip side, johnswentworth actually had a pretty good take on this; that the human brain is instinctively predisposed to over-focus on the risk of their in-group becoming unpopular among everyone else:
First, [AI safety being condemned by the public] sure does sound like the sort of thing which the human brain presents to us as a far larger, more important fact than it actually is. Ingroup losing status? Few things are more prone to distorted perception than that.
Thanks for the helpful comment – I had not seen John’s dialogue and I think he is making a valid point.
Fair point that the lack of impact might not be due to attention span but instead things like having competing messages.
In case you missed it: Angelina Li compiled some growth metrics about EA here; they seem to indicate that FTX’s collapse did not “strangle” EA (though it probably wasn’t good).
The food sector has witnessed a surge in the production of plant-based meat alternatives that aim to mimic various attributes of traditional animal products; however, overall sensory appreciation remains low. This study employed open-ended questions, preference ranking, and an identification question to analyze sensory drivers and barriers to liking four burger patties, i.e., two plant-based (one referred to as pea protein burger and one referred to as animal-like protein burger), one hybrid meat-mushroom (75% meat and 25% mushrooms), and one 100% beef burger. Untrained participants (n=175) were randomly assigned to blind or informed conditions in a between-subject study. The main objective was to evaluate the impact of providing information about the animal/plant-based protein source/type, and to obtain product descriptors and liking/disliking levels from consumers. Results from the ranking tests for blind and informed treatments showed that the animal-like protein [Impossible] was the most preferred product, followed by the 100% beef burger. Moreover, in the blind condition, there was no significant difference in preferences between the beef burger and the hybrid and pea protein burgers. In the blind tasting, people preferred the pea protein burger over the hybrid one, contrary to the results of the informed tasting, which implies the existence of affecting factors other than pure hedonistic enjoyment. In the identification question, although consumers correctly identified the beef burger under the blind condition, they still preferred the animal-like burger.
Thanks for the comment and the followup comments by you and Michael, Ben. First, it’s really cool that Impossible was preferred to beef burgers in a blind test! Even if the test is not completely fair! Impossible has been around for a while, and obviously they would’ve been pretty excited to do a blind taste test earlier if they thought they could win, which is evidence that the product has improved somewhat over the years.
I want to quickly add an interesting tidbit I learned from food science practitioners[1] a while back:
Blind taste tests are not necessarily representative of “real” consumer food preferences.
By that, I mean I think most laymen who think about blind taste tests believe that there’s a Platonic taste attribute that’s captured well by blind taste tests (or captured except for some variance). So if Alice prefers A to B in a blind taste test, this means that Alice in some sense should like A more than B. And if she buys (at the same price) B instead of A at the supermarket, that means either she was tricked by good marketing, or she has idiosyncratic non-taste preferences that makes her prefer B to A (eg positive associations with eating B with family or something).
I think this is false. Blind taste tests are just pretty artificial, and they do not necessarily reflect real world conditions where people eat food. This difference is large enough to sometimes systematically bias results (hence the worry about differentially salted Impossible burgers and beef burgers).
People who regularly design taste tests usually know that there are easy ways that they can manipulate taste tests so people will prefer more X in a taste test, in ways that do not reflect more people wanting to buy more X in the real world. For example, I believe adding sugar regularly makes products more “tasty” in the sense of being more highly rated in a taste test. However, it is not in fact the case that adding high amounts of sugar automatically makes a product more commonly bought. This is generally understood as people in fact having genuinely different food preferences in taste test conditions than consumer real world decisions.
Concrete example: Pepsi consistentlyperforms better than Coca-Cola in blind taste tests. Yet most consumers consistently buy more Coke than Pepsi. Many people (including many marketers, like the writers of the hyperlinks above) believe that this is strong evidence that Coke just has really good brand/marketing, and is able to sell an inferior product well to the masses.
Personally, I’m not so sure. My current best guess is that this discrepancy is best explained by consumer’s genuine drink preferences being different in blind taste tests from real-world use cases. As a concrete operationalization, if people make generic knock-offs of Pepsi and Coke with alternative branding, I would expect faux Pepsi (that taste just like Pepsi) to perform better in blind tastes than faux Coke (that tastes just like Coke), but for more people to buy faux Coke anyway.
For Impossible specifically, I remember doing a blind taste test in 2016 between Impossible beef and regular beef, and thinking that personally I liked [2]the Impossible burger more. But I also remember distinctly that the Impossible burger had a much stronger umami taste, which naively seems to me like exactly the type of thing that more taste testers will prefer in blind test conditions than real-world conditions.
This is a pretty long-winded comment, but I hope other people finds this interesting!
This is lore, so it might well be false. I heard this from practitioners who sounded pretty confident, and it made logical sense to me, but this is different from the claims being actually empirically correct. Before writing this comment, I was hoping to find an academic source on this topic I can quickly summarize, but I was unable to find it quickly. So unfortunately my reasoning transparency here is basically on the level of “trust me bro :/”
To be clear I think it’s unlikely for this conclusion to be shared by most taste testers, for the aforementioned reason that if Impossible believed this, they would’ve done public taste tests way before 2023.
Be less ambitious I don’t have a huge sample size here, but the founders I’ve spoken to since the “EA has a lot of money so you should be ambitious” era started often seem to be ambitious in unhelpful ways. Specifically: I think they often interpret this advice to mean something like “think about how you could hire as many people as possible” and then they waste a bunch of resources on some grandiose vision without having validated that a small-scale version of their solution works.
Founders who instead work by themselves or with one or two people to try to really deeply understand some need and make a product that solves that need seem way more successful to me.[1]
Think about failure The “infinite beta” mentality seems quite important for founders to have. “I have a hypothesis, I will test it, if that fails I will pivot in this way” seems like a good frame, and I think it’s endorsed by standard start up advice (e.g. lean startup).
Of course, it’s perfectly coherent to be ambitious about finding a really good value proposition. It’s just that I worry that “be ambitious” primes people to be ambitious in unhelpful ways.
Two days after posting, SBF, who the thread lists as the prototypical example of someone who would never make a plan B, seems to have executed quite the plan B.
If your content is viewed by 100,000 people, making it more concise by one second saves an aggregate of one day across your audience. Respecting your audience means working hard to make your content shorter.
When the 80k podcast describes itself as “unusually in depth,” I feel like there’s a missing mood: maybe there’s no way to communicate the ideas more concisely, but this is something we should be sad about, not a point of pride.[1]
This is a thoughtful post and a really good sentiment IMO!
When the 80k podcast describes itself as “unusually in depth,” I feel like there’s a missing mood: maybe there’s no way to communicate the ideas more concisely, but this is something we should be sad about, not a point of pride.
As you touched on, I’m not sure 80k is a good negative example, to me it seems like a positive example of how to handle this?
In addition to a tight intro, 80k has a great highlight section, that to me, looks like someone smart tried to solve this exact problem, balancing many considerations.
This highlight section has good takeaways and is well organized with headers. I guess this is useful for 90% of people who only browse at the content for 1 minute.
Thanks for the push back! I agree that 80k cares more about the use of their listener’s time than most podcasters, although this is a low bar.
80k is operating under a lot of constraints, and I’m honestly not sure if they are actually doing anything incorrectly here. Notably, the fancy people who they get on the podcast probably aren’t willing to devote many hours to rephrasing things in the most concise way possible, which really constrains their options.
I do still feel like there is a missing mood though.
To me, economy of words is what’s important, rather than overall length. Long can be wonderful, as long as the writer uses all those words well. Short can be wonderful, if the writer uses enough words to convey their complete thoughts.
Democratizing risk post update Earlier this week, a post was published criticizing democratizing risk. This post was deleted by the (anonymous) author. The forum moderation team did not ask them to delete it, nor are we aware of their reasons for doing so. We are investigating some likely Forum policy violations, however, and will clarify the situation as soon as possible.
Closing comments on posts If you are the author of a post tagged “personal blog” (which notably includes all new Bostrom-related posts) and you would like to prevent new comments on your post, please email forum@centerforeffectivealtruism.org and we can disable them.
We know that some posters find the prospect of dealing with commenters so aversive that they choose not to post at all; this seems worse to us than posting with comments turned off.
Working for/with people who are good at those skills seems like a pretty good bet to me.
E.g. “knowing how to attract people to work with you” – if person A has a manager who was really good at attracting people to work with them, and their manager is interested in mentoring, and person B is just learning how to attract people to work with them from scratch at their own startup, I would give very good odds that person A will learn faster.
We are banning stone and their alternate account for one month for messaging users and accusing others of being sock puppets, even after the moderation team asked them to stop. If you believe that someone has violated Forum norms such as creating sockpuppet accounts, please contact the moderators.
First, early advocates of cryonics and MNT focused on writings and media aimed at a broad popular audience, before they did much technical, scientific work. These advocates successfully garnered substantial media attention, and this seems to have irritated the most relevant established scientific communities (cryobiology and chemistry, respectively), both because many of the established scientists felt that something with no compelling scientific backing was getting more attention than their own “real” work, and because some of them (inaccurately) suspected that media attention for cryonics and MNT had translated into substantial (but unwarranted) funding for both fields.
Second, early advocates of cryonics and MNT spoke and wrote in a way that was critical and dismissive toward the most relevant mainstream scientific fields, and this contributed further to tensions between advocates of cryonics and MNT and the established scientific communities from which they could have most naturally recruited scientific talent and research funding.
Third, and perhaps largely as a result of these first two issues, these “neighboring” established scientific communities (of cryobiologists and chemists) engaged in substantial “boundary work” to keep advocates of cryonics and MNT excluded. For example, in the case of cryonics: according to an historical account by a cryonicist26 (who may of course be biased), established cryobiologists organized to repeatedly label cryonicists as frauds until cryonicists threatened a lawsuit; they passed over cryonics-associated cryobiologists for promotions within the professional societies, or asked them to resign from those societies, and they also blocked cryonicists from obtaining new society memberships via amendments to the bylaws; they threatened to boycott the only supplier of storage vessels suitable for cryonics, forcing cryonicists to build their own storage vessels; they wrote a letter to the California Board of Funeral Directors and Embalmers urging them to investigate cryonicists and shut them down; and so on.
I have recently been wondering what my expected earnings would be if I started another company. I looked back at the old 80 K blog post arguing that there is some skill component to entrepreneurship, and noticed that, while serial entrepreneurs do have a significantly higher chance of a successful exit on their second venture, they raise their first rounds at substantially lower valuations. (Table 4 here.)
It feels so obvious to me that someone who’s started a successful company in the past will be more likely to start one in the future, and I continue to be baffled why the data don’t show this.[1]
I have heard some VCs say that they don’t fund serial entrepreneurs because humans only have enough energy to start one company; I always thought this was kind of dumb, but maybe there is some truth to it.
Although the data don’t not show this either. Companies founded by second time entrepreneurs might be less ambitious, raise money earlier, or have some other difference which is consistent with second time entrepreneurs being more successful
Wild guesses as someone that knows very little about this:
I wonder if it’s because people have sublinear returns on wealth, so their second company would be more mission driven and less optimized for making money. Also, there might be some selection bias in who needs to raise money vs being self funded.
But if I had to bet I would say that it’s mostly noise, and there’s not enough data to have a strong prior.
This post points out that brain preservation (cryonics) is potentially quite cheap on a $/QALY basis because people who are reanimated will potentially live for a very long time with very high quality of life.
It seems reasonable to assume that reanimated people would funge against future persons, so I’m not sure if this is persuasive for those who don’t adopt person affecting views, but for those who do, it’s plausibly very cost-effective.
This is interesting because I don’t hear much about person affecting longtermist causes.
I’m interested in EA organizations that can plausibly be said to have improved the lives of over a billion individuals. Ones I’m currently aware of:
Shrimp Welfare Project – they provide this Guesstimate, which has a mean estimate of 1.2B shrimps per year affected by welfare improvements that they have pushed
Aquatic Life Institute – they provide this spreadsheet, though I agree with Bella that it’s not clear where some of the numbers are coming from.
This is a nitpick, but somehow someone “being an individual” reads to me as implying a level of consciousness that seems a stretch for shrimps. But IDK, seems like a non-crazy choice under some worldviews.
That’s fair. I personally like that this forces people to come to terms with the fact that interventions targeted at small animals are way more scalable than those targeted at larger ones. People might decide on some moral weights which cancel out the scale of small animal work, but that’s a nontrivial philosophical assumption, and I like prompting people to think about whether it’s actually reasonable.
I think “animals that have more neurons or are more complex are morally more important” is not a “nontrivial philosophical assumption”.
It indeed strikes me as a quite trivial philosophical assumption the denial of which would I think seem absurd to almost anyone considering it. Maybe one can argue the effect is offset by the sheer number, but I think you will find almost no one on the planet who would argue that these things do not matter.
It indeed strikes me as a quite trivial philosophical assumption the denial of which would I think seem absurd to almost anyone considering it
On the contrary, approximately everyone denies this! Approximately ~0% of Americans think that humans with more neurons than other humans have more moral value, for example.[1]
Come on, you know you are using a hilariously unrepresentative datapoint here. Within humans the variance of neuron count only explains a small fraction of variance in experience and also we have strong societal norms that push people’s map towards pretending differences like this don’t matter.
Unrepresentative of what? At least in my University ethics courses we spent way more time arguing about the rights of anencephalic children or human fetuses than insects. (And I would guess that neuron count explains a large fraction of the variance in experience between adult and fetal humans, for example.)
In any case: I think most people’s moral intuitions are terrible and you shouldn’t learn a ton from the fact that people disagree with you. But as a purely descriptive matter, there are plenty of people who disagree with you – so much so that reading their arguments is a standard part of bioethics 101 in the US.
It’s unrepresentative of the degree to which people believe that corollaries like neuron count and brain size and behavior complexity are an indicator of moral relevance across species (which is the question at hand here).
Yeah definitely. I don’t want to claim that learning is impossible at a startup – clearly it’s possible – just that, all else equal, learning usually happens faster at existing companies.
Animal Justice Appreciation Note
Animal Justice et al. v A.G of Ontario 2024 was recently decided and struck down large portions of Ontario’s ag-gag law. A blog post is here. The suit was partially funded by ACE, which presumably means that many of the people reading this deserve partial credit for donating to support it.
Thanks to Animal Justice (Andrea Gonsalves, Fredrick Schumann, Kaitlyn Mitchell, Scott Tinney), co-applicants Jessica Scott-Reid and Louise Jorgensen, and everyone who supported this work!
Marcus Daniell appreciation note
@Marcus Daniell, cofounder of High Impact Athletes, came back from knee surgery and is donating half of his prize money this year. He projects raising $100,000. Through a partnership with Momentum, people can pledge to donate for each point he gets; he has raised $28,000 through this so far. It’s cool to see this, and I’m wishing him luck for his final year of professional play!
I was lucky enough to see Marcus play this year at the Australian Open, and have pledged alongside him! Marcus is so hardworking—in tennis alongside his work at High Impact Athletes! Go Marcus!!!
New Zealand let’s go!
First in-ovo sexing in the US
Egg Innovations announced that they are “on track to adopt the technology in early 2025.” Approximately 300 million male chicks are ground up alive in the US each year (since only female chicks are valuable) and in-ovo sexing would prevent this.
UEP originally promised to eliminate male chick culling by 2020; needless to say, they didn’t keep that commitment. But better late than never!
Congrats to everyone working on this, including @Robert—Innovate Animal Ag, who founded an organization devoted to pushing this technology.[1]
Egg Innovations says they can’t disclose details about who they are working with for NDA reasons; if anyone has more information about who deserves credit for this, please comment!
For others who were curious about what time difference this makes: looks like sex identification is possible at 9 days after the egg is laid, vs 21 days for the egg to hatch (plus an additional ~2 days between fertilization and the laying of the egg.) Chicken embryonic development is really fast, with some stages measured in hours rather than days.
I asked Google when chicken embryos start to feel pain and this was the first result (i.e. I didn’t look hard and I didn’t anchor on a figure):
How many chicks per year will Egg Innovations’ change save? (The announcement link is blocked for me.)
This interview with the CEO suggests that Egg Innovations are just in the laying (not broiler) business and that each hen produces ~400 eggs over her lifetime. So this will save ~750,000 chicks a year?
I don’t think they say, unfortunately.
Wow this is wonderful news.
Startups aren’t good for learning
I fairly frequently have conversations with people who are excited about starting their own project and, within a few minutes, convince them that they would learn less starting project than they would working for someone else. I think this is basically the only opinion I have where I can regularly convince EAs to change their mind in a few minutes of discussion and, since there is now renewed interest in starting EA projects, it seems worth trying to write down.
It’s generally accepted that optimal learning environments have a few properties:
You are doing something that is just slightly too hard for you.
In startups, you do whatever needs to get done. This will often be things that are way too easy (answering a huge number of support requests) or way too hard (pitching a large company CEO on your product when you’ve never even sold bubblegum before).
Established companies, by contrast, put substantial effort into slotting people into roles that are approximately at their skill level (though you still usually need to put in proactive effort to learn things at an established company).
Repeatedly practicing a skill in “chunks”
Similar to the last point, established companies have a “rhythm” where e.g. one month per year where everyone has a priority of writing up reflections on how the sales cycle is going, commenting on each other’s writeups, and updating your own. Startups do things by the seat of their pants, which means employees are usually rapidly switching between tasks.
Feedback from experts/mentorship
Startup accelerators like YCombinator partially address this, but still a defining characteristic of starting your own project is that you are doing the work without guidance/oversight.
Moreover, even supposing you learn more at a startup, it’s worth thinking about what it actually is you learn. I know way more about the laws regarding healthcare insurance than I did before starting a company, but that knowledge isn’t super useful to me outside the startup context.
This isn’t a 100% universal knockdown argument – some established companies suck for professional development, and some startups are really great. But by default, I would expect startups to be worse for learning.
I think I agree with this. Two things that might make starting a startup a better learning opportunity than your alternative, in spite of it being a worse learning environment:
You are undervalued by the job market (so you can get more opportunities to do cool things by starting your own thing)
You work harder in your startup because you care about it more (so you get more productive hours of learning)
It depends on what you want to learn. At a startup, people will often get a lot more breadth of scope than they would otherwise in an established company. Yes, you might not have in-house mentors or seasoned pros to learn from, but these days motivated people can fill in the holes outside the org.
It depends what you want to learn
As you said.
Founding a startup is a great way to learn how to found a startup.
Working as a backend engineer in some company is a great way to learn how to be a backend engineer in some company.
(I don’t see why to break it up more than that)
Sam Bankman-Fried’s trial is scheduled to start October 3, 2023, and Michael Lewis’s book about FTX comes out the same day. My hope and expectation is that neither will be focused on EA,[1] but several people have recently asked me about if they should prepare anything, so I wanted to quickly record my thoughts.
The Forum feels like it’s in a better place to me than when FTX declared bankruptcy: the moderation team at the time was Lizka, Lorenzo, and myself, but it is now six people, and they’ve put in a number of processes to make it easier to deal with a sudden growth in the number of heated discussions. We have also made a number of design changes, notably to the community section.
CEA has also improved our communications and legal processes so we can be more responsive to news, if we need to (though some of the constraints mentioned here are still applicable).
Nonetheless, I think there’s a decent chance that viewing the Forum, Twitter, or news media could become stressful for some people, and you may want to preemptively create a plan for engaging with that in a healthy way.
This market is thinly traded but is currently predicting that Lewis’s book will not explicitly assert that Sam misused customer funds because of “ends justify the means” reasoning
I’d be surprised [p<0.1] if EA was not a significant focus of the Michael Lewis book – but agree that it’s unlikely to be the major topic. Many leaders at FTX and Alameda Research are closely linked to EA. SBF often, and publically, said that effective altruism was a big reason for his actions. His connection to EA is interesting both for understanding his motivation and as a story-telling element. There are Manifold prediction markets on whether the book would mention 80′000h (74%), Open Philanthropy (74%), and Give Well (80%), but these markets aren’t traded a lot and are not very informative.[1]
This video titled The Fake Genius: A $30 BILLION Fraud (2.8 million views, posted 3 weeks ago) might give a glimpse of how EA could be handled. The video touches on EA but isn’t centred on it. It discusses the role EAs played in motivating SBF to do earning to give, and in starting Alameda Research and FTX. It also points out that, after the fallout at Alameda Research, ‘higher-ups’ at CEA were warned about SBF but supposedly ignored the warnings. Overall, the video is mainly interested in the mechanisms of how the suspected fraud happened, where EA is only one piece of the puzzle. One can equally get a sense of “EA led SBF to do fraud” as “SBF used EA as a front to do fraud”.
ETA:
The book description[2] “mentions “philanthropy”, makes it clear that it’s mainly about SBF and not FTX as a firm, and describes the book as partly a psychological portrait.
I also created a similar market for CEA, but with 2 mentions as the resolving criteria. One mention is very likely as SBF worked briefly for them.
“In Going Infinite Lewis sets out to answer this question, taking readers into the mind of Bankman-Fried, whose rise and fall offers an education in high-frequency trading, cryptocurrencies, philanthropy, bankruptcy, and the justice system. Both psychological portrait and financial roller-coaster ride, Going Infinite is Michael Lewis at the top of his game, tracing the mind-bending trajectory of a character who never liked the rules and was allowed to live by his own—until it all came undone.”
Yeah, unfortunately I suspect that “he claimed to be an altruist doing good! As part of this weird framework/community!” is going to be substantial part of what makes this an interesting story for writers/media, and what makes it more interesting than “he was doing criminal things in crypto” (which I suspect is just not that interesting on its own at this point, even at such a large scale).
The Panorama episode briefly mentioned EA. Peter Singer spoke for a couple of minutes and EA was mainly viewed as charity that would be missing out on money. There seemed to be a lot more interest on the internal discussions within FTX, crypto drama, the politicians, celebrities etc.
Maybe Panorama is an outlier but potentially EA is not that interesting to most people or seemingly too complicated to explain if you only have an hour.
Yeah I was interviewed for a podcast by a canadian station on this topic (cos a canadian hedge fund was very involved). iirc they had 6 episodes but dropped the EA angle because it was too complex.
Good to know, thank you.
Agree with this and also with the point below that the EA angle is kind of too complicated to be super compelling for a broad audience. I thought this New Yorker piece’s discussion (which involved EA a decent amount in a way I thought was quite fair—https://www.newyorker.com/magazine/2023/10/02/inside-sam-bankman-frieds-family-bubble) might give a sense of magnitude (though the NYer audience is going to be more interested in these sort of nuances than most.
The other factors I think are: 1. to what extent there are vivid new tidbits or revelations in Lewis’s book that relate to EA and 2. the drama around Caroline Ellison and other witnesses at trial and the extent to which that is connected to EA; my guess is the drama around the cooperating witnesses will seem very interesting on a human level, though I don’t necessarily think that will point towards the effective altruism community specifically.
Michael Lewis wouldn’t do it as a gotcha/sneer, but this is a reason I’ll be upset if Adam McKay ends up with the movie.
Update: the court ruled SBF can’t make reference to his philanthropy
Yeah, “touches on EA but isn’t centred on it” is my modal prediction for how major stories will go. I expect that more minor stories (e.g. the daily “here’s what happened on day n of the trial” story) will usually not mention EA. But obviously it’s hard to predict these things with much confidence.
This is a huge relief to hear. I noticed some big positive differences, but I couldn’t tell where from. Thank you.
If I understand this correctly, maybe not in the trial itself:
I guess technically the prosecution could still bring it up.
I hadn’t realized that, thanks for sharing
(I forgot to tell JP and Lizka in at EAG in NY a few weeks ago, but now’s as good a time as any):
Can my profile karma total be two numbers, one for community and one for other stuff? I don’t want a reader to think my actual work is valuable to people in proportion to my EA Forum karma, as far as I can tell I think 3-5x my karma is community sourced compared to my object-level posts. People should look at my profile as “this guy procrastinates through PVP on social media like everyone else, he should work harder on things that matter”.
Yeah, I kind of agree that we should do something here; maybe the two-dimensional thing you mentioned, or maybe community karma should count less/not at all.
Could you add a comment here?
Could see a number of potentially good solutions here, but think the “not at all” is possibly not the greatest idea. Creating a separate community karma could lead to a sort of system of social clout that may not be desirable, but I also think having no way to signal who has and has not in the past been a major contributor to the community as a topic would be more of a failure mode because I often use it to get a deeper sense of how to think about the claims in a given comment/post.
There would be some UX ways to make community clout feel lower status than the other clout, I agree with you that having community clout means more investment / should be preferred over a new account which for all you know is a driveby dunk/sneer after wandering in on twitter.
I’ll cc this to my feature request in the proper thread.
Thank you for the prompt.
The Forum moderation team has been made aware that Kerry Vaughn published a tweet thread that, among other things, accuses a Forum user of doing things that violate our norms. Most importantly:
The user in question said this information came from searching LinkedIn for people who had listed themselves as having worked at Leverage and related organizations.
This is not “doxing” and it’s unclear to us why Kerry would use this term: for example, there was no attempt to connect anonymous and real names, which seems to be a key part of the definition of “doxing”. In any case, we do not consider this to be a violation of our norms.
At one point Forum moderators got a report that some of the information about these people was inaccurate. We tried to get in touch with the then-anonymous user, and when we were unable to, we redacted the names from the comment. Later, the user noticed the change and replaced the names. One of CEA’s staff asked the user to encode the names to allow those people more privacy, and the user did so.
Kerry says that a former Leverage staff member “requests that people not include her last name or the names of other people at Leverage” and indicates the user broke this request. However, the post in question requests that the author’s last name not be used in reference to that post, rather than in general. The comment in question doesn’t refer to the former staff member’s post at all, and was originally written more than a year before the post. So we do not view this comment as disregarding someone’s request for privacy.
Kerry makes several other accusations, and we similarly do not believe them to be violations of this Forum’s norms. We have shared our analysis of these accusations with Leverage; they are, of course, entitled to disagree with us (and publicly state their disagreement), but the moderation team wants to make clear that we take enforcement of our norms seriously.
We would also like to take this opportunity to remind everyone that CEA’s Community Health team serves as a point of contact for the EA community, and if you believe harassment or other issues are occurring we encourage you to reach out to them.
How I wish the EA Forum had responded
I’ve found that communicating feedback/corrections often works best when I write something that approximates what I would’ve wished the other person had originally written.
Because of the need to sync more explicitly on a number of background facts and assumptions (and due to not having time for edits/revisions), my draft is longer than I think a moderator’s comment would need to be, were the moderation team to be roughly on the same page about the situation. While I am the Cathleen being referenced, I have had minimal contact with Leverage 2.0 and the EA Forum moderation team, so I expect this draft to be imperfect in various ways, while still pointing at useful and important parts of reality.
Here I’ve made an attempt to rewrite what I wish Ben West had posted in response to Kerry’s tweet thread:
The Forum moderation team has been made aware that Kerry Vaughn published a tweet thread that, among other things, accuses a Forum user of doing things that violate our norms. Most importantly:
We care a lot about ensuring that the EA Forum is a welcoming place where people are free to discuss important issues related to world improvement. While disagreement and criticism are an important part of that, we want to be careful not to allow for abuse to take place on our platform, and so we take such reports seriously.
After reviewing the situation, we have compiled the following response (our full review is still in process but we wanted to share what we have so far while the issue is live):
While Leverage was not a topic that we had flagged as “sensitive” back in Sept 2019 when the then-anonymous user originally made his post, the subsequent discussion around the individuals and organizations who were part of the Leverage/Paradigm ecosystem prior to its dissolution in June 2019 has led it to be classified as a sensitive topic to which we expend more scrutiny and are more diligent about enforcing our norms.
In reviewing the particular post referenced above, we found a number of odd elements:
In posing and then answering his own “Question” on the EA Forum, the user makes his accusation of Paradigm (the org) being a front or a secret replacement for Leverage (the org) despite having previously acknowledged the recent dissolution of the Leverage/Paradigm ecosystem (and in a context where the two organizations were publicly known to be closely connected).
The user largely acts as if he is sharing work history discovered on LinkedIn as his primary argument despite only 2 of the 13 named individuals having Leverage listed on their LinkedIn profiles.
The user names 4 additional people as having worked at Leverage despite citing no evidence of that fact (they did not have Leverage on their LinkedIn profiles).
The user then names 7 additional individuals who did not have Leverage on their LinkedIn profiles and who the user also did not believe to have originally worked at Leverage.
The user neglects to include any context or timelines from LinkedIn, e.g. whether there had been a recent change in work history at the point of the ecosystem dissolution, whether the individuals in question started at Leverage and then moved to Paradigm, started at Paradigm and then moved to Leverage, or started at either Leverage or Paradigm and then moved on to other projects. It is also unclear which, if any, continued to work at either Leverage or Paradigm after the ecosystem dissolution.
More than 2 years later, after reading a clarifying and detailed post about the relevant history of Leverage and Paradigm (which included a request to protect people’s identities) and discovering that the Forum mods had removed the named individuals from his comment, the only edit the user decided to make was to deanonymize the names he’d compiled of people previously affiliated with the Leverage ecosystem.
When this post was initially brought to our attention in July of 2020, along with an explanation of possible negative consequences for the people listed (including ~4 individuals who the user was spreading potential misinformation about), we tried to get in touch with the then-anonymous user, and when we were unable to, we redacted the names from the comment and left an explanation for how the comment could make its point without using the personal information of the named individuals.
At the time, we had been informed that the user was mistaken about the work history for some of the people he listed, in large part due to relying on his incorrect personal assumptions. We did not consider the way that some of the 4 (and possibly others) might’ve intentionally excluded Leverage from their work histories, as we were focused on the ones who were incorrectly identified as having worked at Leverage and the potential consequences of that misinformation. Without yet knowing or investigating the full extent of the anonymous user’s posting history across multiple accounts, we did not suspect a pattern of hostile posts. Because of these factors, we did not evaluate whether this post might be a case of doxing.
In Dec 2021, Cathleen, one of the 4 who had been listed as working at Leverage (despite no record on LinkedIn), published a detailed account of her experience at Leverage/Paradigm. In it, she shared her perspective on harassment and ill-will that had come from the EA and Rationalist community members, and the negative effects of misinformation spread via public community forums. She explained why she had intentionally excluded Leverage from her LinkedIn many years prior and asked that people protect her identity as well as the identities of others from the Leverage ecosystem due to the risk of cancellation and harassment.
A few days later, the EA Forum user (who had revealed his real identity a couple months prior) returned to his anonymous post from Sept 2019 and deanonymized the first and last names of all 13 individuals he had previously named. This included Cathleen as well as the other 3 individuals who he attributed to Leverage (despite no record on LinkedIn). He accompanied the edits with a false/misleading comment (using the anonymous account) minimizing the substantive merit of prior requests for corrections to his post and claiming that all of the relevant information had actually been originally drawn from LinkedIn.
(At a cursory glance, it’s difficult to determine the most natural interpretation of the scope of Cathleen’s request, in order to assess the likelihood that the user was knowingly violating her wishes. We initially had a quite narrow interpretation, reading the quote out of context, but I think the situation becomes clearer if you take the time to read her entire post or the section that the quote was pulled from, entitled “We want to move forward with our lives (and not be cancelled)”, which includes a direct reference to LinkedIn/her intent to keep her work history private.)
After receiving a new complaint about the potential harm of listing individuals’ names, including the spread of misinformation caused by the user’s updated post, we reviewed the case and this time found no violation. As a compromise, we did offer to ask the user if they would be willing to encode the full names to help protect the individuals from potential negative effects arising from a simple google search.
We have since realized that many people (including people on our moderation team) took the user at his word without carefully reviewing the post. We had become confused about the specifics (falsely believing that he was sharing publicly available information from LinkedIn and thus believing that the information could be reasonably treated as true and that the objections raised were splitting hairs). We also did not accurately recognize the general nature/intent of his original post nor the potential negative effects of allowing the information to stand, and we did not evaluate the deanonymizing edits in the context of Cathleen’s recent public request and voiced concerns.
In Oct 2021, when the user had revealed his identity and his use of multiple anonymous accounts, we also failed to review the complete body of evidence and the ways that his actions had potentially violated our norms (e.g. using multiple anonymous accounts to convey similar views on a topic), as well as notice that the full pattern of posts indicated a type of ill-will that we discourage and that is especially relevant given the sensitive nature of the topic of Leverage/Paradigm.
In retrospect, we recognize that while we would like to give users the benefit of the doubt, when there are complaints of doxing, harassment, or other poor behavior, it makes sense for us to look more carefully at the situation and potentially draw on CEA’s Community Health team’s expertise in assisting individuals who flag that they’ve been wronged by a user on the Forum.
Something else we did not consider (because we unfortunately don’t have the bandwidth to track all the goings on in the EA and Rationalist communities) is that the level of threat experienced by people who had previously been part of the Leverage ecosystem had become quite high. In evaluating cases of disclosing private information or even assembling and publishing public information, context matters.
As an example:
On the face of it, it seems fine to have openly communist views or be LGBTQ, but history has shown us that during certain eras, e.g. the Second Red Scare in the US, creating and posting lists of such people (even if true or otherwise individually knowable) would likely subject them to harassment or worse.
It is not an excuse if someone else could create a similarly damaging list, and it doesn’t seem right to ask people to hide their work history from potential employers on a professional networking platform for the sole purpose of protecting themselves from being subject to defamatory public posts from ill-willed and/or ill-informed EAs and Rationalists. It is already unfortunate that the damage to the reputation of the relevant orgs has made it difficult for individuals to decide when and how to associate themselves with their former projects.
Guilt by association is not a good faith argument here, and at a minimum, it seems reasonable to honor individual wishes for Forum users to refrain from using their full names in affiliation with prior projects when requested (and to be careful not to do so in a way that falsely implies that the person is (or should be) ashamed of their affiliation).
After reviewing the overall situation, we think it’s important for users as well as moderators to recognize that posts about former Leverage/Paradigm staff do not happen in a vacuum. We strongly condemn the sharing of information about an individual’s prior professional or social affiliation in a way that intentionally or negligently exposes them to undue negative consequences.
If people are proud of their work at an organization but feel the need to disassociate themselves from that org publicly, it seems like something has gone wrong.
Given the overall pattern of posts from this user’s accounts across the EA Forum as well as Less Wrong from 2018-2021, it seems plausible to our team that the actions of this user may have actually been a significant contributing factor in fomenting negative sentiment towards this group of people. In light of that, making the decision to list their full names in this comment in 2019 and then editing the comment to include them again in 2021, after Cathleen’s detailed post (which includes information relevant to the comment’s hypothesis as well as a request and argument for privacy), we find it harder to defend an interpretation where there was not an intent to cause harm to the named individuals.
Further, it is our understanding that only a fraction of those named worked for Leverage or Paradigm at the time of the original post, and only ~1 to 2 of those named worked for Leverage or Paradigm at the time of the subsequent deanonymizing edit; given that, and with the assumption that the argument for both relevancy of the post on the EA Forum as well as the argument for wrongdoing relies solely on the pattern of employment at these orgs, the weighing of potential benefit/value vs. cost/harm to prior project members seems particularly clear.
Intimidation and harassment can be executed in subtle ways, and while intent can be hard to ascertain, we encourage participants on the Forum to put in extra effort to ward against their posts landing in a gray area.
We don’t think that every case of bad behavior needs to fit neatly into our listed norms (and in evaluating cases like these, we also think it makes sense to revisit our listed norms to see if we should make changes for clarity or scope)[1], but it seems clear to us that the type of behavior exhibited by this user across their anonymous accounts is neither generous nor collaborative and it also seems likely to interfere with good discourse (not least of all by creating a hostile environment for some members of the community).
While we wish we would’ve done better, given our knowledge at the time, we don’t see this as a major failing, but we do recognize the harm that was caused and we want to emphasize that the use of anonymous accounts to harass individuals or groups is not something that we tolerate.
We have referred this case to the CEA Community Health team for further review. They will look at the totality of content from this user on this and related topics, examining patterns, severity, as well as the time period spanned by relevant posts and comments on the EA Forum as well as LW, as a way of assessing potential and actual negative impacts and intent. With the permission of relevant parties, they will also review registered complaints about the user’s behavior. With their input, we will deliberate further and decide whether there is mitigating action that the Forum moderators can and should take in this particular case.
If you have a world improvement related issue that you believe needs public attention but aren’t sure how to navigate it while minimizing unnecessary harm, we encourage you to reach out to the CEA Community Health team who can help organize your thoughts and perhaps mediate a discussion where more information can be exchanged before escalating to a public post. We recognize that in a situation where you suspect a conspiracy or are otherwise suspicious of others’ actions, it may be harder to prioritize the discussion norms of the Forum, but it is in those moments that the norms are most important to respect.
*(The Forum moderators are currently grappling with an issue that may be relevant to situations involving sensitive topics like these: it does not violate our norms to inadvertently publish false or misleading information – but in the case that a correction or material clarification is made and the OP doesn’t update their post or comment, an argument could be made that the user is either in violation of the norm of scout mindset/willingness to update their view, or (if they do update their understanding but don’t update their post) they could be in violation of knowingly/deliberately spreading misinformation. We generally have not wanted to act as the arbiters of truth, so it’s not yet clear how to best moderate a situation like this.)
To share a brief thought, the above comment gives me a bad juju because it puts a contested perspective into a forceful and authoritative voice, while being long enough that one might implicitly forget that this is a hypothetical authority talking[1]. So it doesn’t feel to me like a friendly conversational technique. I would have preferred it to be in the first person.
Garcia Márquez has a similar but longer thing going on in The Handsomest Drowned Man In The World, where everything after “if that magnificent man had lived in the village” is a hypothetical.
(fwiw I didn’t mind the format and felt like this was Cathleen engaging in good faith.)
I would have so much respect for CEA if they had responded like this.
Hi Ben,
Thank you for your attention to this matter. I must confess that I found the response confusing as you seem to be responding to claims slightly (though importantly!) different from the ones I made in my Twitter post.
Specifically, my view is (1) you are mistaken about the definition of doxing and have instead provided a definition of deanonymization; (2) this person’s behavior meets the definitions of doxing; (3) norms that allow these posts would also allow more serious abuses.
I’d also like to clarify that I am not at the moment accusing you or the EA Forum moderators of failing to enforce EA Forum rules or norms in this case. I am not deeply familiar with the EA Forum rules or norms, so I am indifferent on this question.
This poster violated the norms that I think would be reasonable for a high-quality discussion forum. I also think this poster violated standards of reasonable behavior for a person with high-status membership in an intellectual community. I leave the question of EA Forum rules and norms to you and your team.
Additionally, while I alluded to but did not directly discuss this in the Twitter thread, the EA Forum moderation team took several actions that lessened the harm this poster intended to cause. While I think further actions are warranted, I would nevertheless like to express my gratitude to the team for what they have already done.
On the definition of doxing
This is not the definition of doxing.
A simple internet search reveals the following definitions:
Wikipedia: “Doxing or doxxing is the act of publicly providing personally identifiable information about an individual or organization, usually via the Internet. . . . Doxing may be carried out for reasons such as online shaming, extortion, and vigilante aid to law enforcement”
Urban Dictionary: “Using private information gleaned from the internet to attack someone with whom you disagree, often by publishing their personal info, opening them to abuse and possibly, danger.”
What you provide is the definition of the much narrower activity of deanonymization. I agree that the poster’s behavior is not deanonymization. I never indicated otherwise.
Deanonymization is neither a necessary nor sufficient condition for doxing. It is possible to dox someone even if they maintain no anonymous identities (e.g., by publishing non-public information that does not pertain to an anonymous identity for the purposes of harassment). It is also possible to reveal a person’s anonymous identity without doxing them if the reveal involves neither ill intent nor ill effects.
Why the poster’s behavior is doxing
Doxing has two important components: (1) revealing private information; (2) with ill-intent. Below I argue that the poster’s behavior meets both of these standards.
The information was private
In the EA Forum comment where the doxing occurred, the poster says the following (emphasis mine):
The term “well-known” indicates that the information is not on their LinkedIn profiles, but is known to the poster. Thus, they relied on non-public information to associate the people with the organizations.
However, even if the poster had used only information from LinkedIn profiles and not their own knowledge, this would still constitute a reveal of private information because “private information” is context-dependent. Almost all canonical cases of doxing involve taking information that is findable on the public internet (albeit with difficulty) and then revealing that information in an adversarial context.
Consider, for example, the high-profile case of the doxing of Scott Alexander. Scott Alexander’s real name was not private in the sense of “not findable on the public internet.” Indeed, it could be revealed with a little bit of dedicated Googling. It was also widely known inside the EA and Rationality community. The Times’ threat to publish his real name was nevertheless a threat to dox him because it was not well-known to the readership of The Times.
Who was employed at Leverage and Paradigm was not well-known to the members of the EA community. Most of those who worked at Leverage and Paradigm had little interaction with the EA community. Thus, the information revealed was private in the relevant sense.
The poster acted with ill-intent
There are several ways to see that the poster acted with ill intent.
(1) The social context of this post is that there was a deterioration of the relationship between the Effective Altruism and Leverage Research communities. This is easy to see by reviewing the comments on the “Basic Facts” post, which are substantially negative. While one might dispute that the negativity is fair, accurate, or deserved, it was not considered so by members of Leverage or Paradigm staff. Thus, the EA Forum is an adversarial context from the perspective of Leverage and Paradigm employees.
(2) A review of the posting history of the anonymoose and throwaway accounts indicates that their primary goal was to spread negative information about Leverage and Paradigm.
(3) The context in which the people are doxed is in response to the following “question:” “To what extent is Paradigm Academy a front organization for, or a covert rebrand of Leverage Research?”
This is, of course, a quite blatant attempt to insulate nefariousness where none is warranted. There was never any attempt to conceal the relationship between Leverage and Paradigm by the staff of either organization. No information to indicate otherwise is ever presented by the poster.
(4) There is simply no reason to include the people’s names except for the purposes of doxing. The point of the comment is that there is a heavy association between people that worked at Paradigm and those that worked at Leverage. There is no reason that specific names need to be included to make this point.
(5) The post violated the letter and spirit of Cathleen’s request.
On this point, Ben indicates the following:
However, the full request by Cathleen is as follows (emphasis mine):
As there is some confusion on this point, it is important to be clear. The central complaint in the Twitter thread is that *5 days* after Cathleen’s post, the poster edited their comment to add the names of Leverage and Paradigm employees back to the comment, including Cathleen’s last name. This violates Cathleen’s request.
Ben is correct that the comment originally occurred substantially before Cathleen’s post. But this is immaterial. The point is that the poster re-added the names after the request had been made.
Norms that allow these posts would also allow more serious abuses
A further issue with this understanding of EA norms is that it would allow more serious abuses on the EA Forum. If this is not clear, I can provide some examples.
If you list your organizational affiliation on LinkedIn (and if you are indeed correct that Paradigm Academy was not trying to be a cover for Leverage Research that shielded it from public scrutiny), then I don’t think you get to complain when someone quickly googles you and finds your LinkedIn profile.
Like, if Scott Alexander had listed his real name on his about page, or on LinkedIn, connected to his pseudonym, then yeah, I would have also had very little sympathy for the objections to the New York Times. There clearly are degrees of what kind of connections you can make here, and searching an organization on LinkedIn really does not cross any obvious line here.
Maybe you are objecting to drawing a connection between Leverage Research and Paradigm Academy, but you are also saying that nobody was saying that Paradigm Academy was intentionally trying to not appear to be part of what was broadly known as “Leverage”, so I don’t understand how you can make both of these arguments at the same time.
First, as I noted in my response to Ben, some of the information included in the doxing was from the poster’s personal knowledge and not from the people’s LinkedIn profiles. Thus, you can’t defend the doxing by saying that the information was publicly available. It simply wasn’t.
Second, I am not complaining about someone quickly googling anyone and finding their LinkedIn profile. I am complaining about making it easier to harass people by posting a central repository of information about those people on a forum frequented by people with a history of harassing them.
It is certainly the case that former Leveragers took the release of their names on the EA forum as an attempt to invite harassment by members of the EA community. We were contacted by former Leveragers to see if there was anything we could do to get the names removed for this very reason.
I suspect the actual crux of the discussion here is that most EAs are unaware of the history of poor and bizarre behavior towards Leverage/Paradigm by members of the EA community. To help with the knowledge gap, I’m considering writing a Twitter thread that will share some of the most egregious examples.
Kerry, what are you getting at? What do you think was the harm or the intended harm of Ryan’s posts?
At this point, the moderators are trying to focus on any practical steps we should take. Given that the names in question are encoded, and no one currently listed in the comment has reached out to us, we do not plan to take further action. Anyone who feels that private or incorrect information about themselves is posted on the Forum is always free to contact us.
Hi Ben,
Could you explain what’s not practical about these simple steps that you could take:
1) Create an EA Forum policy against sockpuppeting, and apply it retroactively to this case. This might naturally result in deleting the offending posts or adding a notification indicating they are sockpuppets.
2) Remove the encoded names and replace them with numbers per the edit your team made originally.
3) Change the word “doxing” to “deanonymizing” in the following sentence from the Guide to Norms (since the behavior you intend to prohibit is not doxing):
[Disclaimer, I have very little context on this & might miss something obvious and important]
AFAICT the disagreement between Kerry and Ben stems from interpreting the second part of Cathleen’s ask differently. There seem to be two ways of reading the second part:
1. Asking people to refrain from naming others who are not already tied into this in general.
2. Asking people to refrain from naming others who are not already tied into this in discussions of her post.
To me, it seems pretty clear that she means the latter, given the structure of the two sentences. If she was aiming for the first interpretation, I think she should have used a qualifier like “in general” in the second sentence. In the current formulation, the “And” at the beginning of the sentence connects the first ask/sentence very clearly to the second.
I guess this can be up for debate, and one could interpret it differently, but I would certainly not fault anyone for going with interpretation 2.[1]
If we assume that 2 is the correct reading, Kerry’s claim (cited above) does not seem relevant anymore / Ben’s original remark (cited below) seems correct. The timeline of edits doesn’t change things. Ben’s original remark (emphasis mine):
Even if she meant interpretation 1, it is unclear to me that this would be a request that I would endorse other people enforcing. Her request in interpretation 2 seems reasonable, in part because it seems like an attempt to avoid people using her post in a way she doesn’t endorse. A general “don’t associate others with this organisation” would be a much bigger ask. I would not endorse other organisations asking the public not to connect its employees to them (e.g. imagine a GiveWell employee making the generic ask not to name other employees/collaborators in posts about GiveWell), and the Forum team enforcing that.
I disagree with that interpretation especially given the context of Cathleen’s post. It includes lengthy discussions of poor and bizarre behavior by members of the EA community toward Leverage/Paradigm staff.
I think reading Cathleen’s post and then re-adding her name is either an intentional violation of her wishes or, at a minimum, shows a reckless disregard for her request for privacy.
In any case, I don’t think this issue is central. The original comment already was doxing given the context, which includes the “question” to which it was a reply, the poster’s overall pattern of behavior, and the history of very poor behavior towards Leverage/Paradigm staff by members of the EA community. The poster’s behavior after reading Cathleen’s request for privacy simply makes their intention clearer.
In case it’s helpful and you’ve not read them, I think that the two main pieces of context that would be helpful are Cathleen’s original post and the Twitter thread of Kerry Vaughan’s that Ben West is referring to.
Cathleen has also now commented above which I think clarifies how her request was meant to be taken. Here’s a quote (as context, she’s formatted her comment to be an idea for how CEA might have responded instead):
Hi Ben,
Another question I wanted to ask is whether the EA Forum has rules against creating sock puppet accounts. This is defined as follows:
Wikipedia: “Online, it came to be used to refer to a false identity assumed by a member of an internet community who spoke to, or about, themselves while pretending to be another person.”
Urban dictionary: “A false identity adopted by trolls and other malcontents to support their own postings.”
I ask because it appears to me that the poster acted deceptively in managing their multiple anonymous accounts by (1) using a second anonymous account to reply to their first anonymous account; (2) promoting the actions of their anonymous account from their public account without disclosing that they were behind the anonymous account.
Most forums I’ve been a part of have rules against sockpuppeting, but reviewing the EA Forum rules and guidelines, I did not see anything about that.
Note that the two anonymous accounts are spaced a year apart; seems likely that Ryan lost the password to the 1st.
The obvious thing to do in that case is to disclose this in a comment somewhere.
Also, this defense doesn’t work for his efforts to promote his own anonymous posts from his public account.
Possible Vote Brigading
We have received an influx of people creating accounts to cast votes and comments over the past week, and we are aware that people who feel strongly about human biodiversity sometimes vote brigade on sites where the topic is being discussed. Please be aware that voting and discussion about some topics may not be representative of the normal EA Forum user base.
Huh, seems like you should just revert those votes, or turn off voting for new accounts. Seems better than just having people be confused about vote totals.
And maybe add a visible “new account” flag—I understand not wanting to cut off existing users creating throwaways, but some people are using screenshots of forum comments as evidence of what EAs in general think.
Arguably also beneficially if you thought that we should typically make an extra effort to be tolerant of ‘obvious’ questions from new users.
Thanks! Yeah, this is something we’ve considered, usually in the context of trying to make the Forum more welcoming to newcomers, but this is another reason to prioritize that feature.
I agree.
Yeah, I think we should probably go through and remove people who are obviously brigading (eg tons of votes in one hour and no other activity), but I’m hesitant to do too much more retroactively. I think it’s possible that next time we have a discussion that has a passionate audience outside of EA we should restrict signups more, but that obviously has costs.
When you purge user accounts you automatically revoke their votes. I wouldn’t be very hesitant to do that.
How do you differentiate someone who is sincerely engaging and happens to have just created an account now from someone who just wants their viewpoint to seem more popular and isn’t interested in truth seeking?
Or are you saying we should just purge accounts that are clearly in the latter category, and accept that there will be some which are actually in the latter category but we can’t distinguish from the former?
I think being like “sorry, we’ve reverted votes from recently signed-up accounts because we can’t distinguish them” seems fine. Also, in my experience abusive voting patterns are usually very obvious, where people show up and only vote on one specific comment or post, or on content of one specific user, or vote so fast that it seems impossible for them to have read the content they are voting on.
How about: getting a lot of downvotes from new accounts doesn’t decrease your voting-power and doesn’t mean your comments won’t show up on the frontpage?
Half a dozen of my latest comments have responded to HBDers. Since they get a notification it doesn’t surprise me that those comments get immediate downvotes which hides them from the frontpage and subsequently means that they can easily decrease my voting-power on this forum (it went from 5 karma for a strong upvote to now 4 karma for a strong upvote).
Giving brigaders the power to hide things from the frontpage and decide which people have more voting-power on this forum seems undesirable.
Note: I went through Bob’s comments and think it likely they were brigaded to some extent. I didn’t think they were in general excellent, but they certainly were not negative-karma comments. I strong-upvoted the ones that were below zero, which was about three or four.
I think it is valid to use the strong upvote as a means of countering brigades, at least where a moderator has confirmed there is reason to believe brigading is active on a topic. My position is limited to comments below zero, because the harmful effects of brigades suppressing good-faith comments from visibility and affirmatively penalizing good-faith users are particularly acute. Although there are mod-level solutions, Ben’s comments suggest they may have some downsides and require time, so I feel a community corrective that doesn’t require moderators to pull away from more important tasks has value.
I also think it is important for me to be transparent about what I did and accept the community’s judgment. If the community feels that is an improper reason to strong upvote, I will revert my votes.
Edit: is to are
I agree.
Could you set a minimum karma threshold (or account age or something) for your votes to count? I would expect even a low threshold like 10 would solve much of the problem.
Yeah, interesting. I think we have a lot of lurkers who never get any karma and I don’t want to entirely exclude them, but maybe some combo like “10 karma or your account has to be at least one week old” would be good.
Yeah I think that would be a really smart way to implement it.
Do the moderators think the effect of vote brigading reflect support from people who are pro-HBD or anti-HBD?
Thoughts on the OpenAI Board Decisions
A couple months ago I remarked that Sam Bankman-Fried’s trial was scheduled to start in October, and people should prepare for EA to be in the headlines. It turned out that his trial did not actually generate much press for EA, but a month later EA is again making news as a result of recent Open AI board decisions.
A couple quick points:
It is often the case that people’s behavior is much more reasonable than what is presented in the media. It is also sometimes the case that the reality is even stupider than what is presented. We currently don’t know what actually happened, and should hold multiple hypotheses simultaneously.[1]
It’s very hard to predict the outcome of media stories. Here are a few takes I’ve heard; we should consider that any of these could become the dominant narrative.
Vinod Khosla (The Information): “OpenAI’s board members’ religion of ‘effective altruism’ and its misapplication could have set back the world’s path to the tremendous benefits of artificial intelligence”
John Thornhill (Financial Times): One entrepreneur who is close to OpenAI says the board was “incredibly principled and brave” to confront Altman, even if it failed to explain its actions in public. “The board is rightly being attacked for incompetence,” the entrepreneur told me. “But if the new board is composed of normal tech people, then I doubt they’ll take safety issues seriously.”
The Economist: “The chief lesson is the folly of policing technologies using corporate structures … Fortunately for humanity, there are bodies that have a much more convincing claim to represent its interests: elected governments”
The previous point notwithstanding, people’s attention spans are extremely short, and the median outcome of a news story is ~nothing. I’ve commented before that FTX’s collapse had little effect on the average person’s perception of EA, and we might expect a similar thing to happen here.[2]
Animal welfare has historically been unique amongst EA causes in having a dedicated lobby who is fighting against it. While we don’t yet have a HumaneWatch for AI Safety, we should be aware that people have strong interests in how AI develops, and this means that stories about AI will be treated differently from those about, say, malaria.
It can be frustrating to feel that a group you are part of is being judged by the actions of a couple people you’ve never met nor have any strong feelings about. The flipside of this though is that we get to celebrate the victories of people we’ve never met. Here are a few things posted in the last week that I thought were cool:
The Against Malaria Foundation is in the middle of a nine-month bed net distribution which is expected to prevent 20 million cases of malaria, and about 40,000 deaths. (Rob Mather)
The Shrimp Welfare Project signed an agreement to prevent 125 million shrimps per year from having their eyes cut off and other painful farming practices. (Ula Zarosa)
The Belgian Senate voted to add animal welfare to their Constitution. (Bob Jacobs)
Scott Alexander’s recent post also has a nice summary of victories.
A collection of prediction markets about this event can be found here.
Note that the data collected here does not exclude the possibility that perception of EA was affected in some subcommunities, and it might be the case that some subcommunities (e.g. OpenAI staff) do have a changed opinion, even if the average person’s opinion is unchanged
Just for the record, I think the evidence you cited there was shoddy, and I think we are seeing continued references to FTX in basically all coverage of the OpenAI situation, showing that it did clearly have a lasting effect on the perception of EA.
Reputation is lazily-evaluated. Yes, if you ask a random person on the street what they think of you, they won’t know, but when your decisions start influencing them, they will start getting informed, and we are seeing really very clear evidence that when people start getting informed, FTX is heavily influencing their opinion.
Thanks! Could you share said evidence? The data sources I cited certainly have limitations, having access to more surveys etc. would be valuable.
The Wikipedia page on effective altruism mentions Bankman-Fried 11 times, and after/during the OpenAI story, it was edited to include a lot of criticism, ~half of which was written after FTX (e.g. it quotes this tweet https://twitter.com/sama/status/1593046526284410880 )
It’s the first place I would go to if I wanted an independent take on “what’s effective altruism?” I expect many others to do the same.
There are a lot of recent edits on that article by a single editor, apparently a former NY Times reporter (the edit log is public). From the edit summaries, those edits look rather unfriendly, and the article as a whole feels negatively slanted to me. So I’m not sure how much weight I’d give that article specifically.
Sure, here are the top hits for “Effective Altruism OpenAI” (I did no cherry-picking, this was the first search term that I came up with, and I am just going top to bottom). Each one mentions FTX in a way that pretty clearly matters for the overall article:
Bloomberg: “What is Effective Altruism? What does it mean for AI?”
“AI safety was embraced as an important cause by big-name Silicon Valley figures who believe in effective altruism, including Peter Thiel, Elon Musk and Sam Bankman-Fried, the founder of crypto exchange FTX, who was convicted in early November of a massive fraud.”
Reddit “I think this was an Effective Altruism (EA) takeover by the OpenAI board”
Top comment: ” I only learned about EA during the FTX debacle. And was unaware until recently of its focus on AI. Since been reading and catching up …”
WSJ: “How a Fervent Belief Split Silicon Valley—and Fueled the Blowup at OpenAI”
“Coming just weeks after effective altruism’s most prominent backer, Sam Bankman-Fried, was convicted of fraud, the OpenAI meltdown delivered another blow to the movement, which believes that carefully crafted artificial-intelligence systems, imbued with the correct human values, will yield a Golden Age—and failure to do so could have apocalyptic consequences.”
Wired: “Effective Altruism Is Pushing a Dangerous Brand of ‘AI Safety’”
“EA is currently being scrutinized due to its association with Sam Bankman-Fried’s crypto scandal, but less has been written about how the ideology is now driving the research agenda in the field of artificial intelligence (AI), creating a race to proliferate harmful systems, ironically in the name of “AI safety.”
Semafor: “The AI industry turns against its favorite philosophy”
“The first was caused by the downfall of convicted crypto fraudster Sam Bankman-Fried, who was once among the leading figures of EA, an ideology that emerged in the elite corridors of Silicon Valley and Oxford University in the 2010s offering an alternative, utilitarian-infused approach to charitable giving.”
Ah yeah sorry, the claim of the post you criticized was not that FTX isn’t mentioned in the press, but rather that those mentions don’t seem to actually have impacted sentiment very much.
I thought when you said “FTX is heavily influencing their opinion” you were referring to changes in sentiment, but possibly I misunderstood you – if you just mean “journalists mention it a lot” then I agree.
You are also welcome to check Twitter mentions or do other analysis of people talking publicly about EA. I don’t think this is a “journalist only” thing. I will take bets you will see a similar pattern.
I actually did that earlier, then realized I should clarify what you were trying to claim. I will copy the results in below, but even though they support the view that FTX was not a huge deal I want to disclaim that this methodology doesn’t seem like it actually gets at the important thing.
But anyway, my original comment text:
As a convenience sample I searched twitter for “effective altruism”. The first reference to FTX doesn’t come until tweet 36, which is a link to this. Honestly it seems mostly like a standard anti-utilitarianism complaint; it feels like FTX isn’t actually the crux.
In contrast, I see 3 e/acc-type criticisms before that, two “I like EA but this AI stuff is too weird” things (including one retweeted by Yann LeCun??), two “EA is tech-bro/not diverse” complaints and one thing about Whytham Abbey.
And this (survey discussed/criticized here):
I just tried to reproduce the Twitter datapoint. Here is the first tweet when I sort by most recent:
Most tweets are negative, mostly referring to the OpenAI thing. Among the top 10 I see three references to FTX. This continues to be quite remarkable, especially given that it’s been more than a year, and these tweets are quite short.
I don’t know what search you did to find a different pattern. Maybe it was just random chance that I got many more than you did.
I used the default sort (“Top”).
(No opinion on which is more useful; I don’t use Twitter much.)
Top was mostly showing me tweets from people that I follow, so my sense is it was filtered in a personalized way. I am not fully sure how it works, but it didn’t seem the right type of filter.
Yeah, makes sense. Although I just tried doing the “latest” sort and went through the top 40 tweets without seeing a reference to FTX/SBF.
My guess is that this filter just (unsurprisingly) shows you whatever random thing people are talking about on twitter at the moment, and it seems like the random EA-related thing of today is this, which doesn’t mention FTX.
Probably you need some longitudinal data to have this be useful.
I would guess too that these two events have made it much easier to reference EA in passing. eg I think this article wouldn’t have been written 18 months ago. https://www.politico.com/news/2023/10/13/open-philanthropy-funding-ai-policy-00121362
So I think there is a real jump of notoriety once the journalistic class knows who you are. And they now know who we are. “EA, the social movement involved in the FTX and OpenAI crises” is not a good epithet.
Upvoted, I’m grateful for the sober analysis.
I think this is an oversimplification. This effect is largely caused by competing messages; the modern internet optimizes information for memetic fitness e.g. by maximizing emotional intensity or persuasive effect, and people have so much routine exposure to stuff that leads their minds around in various directions that they get wary (or see having strong reactions to anything at all as immature, since a large portion of outcries on the internet are disproportionately from teenagers). This is the main reason why people take things with a grain of salt.
However, overton windows can still undergo big and lasting shifts (this process could also be engineered deliberately long before generative AI emerged, e.g. via clown attacks which exploit social status instincts to consistently hijack any person’s impressions of any targeted concept). The 80,000 hours podcast with Cass Sunstein covered how Overton windows are dominated by vague impressions of what ideas are acceptable or unacceptable to talk about (note: this podcast was from 2019). This dynamic could plausibly strangle EA’s access to fresh talent, and AI safety’s access to mission-critical policy influence, for several years (which would be far too long).
On the flip side, johnswentworth actually had a pretty good take on this; that the human brain is instinctively predisposed to over-focus on the risk of their in-group becoming unpopular among everyone else:
Thanks for the helpful comment – I had not seen John’s dialogue and I think he is making a valid point.
Fair point that the lack of impact might not be due to attention span but instead things like having competing messages.
In case you missed it: Angelina Li compiled some growth metrics about EA here; they seem to indicate that FTX’s collapse did not “strangle” EA (though it probably wasn’t good).
Plant-based burgers now taste better than beef
https://www.sciencedirect.com/science/article/abs/pii/S0963996923003587
Interesting! Some thoughts:
I wonder if the preparation was “fair”, and I’d like see replications with different beef burgers. Maybe they picked a bad beef burger?
Who were the participants? E.g. students at a university, and so more liberal-leaning and already accepting of plant-based substitutes?
Could participants reliably distinguish the beef burger and the animal-like plant-based burger in the blind condition?
(I couldn’t get access to the paper.)
This Twitter thread points out that the beef burger was less heavily salted.
Thanks for the comment and the followup comments by you and Michael, Ben. First, it’s really cool that Impossible was preferred to beef burgers in a blind test! Even if the test is not completely fair! Impossible has been around for a while, and obviously they would’ve been pretty excited to do a blind taste test earlier if they thought they could win, which is evidence that the product has improved somewhat over the years.
I want to quickly add an interesting tidbit I learned from food science practitioners[1] a while back:
Blind taste tests are not necessarily representative of “real” consumer food preferences.
By that, I mean I think most laymen who think about blind taste tests believe that there’s a Platonic taste attribute that’s captured well by blind taste tests (or captured except for some variance). So if Alice prefers A to B in a blind taste test, this means that Alice in some sense should like A more than B. And if she buys (at the same price) B instead of A at the supermarket, that means either she was tricked by good marketing, or she has idiosyncratic non-taste preferences that makes her prefer B to A (eg positive associations with eating B with family or something).
I think this is false. Blind taste tests are just pretty artificial, and they do not necessarily reflect real world conditions where people eat food. This difference is large enough to sometimes systematically bias results (hence the worry about differentially salted Impossible burgers and beef burgers).
People who regularly design taste tests usually know that there are easy ways that they can manipulate taste tests so people will prefer more X in a taste test, in ways that do not reflect more people wanting to buy more X in the real world. For example, I believe adding sugar regularly makes products more “tasty” in the sense of being more highly rated in a taste test. However, it is not in fact the case that adding high amounts of sugar automatically makes a product more commonly bought. This is generally understood as people in fact having genuinely different food preferences in taste test conditions than consumer real world decisions.
Concrete example: Pepsi consistently performs better than Coca-Cola in blind taste tests. Yet most consumers consistently buy more Coke than Pepsi. Many people (including many marketers, like the writers of the hyperlinks above) believe that this is strong evidence that Coke just has really good brand/marketing, and is able to sell an inferior product well to the masses.
Personally, I’m not so sure. My current best guess is that this discrepancy is best explained by consumer’s genuine drink preferences being different in blind taste tests from real-world use cases. As a concrete operationalization, if people make generic knock-offs of Pepsi and Coke with alternative branding, I would expect faux Pepsi (that taste just like Pepsi) to perform better in blind tastes than faux Coke (that tastes just like Coke), but for more people to buy faux Coke anyway.
For Impossible specifically, I remember doing a blind taste test in 2016 between Impossible beef and regular beef, and thinking that personally I liked [2]the Impossible burger more. But I also remember distinctly that the Impossible burger had a much stronger umami taste, which naively seems to me like exactly the type of thing that more taste testers will prefer in blind test conditions than real-world conditions.
This is a pretty long-winded comment, but I hope other people finds this interesting!
This is lore, so it might well be false. I heard this from practitioners who sounded pretty confident, and it made logical sense to me, but this is different from the claims being actually empirically correct. Before writing this comment, I was hoping to find an academic source on this topic I can quickly summarize, but I was unable to find it quickly. So unfortunately my reasoning transparency here is basically on the level of “trust me bro :/”
To be clear I think it’s unlikely for this conclusion to be shared by most taste testers, for the aforementioned reason that if Impossible believed this, they would’ve done public taste tests way before 2023.
Reversing start up advice
In the spirit of reversing advice you read, some places where I would give the opposite advice of this thread:
Be less ambitious
I don’t have a huge sample size here, but the founders I’ve spoken to since the “EA has a lot of money so you should be ambitious” era started often seem to be ambitious in unhelpful ways. Specifically: I think they often interpret this advice to mean something like “think about how you could hire as many people as possible” and then they waste a bunch of resources on some grandiose vision without having validated that a small-scale version of their solution works.
Founders who instead work by themselves or with one or two people to try to really deeply understand some need and make a product that solves that need seem way more successful to me.[1]
Think about failure
The “infinite beta” mentality seems quite important for founders to have. “I have a hypothesis, I will test it, if that fails I will pivot in this way” seems like a good frame, and I think it’s endorsed by standard start up advice (e.g. lean startup).
Of course, it’s perfectly coherent to be ambitious about finding a really good value proposition. It’s just that I worry that “be ambitious” primes people to be ambitious in unhelpful ways.
Two days after posting, SBF, who the thread lists as the prototypical example of someone who would never make a plan B, seems to have executed quite the plan B.
Longform’s missing mood
If your content is viewed by 100,000 people, making it more concise by one second saves an aggregate of one day across your audience. Respecting your audience means working hard to make your content shorter.
When the 80k podcast describes itself as “unusually in depth,” I feel like there’s a missing mood: maybe there’s no way to communicate the ideas more concisely, but this is something we should be sad about, not a point of pride.[1]
I’m unfairly picking on 80k, I’m not aware of any long-form content which has this mood that I claim is missing
This is a thoughtful post and a really good sentiment IMO!
As you touched on, I’m not sure 80k is a good negative example, to me it seems like a positive example of how to handle this?
In addition to a tight intro, 80k has a great highlight section, that to me, looks like someone smart tried to solve this exact problem, balancing many considerations.
This highlight section has good takeaways and is well organized with headers. I guess this is useful for 90% of people who only browse at the content for 1 minute.
Thanks for the push back! I agree that 80k cares more about the use of their listener’s time than most podcasters, although this is a low bar.
80k is operating under a lot of constraints, and I’m honestly not sure if they are actually doing anything incorrectly here. Notably, the fancy people who they get on the podcast probably aren’t willing to devote many hours to rephrasing things in the most concise way possible, which really constrains their options.
I do still feel like there is a missing mood though.
We also offer audio version of those highlights for all episodes on the ’80k After Hours’ feed: https://80000hours.org/after-hours-podcast/
To me, economy of words is what’s important, rather than overall length. Long can be wonderful, as long as the writer uses all those words well. Short can be wonderful, if the writer uses enough words to convey their complete thoughts.
See the updates here.
New Netflix show ~doubles search traffic for plant-based eating
h/t Lewis Bollard.
Closing comments on posts
If you are the author of a post tagged “personal blog” (which notably includes all new Bostrom-related posts) and you would like to prevent new comments on your post, please email forum@centerforeffectivealtruism.org and we can disable them.
We know that some posters find the prospect of dealing with commenters so aversive that they choose not to post at all; this seems worse to us than posting with comments turned off.
An EA Limerick
(Lacey told me this was not good enough to actually submit to the writing contest, so publishing it as a short form.)
Nice!
Working for/with people who are good at those skills seems like a pretty good bet to me.
E.g. “knowing how to attract people to work with you” – if person A has a manager who was really good at attracting people to work with them, and their manager is interested in mentoring, and person B is just learning how to attract people to work with them from scratch at their own startup, I would give very good odds that person A will learn faster.
Can you give some advice about the topic of attracting good people to work with you, or have any writeups you like?
We are banning stone and their alternate account for one month for messaging users and accusing others of being sock puppets, even after the moderation team asked them to stop. If you believe that someone has violated Forum norms such as creating sockpuppet accounts, please contact the moderators.
@lukeprog’s investigation into Cryonics and Molecular nanotechnology seems like it may have relevant lessons for the nascent attempts to build a mass movement around AI safety:
I have recently been wondering what my expected earnings would be if I started another company. I looked back at the old 80 K blog post arguing that there is some skill component to entrepreneurship, and noticed that, while serial entrepreneurs do have a significantly higher chance of a successful exit on their second venture, they raise their first rounds at substantially lower valuations. (Table 4 here.)
It feels so obvious to me that someone who’s started a successful company in the past will be more likely to start one in the future, and I continue to be baffled why the data don’t show this.[1]
I have heard some VCs say that they don’t fund serial entrepreneurs because humans only have enough energy to start one company; I always thought this was kind of dumb, but maybe there is some truth to it.
Although the data don’t not show this either. Companies founded by second time entrepreneurs might be less ambitious, raise money earlier, or have some other difference which is consistent with second time entrepreneurs being more successful
Wild guesses as someone that knows very little about this:
I wonder if it’s because people have sublinear returns on wealth, so their second company would be more mission driven and less optimized for making money. Also, there might be some selection bias in who needs to raise money vs being self funded.
But if I had to bet I would say that it’s mostly noise, and there’s not enough data to have a strong prior.
Person-affecting longtermism
This post points out that brain preservation (cryonics) is potentially quite cheap on a $/QALY basis because people who are reanimated will potentially live for a very long time with very high quality of life.
It seems reasonable to assume that reanimated people would funge against future persons, so I’m not sure if this is persuasive for those who don’t adopt person affecting views, but for those who do, it’s plausibly very cost-effective.
This is interesting because I don’t hear much about person affecting longtermist causes.
EA Three Comma Club
I’m interested in EA organizations that can plausibly be said to have improved the lives of over a billion individuals. Ones I’m currently aware of:
Shrimp Welfare Project – they provide this Guesstimate, which has a mean estimate of 1.2B shrimps per year affected by welfare improvements that they have pushed
Aquatic Life Institute – they provide this spreadsheet, though I agree with Bella that it’s not clear where some of the numbers are coming from.
Are there any others?
This is a nitpick, but somehow someone “being an individual” reads to me as implying a level of consciousness that seems a stretch for shrimps. But IDK, seems like a non-crazy choice under some worldviews.
That’s fair. I personally like that this forces people to come to terms with the fact that interventions targeted at small animals are way more scalable than those targeted at larger ones. People might decide on some moral weights which cancel out the scale of small animal work, but that’s a nontrivial philosophical assumption, and I like prompting people to think about whether it’s actually reasonable.
I think “animals that have more neurons or are more complex are morally more important” is not a “nontrivial philosophical assumption”.
It indeed strikes me as a quite trivial philosophical assumption the denial of which would I think seem absurd to almost anyone considering it. Maybe one can argue the effect is offset by the sheer number, but I think you will find almost no one on the planet who would argue that these things do not matter.
On the contrary, approximately everyone denies this! Approximately ~0% of Americans think that humans with more neurons than other humans have more moral value, for example.[1]
Citation needed, but I would be pretty surprised if this were false. Would love to hear contrary evidence though!
Come on, you know you are using a hilariously unrepresentative datapoint here. Within humans the variance of neuron count only explains a small fraction of variance in experience and also we have strong societal norms that push people’s map towards pretending differences like this don’t matter.
Unrepresentative of what? At least in my University ethics courses we spent way more time arguing about the rights of anencephalic children or human fetuses than insects. (And I would guess that neuron count explains a large fraction of the variance in experience between adult and fetal humans, for example.)
In any case: I think most people’s moral intuitions are terrible and you shouldn’t learn a ton from the fact that people disagree with you. But as a purely descriptive matter, there are plenty of people who disagree with you – so much so that reading their arguments is a standard part of bioethics 101 in the US.
It’s unrepresentative of the degree to which people believe that corollaries like neuron count and brain size and behavior complexity are an indicator of moral relevance across species (which is the question at hand here).
If the funders get a nontrivial portion of the impact for early-stage projects then I think the AWF (inc. its donors) is very plausible.
Yeah, I am not sure how to treat meta. In addition to funders, Charity Entrepreneurship probably gets substantial credit for SWP, etc.
Yeah definitely. I don’t want to claim that learning is impossible at a startup – clearly it’s possible – just that, all else equal, learning usually happens faster at existing companies.
Thanks! I’m not sure I fully understand your comment – are you implying that the skills you mention are easier to learn in a startup?
Unsurprisingly, I disagree with that view :)