Rubi J. Hudson

• Rubi J. Hudson 19 Jun 2024 2:49 UTC

  12 points

  10 ∶ 2

  in reply to: Austin’s comment on: Why so many “racists” at Man­i­fest?

  I also think of Manifest as something like, a scaled up house party, rather than an arbiter of who is good or notable in forecasting/​EA

  I’ve made a similar point in other comments, but this framing makes things worse. Then it’s not that Richard Hanania has relevant expertise in spite of his controversial statements, it’s that you think he’s a fun guy that you’d like to hang out with. Where people are willing to stomach unsavory co-attendees at a conference in their field, they’re more than happy to skip out on a scaled up house party with them.

• Rubi J. Hudson 18 Jun 2024 17:01 UTC

  13 points

  7 ∶ 0

  in reply to: Austin’s comment on: Why so many “racists” at Man­i­fest?

  I think my peers would factually right, at least directionally, in that attending with someone who has controversial views is evidence of favoring those views.

  As for whether society is a better place if we enforce the norm, I think there’s a couple relevant considerations. The first is the degree of abhorrence. To pull Godwin’s Law, I think that knowingly attending a conference with literal neo-Nazis should be seen as support for their views. The second is the degree of focus. Where it should be fine to attend an academic conference with subject matter experts talking about their work, choosing to attend a “fun forecasting-adjacent festival” where attendees are encouraged to pal around with each other is more deserving of judgment.

  I would also like to clarify, when I talk about peer judgments, I’m not making a point about how Manifest looks to those people. I’m making a point about how it would feel for them to attend. While I understand there are tradeoffs involved and you can’t make the event welcoming to arbitrary potential attendees, I would say by the time you’ve lost Peter Wildeford you’ve gone too far. I would also throw my hat in the ring as someone who works on prediction science and would be hesitant to attend the next Manifest if it had a similar invitee list.

• Rubi J. Hudson 18 Jun 2024 5:58 UTC

  93 points

  56 ∶ 8

  on: Why so many “racists” at Man­i­fest?

  > My plan was then to invite & highlight folks who could balance this out
  
  I think this is basically a misconception of how the social dynamics at play work. People aren’t worried about the relative number of “racists”, they’re worried about the absolute number. The primary concern is not that they will exposed to racism at the conference itself, but rather that attending a conference together will be taken as a signal of support for the racists, saying that they are welcome in the community.

  To pick Hanania as an example, since he has the most clearly documented history of racist statements, I have peers who would absolutely see me choosing to attend the same conference as him as a sign that I don’t think he’s too bad. And if I know that expectation and chose to go anyway, there would be additional merit to that reading.

  To an extent, the more that Manifest is focused on discussions of prediction, the more leeway there is to invite controversial speakers. You can say make a case for ignoring views that are not relevant to the topic at hand. But as Saul says in his other post “although manifest is nominally about prediction markets, it’s also about all the ideas that folks who like prediction markets are also into — betting, philosophy, mechanism design, writing, etc”. In other words, it’s about forming a broader intellectual community. And people are obviously going to be uncomfortable identifying with an intellectual community that includes people that they, and the broader world, consider to be racist.
  

• Rubi J. Hudson 15 Nov 2023 23:07 UTC

  16 points

  1 ∶ 0

  on: Hy­po­thet­i­cal grants that the Long-Term Fu­ture Fund nar­rowly rejected

  Does the LTFF ever counter-offer with an amount that would move the grant past the funding bar for cost-effectiveness? I would guess that some of these hypothetical applicants would accept a salary at 80% of what they applied for, and if the grants are already marginal then a 25% increase in cost-effectiveness could push them over the bar.

• Rubi J. Hudson 6 Oct 2023 17:40 UTC

  11 points

  4 ∶ 3

  on: Which Effec­tive Altru­ism groups are ad­dress­ing the is­sue of fer­til­ity rates?

  Pronatalist.org is not an EA group. It’s great that EA considerations have started entering the public consciousness and I would love if every charity was expected to answer “why is this the most effective thing you could be doing?”, but that doesn’t mean that any group claiming their mission is really important is part of EA. It’s very difficult to argue a rigorous case that promoting pronatalist sentiment is an effective use of money or time, and so far they haven’t.

  Rather than ask how we can build more (and better) groups, ask whether we should.

• Rubi J. Hudson 9 Sep 2023 4:47 UTC

  11 points

  0 ∶ 0

  on: A quick up­date from Nonlinear

  Was Ben Pace shown these screenshots before he published his post?

• Rubi J. Hudson 7 Sep 2023 17:29 UTC

  2 points

  5 ∶ 5

  in reply to: spencerg’s comment on: Shar­ing In­for­ma­tion About Nonlinear

  With regards to #2, I shared your concern, and I thought Habryka’s response didn’t justify that the cost of a brief delay was sufficient if there was a realistic chance of evidence being provided to contradict the main point of this post.

  However, upon reflection, I am skeptical that such evidence will be provided. Why did Nonlinear not provide at least some of the proof they claim to have, in order to justify time for a more comprehensive rebuttal? Or at least describe the form the proof will take? That should be possible, if they have specific evidence in mind. Also, a week seems like longer time than should be needed to provide such proof, which increases my suspicion that they’re playing for time. What does delaying for a week do that a 48h delay would not?

  Edit: Nonlinear has begun posting some evidence. I remain skeptical that the bulk of the evidence supports their side of the narrative, but I no longer find the lack of posting evidence as a reason for additional suspicion.

• Rubi J. Hudson 5 Jul 2023 3:39 UTC

  4 points

  2 ∶ 2

  on: Some thoughts on quadratic funding

  Thanks for writing this up, I find quadratic funding falls into a class of mechanisms that are too clever by half in a way that makes them very fragile to the modelling assumptions.

  Also, I love how it’s often posts with innocuous titles like ’Some thoughts on quadratic funding” that completely demolish an idea.

• Rubi J. Hudson 25 May 2023 14:53 UTC

  33 points

  15 ∶ 0

  in reply to: Geoffrey Miller’s comment on: If you find EA con­fer­ences emo­tion­ally difficult, you’re not alone

  “People saying things that are mildly offensive but not worth risking an argument by calling out, and get tiring after repeated exposure” is just obviously a type of comment that exists, and is what most people mean when they say microaggression. Your paper debunking it alternates between much stricter definitions and claiming an absence of evidence for something that very clearly is going to be extremely hard to measure rigorously.

• Rubi J. Hudson 10 Mar 2023 8:25 UTC

  2 points

  1 ∶ 0

  in reply to: Michael_Cohen’s comment on: Against EA-Com­mu­nity-Re­ceived-Wis­dom on Prac­ti­cal So­ciolog­i­cal Questions

  I’ll edit to comment to note that you dispute it, but I stand by the comment. The AI system trained is only as safe as the mentor, so the system is only safe if the mentor knows what is safe. By “restrict”, I meant for performance reasons, so that it’s feasible to train and deploy in new environments.

  Again, I like your work and would like to see more similar work from you and others. I am just disputing the way you summarized it in this post, because I think that portrayal makes its lack of splash in the alignment community a much stronger point against the community’s epistemics than it deserves.

• Rubi J. Hudson 9 Mar 2023 3:51 UTC

  52 points

  8 ∶ 2

  on: Against EA-Com­mu­nity-Re­ceived-Wis­dom on Prac­ti­cal So­ciolog­i­cal Questions

  How does Rationalist Community Attention/​Consensus compare? I’d like to mention a paper of mine published at the top AI theory conference which proves that when a certain parameter of a certain agent is set sufficiently high, the agent will not aim to kill everyone, while still achieving at least human-level intelligence. This follows from Corollary 14 and Corollary 6. I am quite sure most AI safety researchers would have confidently predicted no such theorems ever appearing in the academic literature. And yet there are no traces of any minds being blown. The associated Alignment Forum post only has 22 upvotes and one comment, and I bet you’ve never heard any of your EA friends discuss it. It hasn’t appeared, to my knowledge, in any AI safety syllabuses. People don’t seem to bother investigating or discussing whether their concerns with the proposal are surmountable. I’m reluctant to bring up this example since it has the air of a personal grievance, but I think the disinterest from the Rationality Community is erroneous enough that it calls for an autopsy. (To be clear, I’m not saying everyone should be hailing this as an answer to AI existential risk, only that it should definitely be of significant interest.)

  I’m someone who has read your work (this paper and FGOIL, the latter of which I have included in a syllabus), and who would like to see more work in similar vein, as well as more formalism in AI safety. I say this to establish my bona fides, the way you established your AI safety bona fides.

  I don’t think this paper is mind-blowing, and I would call it representative of one of the ways in which tailoring theoretical work for the peer-review process can go wrong. In particular, you don’t show that “when a certain parameter of a certain agent is set sufficiently high, the agent will not aim to kill everyone”, you show something more like “when you can design and implement an agent that acts and updates its beliefs in a certain way and can restrict the initial beliefs to a set containing the desired ones and incorporate a human into the process who has access to the ground truth of the universe, then you can set a parameter high enough that the agent will not aim to kill everyone” [edit: Michael disputes this last point, see his comment below and my response], which is not at all the same thing. The standard academic failure mode is to make a number of assumptions for tractability that severely lower the relevance of the results (and the more pernicious failure mode is to hide those assumptions).

  You’d be right if you said that most AI safety people did not read the paper and come to that conclusion themselves, and even if you said that most weren’t even aware of it. Very little of the community has the relevant background for it (and I would like to see a shift in that direction), especially the newcomers that are the targets of syllabi. All that said, I’m confident that you got enough qualified eyes on it that if you had shown what you said in your summary, it would have had an impact similar in scale to what you think is appropriate.

  This comment is somewhat of a digression from the main post, but I am concerned that if someone took your comments about the paper at face value, they would come away with an overly negative perception of how the AI safety community engages with academic work.

Mechanism De­sign for AI Safety—Agenda Creation Retreat

• Rubi J. Hudson 9 Feb 2023 1:51 UTC

  10 points

  3 ∶ 1

  in reply to: zchuang’s comment on: Book Post: The Good It Promises, the Harm It Does: Crit­i­cal Es­says on Effec­tive Altruism

  An EA steelman example of similar points of thinking are EAs who are incredibly anti-working for OpenAI or Deepmind at all because it safety washes and pushes capabilities anyways. The criticism here is the way EA views problems means EA will only go towards solution that are piecemeal rather than transformative. A lot of Marxists felt similarly to welfare reform in that it quelled the political will for “transformative” change to capitalism.

  For instance they would say a lot of companies are pursuing RLHF in AI Safety not because it’s the correct way to go but because it’s the easiest low hanging fruit (even if it produces deceptive alignment).

  I want to address this point not to argue against the animal activist’s point, but rather because it is a bad analogy for that point. The argument against working for safety teams at capabilities orgs or RLHF is not that they reduce x-risk to an “acceptable” level, causing orgs to give up on further reductions, but rather than they don’t reduce x-risk.

• Rubi J. Hudson 30 Jan 2023 20:59 UTC

  16 points

  3 ∶ 0

  on: We’re no longer “paus­ing most new longter­mist fund­ing com­mit­ments”

  Thanks for the communication, and especially giving percentages. Would you be able to either break it down by grants for individuals vs. grants to organizations, or note if the two groups were affected equally? While I appreciate knowing how high the bar has risen in general, I would be particularly interested in how high it has risen for the kinds of applications I might submit in the future.

• Rubi J. Hudson 17 Jan 2023 12:52 UTC

  3 points

  0 ∶ 0

  on: Some in­tu­itions about fel­low­ship programs

  What are your intuitions regarding length? What’s the minimum time needed for a fellowship to be impactful, and at what length does it hit diminishing returns?

Mechanism De­sign for AI Safety—Read­ing Group Curriculum

• Rubi J. Hudson 12 Oct 2022 4:25 UTC

  5 points

  0 ∶ 0

  on: Which AI Safety Org to Join?

  Seems worth asking in interviews “I’m concerned about advancing capabilities and shortening timelines, what actions is your organization taking to prevent that”, with the caveat that you will be BSed.

  Bonus: You can turn down roles explicitly because they’re doing capabilities work, which if it becomes a pattern may incentivize them to change their plan.

• Rubi J. Hudson 31 Aug 2022 20:02 UTC

  3 points

  0 ∶ 0

  on: Red-team­ing con­test: de­mo­graph­ics and power struc­tures in EA

  This comment is object-level, perhaps nitpicky, and I quite like your post on a high level.

  Saving a life via, say, malaria nets gets you two benefits:

  1. The person saved doesn’t die, meeting their preference for continuing to exist

  2. The externalities of that person continuing to live, such as foregone grief by their family and community.

  I don’t think it’s too controversial to say that the majority of the benefit from saving a life goes to the person whose life is saved, rather than the people who would be sad that they died. But the IDinsights survey only provides information about the latter.

  Consider what would happen if beneficiary surveys find the opposite conclusion in future communities, that certain beneficiaries did not care at all about the death of children under the age of 9. It would be ridiculous and immoral to defer to that decision, and not provide any life-saving aid to those children. The reason for this is that the community being surveyed is not the primary beneficiary of aid to their children, their children are, so their preferences make up a small fraction of the aid’s value. But this also goes the other way, if the surveyed community overweights the lives of their children, that isn’t a reason for major deferral. Especially if stated preferences contradict revealed preferences, as they often do.

An­nounc­ing: Mechanism De­sign for AI Safety—Read­ing Group

• Rubi J. Hudson 26 Jul 2022 18:16 UTC

  13 points

  0 ∶ 0

  in reply to: Peter S. Park’s comment on: What ‘equil­ibrium shifts’ would you like to see in EA?

  There are lots of advantages to being based in the Bay Area. It seems both easier and higher upside to solve the Berkeley real estate issue that to coordinate a move away from the Bay Area.