Larks

• Larks 18 Jan 2023 2:56 UTC

  198 points

  115 ∶ 46

  on: Do­ing EA Better

  Overall this post seems like a grab-bag of not very closely connected suggestions. Many of them directly contradict each other. For example, you suggest that EA organizations should prefer to hire domain experts over EA-aligned individuals. And you also suggest that EA orgs should be run democratically. But if you hire a load of non-EAs and then you let them control the org… you don’t have an EA org any more. Similarly, you bemoan that people feel the need to use pseudonyms to express their opinions and a lack of diversity of political beliefs … and then criticize named individuals for being ‘worryingly close to racist, misogynistic, and even fascist ideas’ in essentially a classic example of the cancel culture that causes people to choose pseudonyms and causes the movement to be monolithically left wing.

  I think this is in fact a common feature of many of the proposals: they generally seek to reduce what is differentiated about EA. If we adopted all these proposals, I am not sure there would be anything very distinctive remaining. We would simply be a tiny and interchangeable part of the amorphous blob of left wing organizations.

  It is true this does not apply to all of the proposals. I agree that, for example, EAs should re-invent the wheel less and utilize domain expertise more. But I can’t say this post really caused me to update in their favour vs just randomly including some proposals I already agreed with. I think you would have been better off focusing on a smaller number of proposals and developing the arguments for them in more depth—and in particular considering counterarguments.

  What links here?

  • Buck's comment on Do­ing EA Better by ConcernedEAs (18 Jan 2023 16:49 UTC; 43 points)

Avoid­ing Mu­nich’s Mis­takes: Ad­vice for CEA and Lo­cal Groups

2021 AI Align­ment Liter­a­ture Re­view and Char­ity Comparison

2020 AI Align­ment Liter­a­ture Re­view and Char­ity Comparison

2019 AI Align­ment Liter­a­ture Re­view and Char­ity Comparison

• Larks 4 Aug 2023 2:57 UTC

  136 points

  59 ∶ 3

  on: Univer­sity EA Groups Need Fixing

  One said climate change, two said global health, and two said AI safety. Neither of the people who said AI safety had any background in AI. If after Arete, someone without background in AI decides that AI safety is the most important issue, then something likely has gone wrong (Note: prioritizing any non-mainstream cause area after Arete is epistemically shaky. By mainstream, I mean a cause area that someone would have a high prior on).

  This seems like a strange position to me. Do you think people have to have a background in climate science to decide that climate change is the most important problem, or development economics to decide that global poverty is the moral imperative of our time? Many people will not have a background relevant to any major problem; are they permitted to have any top priority?

  I think (apologies if I am mis-understanding you) you try to get around this by suggesting that ‘mainstream’ causes can have much higher priors and lower evidential burdens. But that just seems like deference to wider society, and the process by which mainstream causes became dominant does not seem very epistemically reliable to me.

Sam Alt­man fired from OpenAI

• Larks 18 Apr 2023 2:29 UTC

  133 points

  56 ∶ 4

  on: Up­dates to the Effec­tive Ven­tures US board

  Thanks for sharing this; I especially appreciate the transparency that she resigned because of strategic disagreements.

  I realize I am quite repetitive about this, but I really think EV/​CEA would benefit from being more transparent with the community, especially about simple issues like ‘who is currently in charge’. In this case I noticed the change on the website 18 days ago, and the actual handover may(?) have taken place prior to that point. My impression is that most normal organizations with public stakeholders announce leadership changes more or less immediately and I don’t understand why EV doesn’t.

Con­cerns with the Wel­lbe­ing of Fu­ture Gen­er­a­tions Bill

2018 AI Align­ment Liter­a­ture Re­view and Char­ity Comparison

• Larks 28 Dec 2021 19:12 UTC

  106 points

  0 ∶ 0

  on: Democratis­ing Risk—or how EA deals with critics

  The linked article seems to overstate the extent to which EAs support totalitarian policies. While it is true that EAs are generally left-wing and have more frequently proposed increases in the size & scope of government than reductions, Bostrom did commission an entire chapter of his book on the dangers of a global totalitarian government from one of the world’s leading libertarian/​anarchists, and longtermists have also often been supportive of things that tend to reduce central control, like charter cities, cryptocurrency and decentralised pandemic control.

  Indeed, I find it hard to square the article’s support for ending technological development with its opposition to global governance. Given the social, economic and military advantages that technological advancement brings, it seems hard to believe that the US, China, Russia etc. would all forgo scientific development, absent global coordination/​governance. It is precisely people’s skepticism about global government that makes them treat AI progress as inevitable, and hence seek other solutions.

• Larks 31 Mar 2022 17:19 UTC

  94 points

  0 ∶ 0

  in reply to: Jack Lewars’s comment on: I feel anx­ious that there is all this money around. Let’s talk about it

  it concerns me that entry level positions in EA are now being advertised at what would be CEO-level salaries at other nonprofits

  Suppose there was no existing nonprofit sector, or perhaps that everyone who worked there was an unpaid volunteer, so the only comparison was to the private sector. Do you think that the optimal level of compensation would differ significantly in this world?

  In general I’m skeptical that the existence of a poorly paid fairly dysfunctional group of organisations should inspire us to copy them, rather than the much larger group of very effective orgs who do practice competitive, merit based compensation.

• Larks 6 Jun 2020 3:52 UTC

  93 points

  1 ∶ 0

  in reply to: Robert_Wiblin’s comment on: Will protests lead to thou­sands of coro­n­avirus deaths?

  I think this is the wrong question.

  The point of lockdown is that for many people it is individually rational to break the lockdown—you can see your family, go to work, or have a small wedding ceremony with little risk and large benefits—but this imposes external costs on other people. As more and more people break lockdown, these costs get higher and higher, so we need a way to persuade people to stay inside—to make them consider not only the risks to themselves, but also the risks they are imposing on other people. We solve this with a combination of social stigma and legal sanctions.

  The issue is exactly the same with ideologies. To environmentalists, preventing climate change is more important than covid. To pro-life people, preventing over half a million innocent deaths every year is more important than covid. To animal rights activists, ending factory farming is more important than covid. To anti-lockdown activists, preventing mass business failure and a depression is more important than covid. But collectively we are all better off if everyone stops holding protests for now.

  The correct question is “is it good if I, and everyone else who thinks their reason is as good as I think this one is, breaks the lockdown?” Failure to consider this, as it appears most people have, is to grossly privilege this one cause over others and defect in this iterated prisoners dilemma—and the tragic consequence will be many deaths.

• Larks 9 Nov 2023 21:34 UTC

  89 points

  28 ∶ 1

  on: An­nounc­ing Athena—Women in AI Align­ment Research

  Thanks for sharing these studies explaining why you are doing this. Unfortunately, in general I am very skeptical of the sort of studies you are referencing. The researchers typically have a clear agenda—they know what conclusions they want to come to ahead of time, and what conclusions will most advantageous to their career—and the statistical rigour is often lacking, with small sample sizes, lack of pre-registration, p-hacking, and other issues. I took a closer look at the four sources you referenced to see if these issues applied.

  When more women participate in traditionally male-dominated fields like the sciences, the breadth of knowledge in that area usually grows, a surge in female involvement directly correlates with advancements in understanding[1]. [emphasis added]

  The link you provide here, to a 2014 article in National Geographic, has a lot of examples of cases where male researchers supposedly overlooked the needs of women (e.g. not adequately studying how women’s biology affects how drugs and seat belts should work, or the importance of cleaning houses), and suggests that increasing number of female scientists helped address this. But female scientists being better at understanding women seems less relevant to AI technical alignment work, because AIs are not female or male. Maybe it is useful for understanding what distinctly female values we want AIs to promote, but it doesn’t seem particularly relevant for things like Interpretability or most other current research agendas. The article also suggests that women are more communal and emotionally aware, vs men who are more agentic. But it doesn’t really make any claims about overall levels of understanding ‘directly correlating’ with female involvement, especially in more abstract, less biological fields, and the word ‘correlate’ literally does not appear in the text.

  Cox & Fisher (2008) found that women in a single-sex environment in a software engineering course reported higher levels of enjoyment, fairness, motivation, support, and comfort and allowed them to perform at a level that exceeded that of the all-male groups in the class [1].

  The first paper describes a n=7 study of a female group project, which apparently scored more highly than other group projects run by men. The study was not pre-registered, blinded or randomised, the researcher was an active participant, and there was no control. The author also obliquely references the need to avoid ″rigid marking schemes’ if these might reveal the all-female group performing worse, which suggests a bias to me.

  Kahveci (2008) explored a program for women in science, mathematics, and engineering and found that it helped marginalized women move towards legitimate participation in these fields and enhanced a sense of community and mutual engagement [2].

  The second paper describes a n=74 study of a women-in-science program, where the positive result is basically that the participants gave positive reviews to the program and said it made them more likely to do science. The study was not pre-registered, blinded or randomised, the researcher was an active participant, and there was no control. The only concrete example provided of a student switching major was from Biology to Exercise Physiology, which seems like a move away from core science.

  “It is not about men against women, but there is evidence to show through research that when you have more women in public decision-making, you get policies that benefit women, children and families in general. When women are in sufficient numbers in parliaments they promote women’s rights legislation, children’s rights and they tend to speak up more for the interests of communities, local communities, because of their close involvement in community life. [2]

  The link here goes to a web page with a quote from Oxfam. There are no links to the evidence or research that supposedly backs up the claim.

  Overall, my opinion of the linked research is it has very little scientific merit. They provide some interesting anecdotes, and the authors have some theories that someone else could test. But to the extent you are highlighting them because they are cruxes for your theory of change, they seem very weak. If your ‘Why We Are Doing This’ had been premised on ‘well some women just like sex-segregated programs, so proving this option will help with recruitment’ then I would have said fair enough. But if, as this post suggests, your theory of change is based on these sorts of dubious studies then that makes me significantly less optimistic about the project.

  What links here?

  • rachelAF's comment on An­nounc­ing Athena—Women in AI Align­ment Research by Claire Short (15 Nov 2023 3:30 UTC; 43 points)

Will protests lead to thou­sands of coro­n­avirus deaths?

• Larks 29 Dec 2021 14:00 UTC

  85 points

  0 ∶ 0

  in reply to: Guy Raveh’s comment on: Democratis­ing Risk—or how EA deals with critics

  Freedom of speach and freedom of research are important, and as long as someone doesn’t call to intentionally harm or discriminate against another, it’s important that we don’t condition funding on agreement with the funders’ views.

  This seems a very strange view. Surely an animal rights grantmaking organisation can discriminate in favour of applicants who also care about animals? Surely a Christian grantmaking organisation can condition its funding on agreement with the Bible? Surely a civil rights grantmaking organisation can decide to only donate to people who agree about civil rights?

• Larks 20 Jan 2023 3:14 UTC

  83 points

  27 ∶ 9

  in reply to: RobBensinger’s comment on: FLI FAQ on the re­jected grant pro­posal controversy

  FTX collapsed on November 8th; all the key facts were known by the 10th; CEA put out their statement on November 12th. This is a totally reasonable timeframe to respond. I would have hoped that this experience would make CEA sympathetic to a fellow EA org (with much less resources than CEA) experiencing a media crisis rather than being so quick to condemn.

  I’m also not convinced that a Head of Communications, working for an organization with a very restrictive media policy for employees, commenting on a matter of importance for that organization, can really be said to be operating in a personal capacity. Despite claims to the contrary, I think it’s pretty reasonable to interpret these as official CEA communications. Skill at a PR role is as much about what you do not say as what you do.

Public Fundrais­ing has Pos­i­tive Externalities

• Larks 16 Apr 2021 13:16 UTC

  77 points

  0 ∶ 0

  in reply to: Will Bradshaw’s comment on: Con­cerns with ACE’s Re­cent Behavior

  Having written a similar post in the past, it’s worth keeping in mind the amount of time they take to write is huge. Hypatia seems to have done a very good job expressing the facts in a way which communicates why they are so concerning while avoiding hyperbole. While giving organisations a chance to read a draft can be a good practice to reduce the risk of basic factual mistakes (and one I try to follow generally), it’s not obligatory. Note that we generally do not afford non-EA organisations this privilege, and indeed I would be surprised if ACE offered Connor the chance to review their public statement which pseudonymously condemned him. Doing so adds significantly to the time commitment and raises anonymity risks[1], especially if one is worried about retaliation from an organisation that has penalized people for political disagreements in the past.

  [1] As an example, here is something I very nearly messed up and only thought of at the last minute: you need to make a fresh copy of the google doc to share without the comments, or you will reveal the identity of your anonymous reviewers, even if you are personally happy to be known.

  What links here?

  • Jakub Stencel's comment on Con­cerns with ACE’s Re­cent Behavior by Hypatia (21 Apr 2021 19:23 UTC; 241 points)
  • What should the norms around pri­vacy and eval­u­a­tion in the EA com­mu­nity be? by NunoSempere (16 Jun 2021 17:31 UTC; 61 points)
  • Larks's comment on EA Fo­rum fea­ture sug­ges­tion thread by Aaron Gertler (21 Apr 2021 14:53 UTC; 57 points)
  • Max_Daniel's comment on Con­cerns with ACE’s Re­cent Behavior by Hypatia (16 Apr 2021 14:44 UTC; 8 points)

• Larks 1 Apr 2023 1:12 UTC

  75 points

  39 ∶ 2

  on: Cri­tiques of promi­nent AI safety labs: Red­wood Research

  About 10+ people (5 Constellation members) have mentioned that there is social pressure to defer or act a certain way when at Constellation.

  At least as written, this is so broad as to be effectively meaningless. All organisations exert social pressure on members to act in a certain way (e.g. to wipe down the exercise machines after use). Similarly, basically all employers require some degree of deference to management; typical practice is that management solicit feedback from workers but in turn compliance with instructions is mandatory.

  What you describe could be bad… or it could be totally typical. There’s no real way for the reader to judge based on what you’ve written.

  What links here?

  • Cri­tiques of promi­nent AI safety labs: Red­wood Research by Omega (31 Mar 2023 8:58 UTC; 339 points)
  • Cri­tiques of promi­nent AI safety labs: Red­wood Research by Omega. (LessWrong; 17 Apr 2023 18:20 UTC; 1 point)